Delphi Tokyo. All patches. RO SDK Latest - 1 (havent had the chance to update to the latest one)
I made a sample project which I sent to the support email.
It is a modified version of the Supertcp chat using synapse and Olympia.
After compiling server and client try the following scenarios, all of them from the Delphi IDE so you can see the behaviour.
Scenario 1:
Turn on Server. Set the proper connection parameters for Olympia and for the server port itself. Start the server.
Close the server by clicking on the [X] (aka a controlled shutdown).
Server will shutdown properly. All good.
Scenario 2:
Turn on Server. Set the proper connection parameters for Olympia and for the server port itself. Start the server.
Run client (you can just run the executable for the client). Enter the connection string to the server.
You will see at least one timeout (probably two) and then a connection. The server received only one attempt and it was a successful connection.
First question, if you experienced the issue, why are we getting multiple timeouts on the client side? (i think im missing something, cant see what it is) if it was successful from the get go?
Log off the client. Close the client on the [x]. Controlled shutdown.
Close the server via a controlled shutdown. All good.
Scenario 3:
Turn on Server. Set the proper connection parameters for Olympia and for the server port itself. Start the server.
Run client (you can just run the executable for the client). Enter the connection string to the server.
You will see at least one timeout (probably two) timeouts and then the connection succeded.
Close the server via a controlled shutdown, while there is a client connected. Server will spin forever and never shutdown.
This scenario is happening in our production servers, it executes all the shutdown process but somewhere it gets stuck and we end up with a zombie process.
Scenario 4:
x. Comment the Exit statement on the "ServerClientConnected " method. The objective of the following lines is to be able to recover server Event registrations when it is turned off and then turned back on before session expires. It will register the event callbacks assuming they have existing sessions. This will work, but then if clients login and the server is shutdown on a controlled way, it will end up on a loop. This loop will disappear if the code that adds the sessions to the event repository is removed.
Follow the same steps as Scenario 3 to reproduce after done so.
We spent days on the real case looking for possible causes and ended up with that sample that seems to reproduce it (we use injection via Spring4Delphi so it makes it harder). Our main concern is that without the ability for the server to automatically register events our product can not be “certified” as production ready. We cant pass the responsibility of recovering from a controlled or uncontrolled service shutdown to the client if the session has not expired. It is assume that once a client “registers” to a service they will be able to fully recover their connection and settings if a server is stopped for a small period of time for a maintenance, replacement, redirection etc.
Also to make it more interesting, we use local channel and server on all our services. The reasoning is that they are intended to use themselves e.g. they will login, but even the login server will log in using itself so it can be tracked down by other tools. The notification server will subscribe itself to notifications so it can use the notification methods internally too.
just set ROChannel.SynchronizeEvents := True. Timeout error will be gone
Scenario 3:
add ROChannel.Active := False; into TSuperTCPChannelChat_ClientMainForm.ShutdownServer.
procedure TSuperTCPChannelChat_ClientMainForm.ShutdownServer;
begin
edOutput.Lines.Add('Server has been shutdown!');
ROChannel.Active := False;
FConnected:=False;
end;
The synchronizeEvents addressed the issue of the multiple timeout. Thank you. Will check if we process the events on a background thread or if we can do it on the main thread.
Regarding the client, in reality the shutdownserver code is just to illustrate that the server is shutting down. In reality clients will not be notified if a service is going down because it is not always a controlled shutdown, it could be that the machine crash, it was killed (the process), there was a network interruption, it was forcely restarted or it simply got stuck. If you run the server from the Delphi IDE and you just close the server while there is a client connected it will hang there forever. If you run it form the outside the IDE you will see it remaining in memory in a zombie status.
Also if I follow your suggestion to close the client connection, then it will not reconnect once the server comes back on, unless i tell the client ROChannel.Active := True, but how do i know when to do that? the idea was that once the server comes back, the client will simply sense it and reconnect. (the entire purpose of the reconnection mechanism)
Interesting enough, if I replace the components to use Indy instead of synapse, the scenario 3 works and even if there is a client connected and the program is closed, it will shutdown properly, no zombies.
So indeed there is something going on. We cant switch that simply to Indy so it is only good for testing for now.
Scenario 4 is hanging after the replacement and im trying to figure out why.
Scenario 4 after replacing components to use Indy it was still hanging on the moment it was attempting to log the reconnection on the memo component. (Probably some sort of threading issue) If I removed that logging part it will work, so it seems to point out an issue with Synapse under those conditions.
yep, this is known, standard Delphi GlobalNameSpace object may lock components at destroying …
as a workaround, create a server manually via global variable:
try to use solution with Log/WMLog from the MegaDemo server:
const
WM_LOG_MESSAGE = WM_APP + 1;
..
TMegaDemoServerMainForm = class(TForm)
..
procedure WMLog(var Message: TMessage); message WM_LOG_MESSAGE;
procedure Log(const someText: string);
..
procedure TMegaDemoServerMainForm.Log(const someText: string);
var
p: pChar;
begin
if Application.Terminated then Exit;
if not cbVerbose.Checked then Exit;
GetMem(p, (Length(someText) + 1)*SizeOf(Char));
Move(someText[1], p^, (Length(someText)+ 1)*SizeOf(Char));
PostMessage(Handle, WM_LOG_MESSAGE, 0, integer(p));
{ Access to the VCL may only happen from within the main thread. To allow
Log to be called from within the Service implementattion, we must ensure
it's threadsafe.
So instread of just addint the log message to the Memo, we'l send a
PostMessage to the window, which wil then later be handled within the
main thread.
As a side benefit, the secution of the Log doe snot need to wait for this
logging to happen (as usage of, for example, Synchronize would require),
which will in turn make the server more respinsible for a simultaneous
calls. }
end;
procedure TMegaDemoServerMainForm.WMLog(var Message: TMessage);
var
p: pChar;
begin
try
p := pChar(Message.LParam);
Memo.Lines.Add(p);
Freemem(p);
except
on E: Exception do
Memo.Lines.Add(E.Classname + ': ' + E.Message);
end;
end;