How to reproduce channel is busy throwing EROChannelBusy

parsecstrap · July 27, 2017, 2:29pm

I am accessing my remobjects sdk services (TROIndyHTTPServer + TROIndyHTTPChannel + TROXmlRpcMessage) with this code:

function CreateTask(const AIndex: Integer): ITask;
begin
  Result := TTask.Create(procedure()
  var
    RequestBody: TStringStream;
    BodyContent: string;
    RESTClient: TRESTClient;
    RESTRequest: TRESTRequest;
    RESTResponse: TRESTResponse;
    HTTPBasicAuthenticator: THTTPBasicAuthenticator;
  begin
    RESTClient := TRESTClient.Create(nil);
    RESTRequest := TRESTRequest.Create(nil);
    RESTResponse := TRESTResponse.Create(nil);
    HTTPBasicAuthenticator := THTTPBasicAuthenticator.Create('User', 'Password');

    RESTClient.Authenticator := HTTPBasicAuthenticator;
    RESTClient.BaseURL := 'http://server:1234';
    RESTClient.ContentType := 'application/xml';

    RESTRequest.Client := RESTClient;
    RESTRequest.Timeout := 300000;
    RESTRequest.Response := RESTResponse;
    RESTRequest.Method := rmPOST;
    RESTRequest.Resource := 'XmlRpc';
    RESTRequest.Accept := 'application/xml, text/plain; q=0.9, text/html;q=0.8,';

    BodyContent := 'some valid xmlrpc call';
    RESTRequest.ClearBody;
    RESTRequest.AddBody(BodyContent, ctAPPLICATION_XML);
    RESTRequest.Execute;

    HTTPBasicAuthenticator.Free;
    RESTClient.Free;
    RESTRequest.Free;
    RESTResponse.Free;

   end
  );
end;

Everything works fine for me!

But I have a client and he is accessing the services from a C# application und is getting the “channel is busy” error.

I have tried to reproduce it with the delphi code without using the ServerAccess files, but I couldn’t.

How could he get the error and how could I reproduce this error in my code above?

Is it possible to get the error using for examle the tool like postman?

Thank you!

EvgenyK · July 28, 2017, 6:19am

“channel is busy” exception in ROD usually is appeared when the same channel is used simultaneously.

simple testcase:
try to call server method from several background threads on the same channel in the same time.

parsecstrap · July 28, 2017, 7:57am

I have tried to do it within the delphi code (see my first post). Within the function
CreateTask(const AIndex: Integer): ITask;
I am creating multiple threads and then start them as follows:

  Tasks[0] := CreateTask(0);
  Tasks[1] := CreateTask(1);
  Tasks[2] := CreateTask(2);
  Tasks[3] := CreateTask(3);
  Tasks[4] := CreateTask(4);
  Tasks[5] := CreateTask(5);

  for I := Low(Tasks) to High(Tasks) do
    Tasks[I].Start;

Is it a call on the same channel in the same time?!

I am also trying to reproduce the error and use Webserver Stress Tool https://www.de.paessler.com/tools/webstress and send hundreds of requests to the server. But I am not getting the error.

Could you please create a sample code how can I call a server method from several background threads on the same channel in the same time?

Thank you!

EvgenyK · July 28, 2017, 8:08am

I don’t see that you are using TROIndyHTTPChannel in your CreateTask.
EROChannelBusy is raised only in RO specific channels.

testcase:
create a new RO VCL standalone project with default settings.
server-side, in GetServerTime set delay via sleep(5000) and then return Now.
client-side: call this method every 0.5-1 seconds via timer as (RORemoteService as INewService).GetServerTime.
all calls will be used ROChannel so at second call, EROChannelBusy will be raised.

parsecstrap · July 28, 2017, 8:11am

I am accessing the RO Services from a REST Client, so sending http reqeust through JSON/XML Message.
My client application doesn’t know anything about the remobjects… Do you know what I mean?

EvgenyK · July 28, 2017, 8:17am

probably I misunderstood you.

I can help you only with RO code and I can’t with 3rd party client that uses unknown libraries.
I can suggest to address to code of that 3rd party libraries and check why “channel is busy” was called inside it

parsecstrap · July 28, 2017, 8:20am

Yes, I can do this next week, but before I wanted to know why this error is happening. I have tried to send multiple XmlRpc HTTP requests to the RO Server but everything was fine and I didn’t get any “channel is busy” error…

But thank you for your answers!

parsecstrap · August 10, 2017, 6:30am

My client access in C# the RO services as follows and is getting the error “channel is busy”. He is sending multiple requests at the same time through the following code:

var client = new HttpClient()
….
response = await client.SendAsync(request);

The request is going to http://servername:1234/XmlRpc
The body content is a valid XML-RPC request

I am not a C# expert. Could you please verify if the error “channel is busy” is not the problem of the RO servers, but the problem of the C# code above?

Could you advice a solution in C# how the RO services could be accessed?

Thank you very much for your help!

antonk · August 10, 2017, 8:38am

Hello

The C# code provided doesn’t look like it uses any of RO SDK code. So it is not possible to say what is wrong there and why it fails, especially without error message and/or error stacktrace.

You need to either use a client-side RO SDK components for .NET or to use HttpAPI https://talk.remobjects.com/t/new-ro-sdk-feature-http-api/12993

Regards

parsecstrap · August 10, 2017, 8:50am

No, the client doesn’t use it, it accesses the RO webservices through the XmlRpc per HTTP request… internally it doesn’t know anything about the SDK. But he is getting the channel is busys error… and I don’t undesrtand why!?

The client is independent from SDK, so it can’t use any RemObject Framework.

antonk · August 10, 2017, 11:26am

What is the stacktrace and full exception message of these errors?

parsecstrap · August 10, 2017, 11:54am

I can’t give this information because my customer doesn’t give it to me The only thing which he gives -> the channel is busy

parsecstrap · December 8, 2017, 7:08am

Hello! That’s me again!

Sorry, but the problem with busy channel still exists and I don’t have a clue how to solve this! I need urgent your help!

I’ll try to describe how the busy channel problem occurs:

Client (some 3d party application written not by me and wich doesn’t know anything about RemObjects): it is a C# application which sends to the Server XML-RPC requests throuh HTTP and as I have just said it doesn’t use any RO components.

Server (written by me): a service based on Remoting SDK and uses TROClassFactory. This service executes SQL queries and some routines need a couple of minutes to be executed.

The problem is that somhow the above mentioned C# application is able to send two and more XML-RPC requests so they are coming REALLY AT THE SAME TIME on the server and produces the error busy channel because the service execute the SQL query routine.

In order to find out when the requests are coming in, I have logging in the event TRORemoteDataModule.OnActivate.
I have also saved the ClientID and it is in each XML-RPC request different!

I did also load tests with postman, soapui (https://www.soapui.org/) and could not reproduce the error.

I have also tried # TROSynchronizedSingletonClassFactory, but then nobody can access or should wait the service until it has finished the executing something.

Using the # TROPooledClassFactory produces also the busy channel error.

My questions are:

Why the error busy channel occurs in case of different ClientIDs and independent HTTP requests?
How does ROChannel work exactly? E.g. in my case is it busy only during a long-running SQL query is being executed? Or is it just responsible for the bypassing of a HTTP request and after that the channel is still available for other requests? It seems actually it is busy
Should I maybe configure the Factories, TRORemoteDataModule on some other way?!

I hope, you could help us! We have a big issue in production with this busy channel error!

Thank you in advance!

antonk · December 8, 2017, 9:28am

Hello

Let’s start with the most obvious things to check:
a) Is it possible to get the exact error message?
b) You mentioned that client code uses a HttpClient instance to access the server. When several requests are sent to the server at the same time - are they sent using DIFFERENT HttpClient instances or the same instance is used to send these requests?
c) On the server side could you wrap the entire service method being called into a try/except clause and add there logging of any exceptions occurred (just don’t forget to re-raise these exceptions again)?

This will allow to narrow down the issue search area.

Thanks in advance

EvgenyK · December 8, 2017, 12:14pm

are you sure that Channel is busy. Try again later delphi exception is raised on server-side in case C# application sends several requests?

usual channel in non-async mode can send & receive only one request. this operation includes sending request to server, executing of server method and receiving server response.

you can post your requests in several threads, in this case, personal channel is required for each thread, or send requests in async mode. in this case, next request will be send to server after previous one was finished. See _AsyncEx interfaces in _Intf (ofc, _AsyncEx is ROD feature ).
another way - use super channels. they are also available in Remoting SDK.

Edit: can you try to replace IndyHttp with Synapse or WinHttp plain http servers?
these components are interchangeable

parsecstrap · December 11, 2017, 11:21am

The problem was the communication between multiple Remoting SDK Services. They are accssing each other thoruh a automatic generated “server access” which returns a singlton instance of a “server access” which has of course one channel.