private void NativeDataSocketSendAsMuchAsPossible(Byte[] buffer, Int32 offset, Int32 size)
{
this.StartTimeoutTimer();
try
{
this._sslStream.Write(buffer, offset, size);
}
catch (IOException)
{
throw new SocketException();
}
catch (Exception)
{
// Due to a bug in the Mono Ssl code (namely in the MobileAuthenticatedStream methods)
// it can raise AggregateException
// Unfortunately we cannot here properly catch AggregateException here, so we have to catch everything
// and to re-raise a more expected exception
throw new SocketException();
}
finally
{
this.StopTimeoutTimer();
}
}
I have an application which sends a large amount of data which systematically generates this exception.
See the callstack below:
Exception in collectAndUploadData(testapb)
Date/time: 2023-04-11 10:21:28
Computer name: W10DEVELFV2
User name: admin
Operating system: Microsoft Windows NT 6.2.9200.0
System up time: -22.08:25:21.9060000
Program up time: 00:02:07.0485021 (start time 2023-04-11 10:19:21)
Physical memory: 37836.1/ 65535.5MB
Process: “E:\p4\main\Source.NET\Cloud\src\CloudCollect\bin\Debug\CloudCollectFW.exe” testapb testapp 10002
Version: CloudCollectFW, Version=23.2.1.3, Culture=neutral, PublicKeyToken=null v23.2.1.3
.NET runtime: v4.0.30319
Path: E:\p4\main\Source.NET\Cloud\src\CloudCollect\bin\Debug
Exception: The operation completed successfully [Source:RemObjects.SDK]
at RemObjects.SDK.SslConnection.NativeDataSocketSendAsMuchAsPossible(Byte[] buffer, Int32 offset, Int32 size)
at RemObjects.SDK.SslConnection.DataSocketSendAsMuchAsPossible(Byte[] buffer, Int32 offset, Int32 size)
at RemObjects.SDK.Connection.DataSocketSend(Byte[] buffer, Int32 offset, Int32 size)
at RemObjects.SDK.Connection.Send(Byte[] buffer, Int32 offset, Int32 size)
at RemObjects.SDK.Http.HttpOutgoingRequestResponse.WriteBodyToConnection(Connection connection)
at RemObjects.SDK.Http.HttpClient.TryDispatch(HttpClientRequest request)
at RemObjects.SDK.IpHttpClientChannel.IntDispatch(Stream request, IMessage response)
at RemObjects.SDK.ClientChannel.Dispatch(IMessage message)
at OffCloudServer.SyncServerService_Proxy.feedSyncTableData(String& jsonSyncResponse, Binary data, String fragmentStartKey, Boolean bDestinationFragmentHasRecords, Boolean bConsiderFragmentPartial, SyncHashEntry& hash, String& errormsg) in E:\p4\main\Source.NET\base\OffCloudServerClient\RemoteServer_Intf.cs:line 3158
at CloudCollect.UploadCloudData.uploadListToCloudCollectServer[T](CloudServerClient syncserverclient, Type elemType, IEnumerable`1 list) in E:\p4\main\Source.NET\Cloud\src\CloudCollect\UploadCloudData.cs:line 288
at CloudCollect.UploadCloudData.collectAndUploadData(String apb, String appname, String appversion) in E:\p4\main\Source.NET\Cloud\src\CloudCollect\UploadCloudData.cs:line 386
FYI: This is the IROService function that fails
bool feedSyncTableData(ref string jsonSyncResponse, RemObjects.SDK.Types.Binary data, string fragmentStartKey, bool bDestinationFragmentHasRecords, bool bConsiderFragmentPartial, out SyncHashEntry hash, out string errormsg);
We set these properties
if (message is RemObjects.SDK.BinMessage)
{
((RemObjects.SDK.BinMessage)message).MaxMessageSize = Int32.MaxValue;
((RemObjects.SDK.BinMessage)message).MaxDecompressedMessageSize = Int32.MaxValue;
((RemObjects.SDK.BinMessage)message).UseCompression = true; // FVC: Note: By default true
}
FYI: We also set
if (this.clientChannel is RemObjects.SDK.IpHttpClientChannel)
{
((RemObjects.SDK.IpHttpClientChannel)this.clientChannel).Timeout = 5 * 60; // FVC: To check if this works
}
but this is probably not relevant
FYI: I don’t know if it is related but this was another error we received. The sending seems to work but the decompressing fails
Exception: Invalid BinMessage signature found
Message size limit was reached while trying to decompress the message: 5242880 [Source:RemObjects.SDK]
bij RemObjects.SDK.BinHeader.ReadFromStream(Stream stream)
bij RemObjects.SDK.BinMessage.InternalReadFromStream(Stream stream)
bij RemObjects.SDK.Message.ReadFromStream(Stream stream)
bij RemObjects.SDK.IpHttpClientChannel.IntDispatch(Stream request, IMessage response)
bij RemObjects.SDK.ClientChannel.Dispatch(IMessage message)
bij OffCloudServer.SyncServerService_Proxy.feedSyncTableData(String& jsonSyncResponse, Binary data, String fragmentStartKey, Boolean bDestinationFragmentHasRecords, Boolean bConsiderFragmentPartial, SyncHashEntry& hash, String& errormsg)
bij CloudCollect.UploadCloudData.uploadListToCloudCollectServer[T](CloudServerClient syncserverclient, Type elemType, IEnumerable`1 list)
bij CloudCollect.UploadCloudData.collectAndUploadData(String apb, String appname, String appversion)
Is there some server maxmessagesize param we shoud set?
OK, I added this parameter. I don’t know how to reproduce this last issue so I cannot tell if it is fixed.
FYI: I set this value now for all my RO connections, is there any drawback to the option? It is not very clear why this is an option.
The documentation on BinMessage would even let me believe that maximizing the message sizes should prevent the need for this enforce parameter
FYI 2: I still think that the documentation lets me believe that maximizing the message sizes should prevent the need for this enforce parameter to be false
It looks like the EnforceMaxMessageSize solution does not work.
I have reduced the tabledata (to 100000 records) and I still get this (for ~50MB)
Exception: Invalid BinMessage signature found
Message size limit was reached while trying to decompress the message: 5242880 [Source:RemObjects.SDK]
at RemObjects.SDK.BinHeader.ReadFromStream(Stream stream)
at RemObjects.SDK.BinMessage.InternalReadFromStream(Stream stream)
at RemObjects.SDK.Message.ReadFromStream(Stream stream)
at RemObjects.SDK.IpHttpClientChannel.IntDispatch(Stream request, IMessage response)
at RemObjects.SDK.ClientChannel.Dispatch(IMessage message)
at OffCloudServer.SyncServerService_Proxy.feedSyncTableData(String& jsonSyncResponse, Binary data, String fragmentStartKey, Boolean bDestinationFragmentHasRecords, Boolean bConsiderFragmentPartial, SyncHashEntry& hash, String& errormsg) in E:\p4\main\Source.NET\base\OffCloudServerClient\RemoteServer_Intf.cs:line 3158
at CloudCollect.UploadCloudData.uploadListToCloudCollectServer[T](CloudServerClient syncserverclient, Type elemType, IEnumerable`1 list) in E:\p4\main\Source.NET\Cloud\src\CloudCollect\UploadCloudData.cs:line 295
at CloudCollect.UploadCloudData.collectAndUploadData(String apb, String appname, String appversion) in E:\p4\main\Source.NET\Cloud\src\CloudCollect\UploadCloudData.cs:line 401
public void ReadFromCompressedStream(Stream stream, Int32 maxSize, Boolean enforceMaxSize)
{
...
if (enforceMaxSize && (size > maxSize))
{
throw new SerializerException("Message size limit was reached while trying to decompress the message: " + maxSize);
}
I have checked the client code and all settings are as I stated.
However decompression occurs on the server side (i presume). Are you sure I don’t need to adjust server code?
It was unclear you were talking about server code. I don’t (explicitely) use a BinMessage in my server code.
How can I set this?
I notice a ServerMessage property in my INetworkServer object (serverROHTTPs_.NetworkServer.ServerMessage)
Is it sufficient if I set the EnforceMaxMessageSize there (and on the client for large client downloads)?
FYI: Is there a way to know the exception I receive is a server exception?
FYI2: Even for 5MB data I get the error without the enforce. How come the default configuration is so limited? What is the use/advantage of enforce=true?
BinMessage message = new BinMessage();
// Set properties here
message.EnforceMaxMessageSize = false;
server.NetworkServer.RegisterServerMessage(message);
Some channels like super tcp also have limitation for package size.
I suggest to review error messages and detect in RO/DA code what settings cause such errors.
Imagine that your server is a chat server and you should protect it from big inappropriate requests.
these limitations will reject these big requests on channel or message level w/o processing it
note: these limitations were added initially to Remoting SDK in 2004
