One of our customers has reported that they are unable to use our client discover RemObjects servers and as we’ve been unable to reproduce the issue they have provided us with actual machine having the issue.
All products involved (clients and servers) are using RemObjects SDK version 9.0.97.1245 and have been built under Delphi 10.1 Berlin under Windows 10. They are variously running on uo-to-date Windows 10 Enterprise and Windows 10 Professional.
The non-functional client is a Chinese edition of Windows, all others are the EN-GB edition. At the moment this is the only obvious difference between the setups.
On startup, the client attempts to discover available servers. With the same executable, every machine except for the one provided by our customer receives the expected response from a number of servers. The customer’s machine receives an error message indicating that the message could not be read due to an invalid stream length.
I’ve tracked the error down to the “discovery” message the client sends on startup. It is incorrectly formatted on the customer’s machine. Below are two Wireshark traces. The first is the transmitted request, the second is the response.
0000 ff ff ff ff ff ff e4 54 e8 a3 d4 25 08 00 45 00 .......T...%..E.
0010 00 77 46 9b 00 00 80 11 29 37 0a 0c c0 98 ff ff .wF.....)7......
0020 ff ff fb bd 1f a3 00 63 23 80 7b 41 42 41 44 45 .......c#.{ABADE
0030 34 42 32 2d 42 38 46 45 2d 34 33 42 45 2d 39 39 4B2-B8FE-43BE-99
0040 45 45 2d 36 42 34 30 34 37 32 38 33 44 38 39 7d EE-6B4047283D89}
0050 52 4f 31 30 37 00 00 00 00 00 00 00 14 3f 3f 3f RO107........???
0060 4d 3f 3f 49 1f 3f 0a 00 00 00 54 53 4d 53 65 72 M??I.?....TSMSer
0070 76 69 63 65 0d 00 00 00 54 53 4d 47 65 74 53 75 vice....TSMGetSu
0080 6d 6d 61 72 79 mmary
0000 e4 54 e8 a3 d4 25 4c cc 6a 2c 91 fb 08 00 45 00 .T...%L.j,....E.
0010 00 a7 14 16 00 00 80 11 90 fd 0a 0c c0 82 0a 0c ................
0020 c0 98 1f a3 fb bd 00 93 50 87 7b 41 42 41 44 45 ........P.{ABADE
0030 34 42 32 2d 42 38 46 45 2d 34 33 42 45 2d 39 39 4B2-B8FE-43BE-99
0040 45 45 2d 36 42 34 30 34 37 32 38 33 44 38 39 7d EE-6B4047283D89}
0050 52 4f 31 30 37 00 01 00 00 00 00 00 14 3f 3f 3f RO107........???
0060 4d 3f 3f 49 1f 3f 0a 00 00 00 54 53 0c 00 00 00 M??I.?....TS....
0070 45 52 4f 45 78 63 65 70 74 69 6f 6e 35 00 00 00 EROException5...
0080 53 74 72 65 61 6d 20 72 65 61 64 20 65 72 72 6f Stream read erro
0090 72 3a 20 49 6e 76 61 6c 69 64 20 73 74 72 69 6e r: Invalid strin
00a0 67 20 6c 65 6e 67 74 68 20 22 31 39 31 39 32 34 g length "191924
00b0 33 30 38 35 22 3085"
The string length in question is 1919243085 (Hex 0x7265534D) which you can see in the first message, listed in network byte order at offset 0x006C. Clearly the four bytes which are supposed to be the message length are at offset 0x0066 (Hex 0x0000000A).
Looking through the binary message format t appears that offet 0x0050 is where the header begins. This reads “R0107” followed by a number of flags and message type bytes, followed by a GUID, broken down into its 16 bytes. In the message sent, it appears that the last six bytes of the GUID have been overwritten with the message data which ought to have followed at 0x006C.
Indeed, you can see in the reply that the GUID has been copied and includes the four bytes of the length and the first two bytes of the message string!
What in the world is happening to cause these differences in behaviour? This product has been in use for a decade and we’ve never seen anything like this.
Additional question too: I’ve stepped through the creation of the GUID which begins at 0x005C. It was a very normal looking random number which I’ve tracked through to its being added to the outgoing message buffer. However, every trace I’ve taken while investigating this issue has shown me a GUID with lots of instances of 0x3F as bytes. What’s causing this? It happens on the machines that are working and the one which isn’t, so I don’t think it’s the cause - could it be a symptom though?
Thanks in advance for any pointers anyone has!