Implementing IOCP as a transport mechanism?

bobokonijn · November 29, 2012, 10:52am

Hello Remobjects team,

are there any plans to implement IOCP as a transport mechanism in Remobjects SDK? I have read some discussions and blogs about it lately and it seems that this mechanism is suitable for much higher throughput and concurrency than Indy etc.

sergeyl · November 30, 2012, 10:22am

Hello,

IOCP is hardly a transport. It is an API to handle multiple I/O requests, can be bound to any file handle including disk files and sockets. Yes, it seems to provide a more efficient threading model than the classic “dedicated thread for each request”. But there is an important disadvantage for us: unlike Indy this API is Windows only.
Due to some issues with Indy and Synapse, the libraries we currently use, there are discussions about creating our own channels and servers to go away from third party dependence. We’ll consider this API too but mind you, it is not a matter of the near future.

Best regards - Sergey

deksden · December 3, 2012, 11:48am

Synopse’s mORMot project leverage IOCP on Windows, its fast and cool, but yes, its not cross-platform. And yes, its much, much faster than Indy!

Maybe its better to use some frontend server, like NGINX, as frontend, and use some plugin to process request. For Linux/OSX, NGINX have best performance, much better then Apache!

bobokonijn · December 3, 2012, 12:48pm

Only Indy is cross-platform… but I personally never use it because Synapse is rock steady.

chrisroyal · December 3, 2012, 4:03pm

Has anyone tried msgconnect with RO?

wuping · May 26, 2019, 1:48am

I would like to ask if IOCP still on the roadmap. It would be a great addition for Windows based RO server! Please consider it

estebanp · May 26, 2019, 4:52am

Hi,
On a future version of RO, they will be implementing some of the protocols based on the Grijjy open source communication classes. Grijjy implemented IOCP on them, even if RO doesnt directly support them (hopefully they will) it could be possible to modify them so they can be used.

Here it is a link to their source regarding IOCP.

github.com

grijjy/DelphiScalableClientSockets/blob/master/HTTP.md

# Scalable HTTP sockets for the cloud, Part 2

In this article we will expand on our TgoHttpClient class by adding some core new features including non-blocking http responses, unifying HTTP 1.1, HTTP/S and HTTP/2 support into a common class, incremental data transfers, various fixes and performance improvements.

[In a previous article](https://blog.grijjy.com/2017/01/09/scalable-https-and-tcp-client-sockets-for-the-cloud/) we discussed the scalable client socket problem and how it creates a bottleneck for services that need to communicate over TCP as a client socket.  This model is common in backend services when you interact with third-party providers such as databases, remote push notification services (Apple or Google) or just about any JSON/REST based HTTP API.

If you use a typical HTTP client or component in your Windows or Linux service, you are almost always using [Berkeley based sockets](https://en.wikipedia.org/wiki/Berkeley_sockets).  There are numerous scale issues presented by the Winsock stack under Windows including port limitations, connection timeout retry and delay limitations, page pool issues and much more.  As discussed previously, most of these limitations are worked around in Windows by using Windows IO Completion Ports (IOCP).  IOCP has been available for quite some time but it is notoriously difficult to use and has been almost exclusively used for creating server applications for TCP sockets.  

IOCP is well designed solution to scalable client sockets but it is rarely used as a foundation model that allows you to build client sockets for other protocols.  In our previous article on [Scalable HTTP/S and TCP client sockets for the cloud](https://blog.grijjy.com/2017/01/09/scalable-https-and-tcp-client-sockets-for-the-cloud/), we demonstrated how to build a foundation class that uses IOCP to create client socket protocols with a brief examination of the HTTP protocol.

> This foundation model could easily be expanded and future client protocols and the apps that rely upon them could operate without modification on other operating systems such as Linux.  Internally we have done this on Linux directly using FPC and Lazarus and hope to soon demonstrate this running for Linux on the latest Delphi compilers.   

IOCP is at the heart of how Windows IIS operates in conjunction with the HTTP.SYS kernel driver and scales up efficiently.  On Linux we have similar technologies such as EPOLL and KQueue and they are widely utilized in server based sockets.  These technologies are also a perfect match for client based socket issues.

The TgoHttpClient class demonstrates how you can leverage IOCP and our foundation class TgoSocketConnection to create client sockets for the purpose of HTTP and take advantage of the many benefits of IOCP at the same time.  

## Non-blocking HTTP

IOCP is designed to be non-blocking and handle numerous activities simultaneously.  This is due to the fact that IOCP operations occur within a thread pool.

This file has been truncated. show original

wuping · October 21, 2019, 12:39pm

Is IOCP included already with RoSDK latest release?
@sergeyl

EvgenyK · October 22, 2019, 10:07am

Hi,

We have implemented a few Grijjy-based channels in ROD v10:

TROGrijjyHttpChannel
TROGrijjySuperHttpChannel
TROGrijjySuperTCPChannel

wuping · November 2, 2019, 6:42pm

@EvgenyK
I have 2 questions:

Are all of these three channels using IOCP, or just the HTTP channel?
They are not included in the Service Tester? Can we have them in the Service Tester?

EvgenyK · November 4, 2019, 8:39am

Hi,

Though TgoHttpClient supports blocking and non-blocking modes, we use it only in blocking mode so
IOCP won’t work for this component.

will add Grijjy channels to Service Tester

wuping · November 4, 2019, 12:24pm

I see.

So the value of these Grijjy based new components, is not about IOCP, but about the capability of being cross-platform. Is that correct?

EvgenyK · November 4, 2019, 1:08pm

Yes, they work for Windows and Linux platform and can be replacement for users who dislikes using Indy components.

RemObjectsSoftware · November 6, 2019, 9:17am

bugs://83479 got closed with status wontfix.

EvgenyK · November 6, 2019, 9:18am

Grijjy channels aren’t compatible with Delphi7 that is used for compiling Service Tester …

wuping · November 6, 2019, 5:03pm

Understand. Not a problem.

claudio.piffer · November 7, 2019, 9:57am

Hi EvgenyK

it would be great if you will can release the source code of the ROServiceTester on github (only users with a license) so we can add the missing features, port the product to new versions of Delphi, etc. etc…

Best regards

EvgenyK · November 7, 2019, 10:01am

Hi,

see mh’s answer at RO SDK Service Tester is buggy, why not "open-source" it for licensed users?

claudio.piffer · November 7, 2019, 10:12am

Thank EvgenyK

I had missed the answer! For the third-party components most of us probably already have them

EvgenyK · November 7, 2019, 10:33am

Service Tester uses a lot of 3rd party components are released in 2006.
Some of them don’t exist anymore, like Eagle Software CDK, etc
I don’t think, that anybody except us continue to use them