Implementing IOCP as a transport mechanism?

estebanp · May 26, 2019, 4:52am

Hi,
On a future version of RO, they will be implementing some of the protocols based on the Grijjy open source communication classes. Grijjy implemented IOCP on them, even if RO doesnt directly support them (hopefully they will) it could be possible to modify them so they can be used.

Here it is a link to their source regarding IOCP.

github.com

grijjy/DelphiScalableClientSockets/blob/master/HTTP.md

# Scalable HTTP sockets for the cloud, Part 2

In this article we will expand on our TgoHttpClient class by adding some core new features including non-blocking http responses, unifying HTTP 1.1, HTTP/S and HTTP/2 support into a common class, incremental data transfers, various fixes and performance improvements.

[In a previous article](https://blog.grijjy.com/2017/01/09/scalable-https-and-tcp-client-sockets-for-the-cloud/) we discussed the scalable client socket problem and how it creates a bottleneck for services that need to communicate over TCP as a client socket.  This model is common in backend services when you interact with third-party providers such as databases, remote push notification services (Apple or Google) or just about any JSON/REST based HTTP API.

If you use a typical HTTP client or component in your Windows or Linux service, you are almost always using [Berkeley based sockets](https://en.wikipedia.org/wiki/Berkeley_sockets).  There are numerous scale issues presented by the Winsock stack under Windows including port limitations, connection timeout retry and delay limitations, page pool issues and much more.  As discussed previously, most of these limitations are worked around in Windows by using Windows IO Completion Ports (IOCP).  IOCP has been available for quite some time but it is notoriously difficult to use and has been almost exclusively used for creating server applications for TCP sockets.  

IOCP is well designed solution to scalable client sockets but it is rarely used as a foundation model that allows you to build client sockets for other protocols.  In our previous article on [Scalable HTTP/S and TCP client sockets for the cloud](https://blog.grijjy.com/2017/01/09/scalable-https-and-tcp-client-sockets-for-the-cloud/), we demonstrated how to build a foundation class that uses IOCP to create client socket protocols with a brief examination of the HTTP protocol.

> This foundation model could easily be expanded and future client protocols and the apps that rely upon them could operate without modification on other operating systems such as Linux.  Internally we have done this on Linux directly using FPC and Lazarus and hope to soon demonstrate this running for Linux on the latest Delphi compilers.   

IOCP is at the heart of how Windows IIS operates in conjunction with the HTTP.SYS kernel driver and scales up efficiently.  On Linux we have similar technologies such as EPOLL and KQueue and they are widely utilized in server based sockets.  These technologies are also a perfect match for client based socket issues.

The TgoHttpClient class demonstrates how you can leverage IOCP and our foundation class TgoSocketConnection to create client sockets for the purpose of HTTP and take advantage of the many benefits of IOCP at the same time.  

## Non-blocking HTTP

IOCP is designed to be non-blocking and handle numerous activities simultaneously.  This is due to the fact that IOCP operations occur within a thread pool.

This file has been truncated. show original

wuping · October 22, 2019, 10:00am

Is IOCP included already with RoSDK latest release?
@sergeyl

EvgenyK · October 22, 2019, 10:08am

Hi,

We have implemented a few Grijjy-based channels in ROD v10:

TROGrijjyHttpChannel
TROGrijjySuperHttpChannel
TROGrijjySuperTCPChannel

wuping · November 3, 2019, 12:00am

@EvgenyK
I have 2 questions:

Are all of these three channels using IOCP, or just the HTTP channel?
They are not included in the Service Tester? Can we have them in the Service Tester?

EvgenyK · November 4, 2019, 8:39am

Hi,

Though TgoHttpClient supports blocking and non-blocking modes, we use it only in blocking mode so
IOCP won’t work for this component.

will add Grijjy channels to Service Tester

wuping · November 4, 2019, 12:24pm

I see.

So the value of these Grijjy based new components, is not about IOCP, but about the capability of being cross-platform. Is that correct?

EvgenyK · November 4, 2019, 1:08pm

Yes, they work for Windows and Linux platform and can be replacement for users who dislikes using Indy components.

RemObjectsSoftware · November 6, 2019, 9:17am

bugs://83479 got closed with status wontfix.

EvgenyK · November 6, 2019, 9:18am

Grijjy channels aren’t compatible with Delphi7 that is used for compiling Service Tester …

wuping · November 6, 2019, 5:03pm

Understand. Not a problem.

claudio.piffer · November 7, 2019, 9:57am

Hi EvgenyK

it would be great if you will can release the source code of the ROServiceTester on github (only users with a license) so we can add the missing features, port the product to new versions of Delphi, etc. etc…

Best regards

EvgenyK · November 7, 2019, 10:01am

Hi,

see mh’s answer at RO SDK Service Tester is buggy, why not "open-source" it for licensed users?

claudio.piffer · November 7, 2019, 10:12am

Thank EvgenyK

I had missed the answer! For the third-party components most of us probably already have them

EvgenyK · November 7, 2019, 10:33am

Service Tester uses a lot of 3rd party components are released in 2006.
Some of them don’t exist anymore, like Eagle Software CDK, etc
I don’t think, that anybody except us continue to use them

claudio.piffer · November 7, 2019, 12:12pm

Hi EvgenyK

ah ok!

But in this case the product can never evolve (see for example integrate the Grijjy channels).
ROServiceTester is a very useful accessory product. More than once I thought of writing one myself to be able to customize it but I never have the time to do it. So having the possibility of the source to make the community evolve it would be a very nice thing. But I also understand the “problem” of third-party components if they are no longer on the market.

EvgenyK · November 7, 2019, 12:35pm

If you have any ideas about improvement of Service Tester - you are welcome.

wuping · November 7, 2019, 1:31pm

CDK was indeed quite historic, and only old Delphi-ers knew about it

I thought it was acquired by DevExpress and they suspend its support after Delphi 7. But if you have DevExpress VCL subscription, you might still get its source code by contacting DevExpress or Julian their CTO directly.

EvgenyK · November 7, 2019, 1:41pm

This was one from 3rd party libraries. so it can be problematic to get them all updated for the latest version of Delphi.

estebanp · November 7, 2019, 4:31pm

@EvgenyK out of curiosity and because you have the experience, how problematic and most certainly hard is to align IOCP (like the one grijjy implemented) to RO SDK. The benefits are obvious but I’m sure that it was skipped due to its complexities, care to give details on the consideration?

Thank you.

EvgenyK · November 8, 2019, 11:11am

Hi,

IOCP works in asynchronous mode only so it can’t be suitable for usual (non-async) methods where client waits result of execution.
Client-side won’t be much (if any) benefit from IOCP unless you are doing hundreds req/s from the same client (an in this case it means that you are doing something terribly wrong).
Currently RO SDK for Delphi emulates asynchronous requests via sending them via usual (non-async) requests.
Implementation of IOCP will require rewriting significant portions of RO SDK code so it can be a bit compilcatied and non-reasonable at this moment because only Grijjy library (XE8+, Windows&Linux only) supports it