I use a lot of event sinks. Almost every action is notified / broadcasted to any listening service so it can act on it. Because we need redundancy of services, we need Olympia, is the only way to have two services and multiple clients connecting to each service on different computers receiving notifications.
As a good news, we have a lot of custom Delphi servers, so using Olympia as a way to propagate messages across multiple instances should not be an issue. Only problem with Olympia is the lack of redundancy, there is no Active/passive or Active/Active node scenario there so when it goes down. Everything goes down, until it is restarted, it usually recovers fast and it doesn’t go down as often so its fine.
On the other hand, I have a service that handles “Master Controller” scenarios. This scenario is the case where you have multiple services of the same kind running and requests can be made to any of them but only one should perform a specific action.
e.g. You have a service that once a session expires, removes the session. If you have redundancy in place for this service and have two of them running. Both will try to remove the session originating a race condition. But if internally you can have a way to check if you are the designated “master” controller, then internally in your code you can skip the code after.
It was implemented by making all services registering as controllers to a service “registry”. First one to do it, becomes the “Master Controller” (or Active controller). From this point on, this registration service will send out a heart beat to all registered controllers to make sure they are alive. Once they get the heart beat they reply asking to be controllers again, but only the
designated “master controller” will receive confirmation that he remains as controller. This heart beat also allows each service to update their status as “master controllers” or not.
If the official master controller doesnt report itself, after a few attempts the registar service will then pass the master controller crown to a different service and continue from there.
This works for multiple readers, single writer scenarios. Something like this could also make a redundant Olympia work, but I haven’t try it.
Using the DbRepository etc as a sample to implement your own Olympia is not that simple and it is 90% of what Olympia does.
I added on top of RO event sinks a topic, header and direct exchanges mechanism similar to other messaging queues, but RO shows some cracks on service crash scenarios and heavy threaded load. Recovery is though and usually involves restarting services which on a “micro” services scenario can not be done.
Replaying messages will be nice and thats where the “proxy scenario” where those calls can be persisted somewhere and “replayed” when a service recovers could be nice.