I've been working on req-broker-rep messaging pattern for an application
and so far its working great. Thanks for the help. Right now it does
what I need it to do for the current application, but I've realized
there is a lot more potential if I extend this pattern. By the way I'm
running zmq on raspberry pi3's. The broker enforces a 1:1 relationship
between client and worker.
My broker has a special command that allows it to create the requested
service. It responds okay when the service is created. The client can
then send messages to the newly apwned service. This means that the
broker could be sitting on a remote machine with no application
processes running, get a message to start a service, then pass requests
along to that service. The broker enforces a 1:1 relationship between
client and worker. The last request will usually be for the service to
self-terminate. But doesn't have to be. Its worthwhile to note that in
my applications none of the modules store state. So I think that makes
them naturally restful. However, there's nothing to prevent a service
from always running and storing state if that's whats required.
What if each client was also a worker? Then I would have:
req/rep---broker---rep/req. Nice. That's the reason for the one-one
relationship and the service self-terminating. The calling client (in
my application) is itself a worker to some other client. So I'm
creating a dynamic, distributed, on demand pipe-lining system. This is
pretty tasty sounding to me. I've already done it, but I want to go
further. Lets make it scalable.
How do you discover services that are not running? The broker could be
pre-populated with information about the services it can start. Clients
could query the broker about available services. Wait... what about if a
new service popped into existence only long enough to register itself
with the broker then die and wait to be spawned again on command.
Cool. Hidden services? Sure, just pass the name to the broker and it
will happily start it. That way the modules in question are not public
knowledge to other modules.
Where are these brokers, and how do I find them? Brokers are always
running (well maybe not) so they should broadcast messages at regular
intervals to other brokers notifying them of their existence on the
network. Then the other brokers who receive this message will broadcast
back their location and the services they know about. A client need
only know of the existence of one broker to get in on the game. So
really each machine with clients could run its own broker on a known
socket for the express reason of discovering other unknown brokers even
though it may not have any services available locally. To cut down on
traffic you could have gateway super-brokers who sit at network access
points and collect information about brokers on the upstream side.
I'm not enforcing any specific way for clients and workers to
communicate with respect to their particular application except that the
only requirement is for two parts in a multi-part message. The first
part is the command which could be a command to the broker or a worker.
And the second is the data (if any) that corresponds to the command.
Like filename or module name or any kind of data that the worker needs
to do its thing. If a command to a worker is something like read, then
the data in the request could be blank but be filled in when the
response comes back. The first part which was a command field in the
request becomes a status field in the reply (ex okay, good, error etc).
In the case of error, the data part would be the error message. Its up
to the application to decide what the commands should be (apart from
broker commands) and what the data part of the message actually means.
Just adhere to the contract that all requests must receive a reply.
Sorry this was long winded. Maybe it was just a way to get the thoughts
out of my head and onto virtual paper. Anybody have any thoughts on
this? ZMQ really is radioactive eh? I will publish this part of
application to a public github soon. There's nothing proprietary in
it. Our secret sauce is weird serial protocols that emulate tape
cassette and bubble memory readers that replaced punch tape machines.
Extending it this way is an after work project for now. I have to work
on the secret sauces.
--
<-Douglas Petican->
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
