Have a look at Zyre, it deals with a lot of this.
On Thu, Jun 16, 2016 at 2:35 AM, Douglas Petican <[email protected]> wrote: > I've been working on req-broker-rep messaging pattern for an application and > so far its working great. Thanks for the help. Right now it does what I > need it to do for the current application, but I've realized there is a lot > more potential if I extend this pattern. By the way I'm running zmq on > raspberry pi3's. The broker enforces a 1:1 relationship between client and > worker. > > My broker has a special command that allows it to create the requested > service. It responds okay when the service is created. The client can then > send messages to the newly apwned service. This means that the broker could > be sitting on a remote machine with no application processes running, get a > message to start a service, then pass requests along to that service. The > broker enforces a 1:1 relationship between client and worker. The last > request will usually be for the service to self-terminate. But doesn't have > to be. Its worthwhile to note that in my applications none of the modules > store state. So I think that makes them naturally restful. However, > there's nothing to prevent a service from always running and storing state > if that's whats required. > > What if each client was also a worker? Then I would have: > req/rep---broker---rep/req. Nice. That's the reason for the one-one > relationship and the service self-terminating. The calling client (in my > application) is itself a worker to some other client. So I'm creating a > dynamic, distributed, on demand pipe-lining system. This is pretty tasty > sounding to me. I've already done it, but I want to go further. Lets make > it scalable. > > How do you discover services that are not running? The broker could be > pre-populated with information about the services it can start. Clients > could query the broker about available services. Wait... what about if a new > service popped into existence only long enough to register itself with the > broker then die and wait to be spawned again on command. Cool. Hidden > services? Sure, just pass the name to the broker and it will happily start > it. That way the modules in question are not public knowledge to other > modules. > > Where are these brokers, and how do I find them? Brokers are always running > (well maybe not) so they should broadcast messages at regular intervals to > other brokers notifying them of their existence on the network. Then the > other brokers who receive this message will broadcast back their location > and the services they know about. A client need only know of the existence > of one broker to get in on the game. So really each machine with clients > could run its own broker on a known socket for the express reason of > discovering other unknown brokers even though it may not have any services > available locally. To cut down on traffic you could have gateway > super-brokers who sit at network access points and collect information about > brokers on the upstream side. > > I'm not enforcing any specific way for clients and workers to communicate > with respect to their particular application except that the only > requirement is for two parts in a multi-part message. The first part is the > command which could be a command to the broker or a worker. And the second > is the data (if any) that corresponds to the command. Like filename or > module name or any kind of data that the worker needs to do its thing. If a > command to a worker is something like read, then the data in the request > could be blank but be filled in when the response comes back. The first part > which was a command field in the request becomes a status field in the reply > (ex okay, good, error etc). In the case of error, the data part would be the > error message. Its up to the application to decide what the commands should > be (apart from broker commands) and what the data part of the message > actually means. Just adhere to the contract that all requests must receive > a reply. > > Sorry this was long winded. Maybe it was just a way to get the thoughts out > of my head and onto virtual paper. Anybody have any thoughts on this? ZMQ > really is radioactive eh? I will publish this part of application to a > public github soon. There's nothing proprietary in it. Our secret sauce is > weird serial protocols that emulate tape cassette and bubble memory readers > that replaced punch tape machines. Extending it this way is an after work > project for now. I have to work on the secret sauces. > > > -- > <-Douglas Petican-> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
