I just terminated most of my porting of the P2P layer of our cluster to ZMQ, (ab-)using a lot the collector device I have presented last week (see http://www.tzone.org/~fabien/hg/index.cgi/collector/). I find out that it shared a lot with two other pattern used in ZMQ, the PUB socket and the LRU device, and so there is probably some stuff to share to make them more easy to implement both, which can probably be useful for other usages.
Here the feature set: Feature 1 - Add a new msg LABEL flag marking a frame as "labeled". This flag would allow to distinguish between frames from the original sender and frame added by REQ and ROUTER sockets. This will greatly ease the usage of any socket that connect to a ROUTER socket, including the REQ socket itself. Feature 2 - Embedded flags inside the msg_t structure. By allowing frame flags to be set inside the msg_t structure, it would be far easier to keep forward compatibility with new flags for proxy device that need to copy msg frames in a transparent manner. Just as an example, a queue socket would simply be a while True: out.send(in.recv()) loop, which will now take care of the multipart flags without problem. Feature 3 - Add a timeout sockopt. The timeout would allow the recv method to return with a ETIMEOUT error after a specified amount of time inactivity. It can also be queried by device to implement a similar mechanism inside the device polling scheme. May be the socket could also reset itself, just like if the socket were close and reopen again (removing the need for the user to do it itself). Feature 4 - Add a ready sockopt. This sockopt allow a socket to be marked as ready to read a new message. It will be send on any messages send by the socket as well as the connection handshake time, and will be reset on the next successful recv of the socket (this information does not need to be propagated however). On the other end, sockets could be setup (another sockopt) to only send to socket which were marked as ready (either in a lb or dist manner). Just having this option will make the LRU device a simple queue with a special socket option. You'll be able to use REP device on the worker just as usual and could probably even handled disconnection of the standby workers more efficiently. Feature 5 - The COLLECTORÂ socket. The COLLECTORÂ socket is related to the PATCH socket just like the REQ socket is related to XREQ. COLLECTOR socket would allow reception only on sockets that it has send a message previously, and only until it received a READY signal from this socket. All other messages would be dropped, although any READY signal will push back the socket in the eligible list for the next send request. All those features can be emulated in "client" space, but they are clearly less efficient than doing it directly in the library itself. Most of the current existing pattern would be greatly simplified using them, avoiding weird usage like DEALER socket as worker end point in LRU, and probably unseen and interesting new patterns could probably build using them. I'm volunteer to work on all of them, I would just like to have your agreement on it before starting. I'm still a very new to ZMQ and even to network programming, so I could have miss a lot of things in this proposal. Thanks, Fabien _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
