We've had a discussion on the list, some time ago when I made that "multipart is dead!" blog posting... of course as many people pointed out, it's an essential feature.
Deprecating the request-reply pattern is going to take time. No hurry. I like the idea of an atomic API scatter/gather send. This also removes that bogus implication that multipart is somehow a form of streaming. On Sat, Sep 12, 2015 at 1:09 AM, Doron Somech <[email protected]> wrote: > I think it is a good solution to the problem. We can continue and support > old API which will not be thread safe and have thread safe methods for > multipart messages. We then can drop those tests which already problematic. > Binding will have some work adapting to new safe API. > > On Sep 11, 2015 4:46 PM, "Joe McIlvain" <[email protected]> wrote: >> >> Solution: Add new libzmq API for sending and receiving multi-part messages >> atomically/statelessly. >> >> --- >> >> With this commit, the REQ, REP, DEALER, and ROUTER types are marked as >> soon-to-be deprecated in favor of the CLIENT and SERVER types on libzmq >> master: >> >> https://github.com/hintjens/libzmq/commit/f3ee8c69dbb50322a931534a55940fef85ad71dd >> >> This email is not about fighting the deprecation, but I do think that one >> specific deficit in the current implementation of the CLIENT and SERVER >> types: the lack of multi-part messages. >> >> I think the lack of multi-part message support will limit the usefulness >> of the CLIENT and SERVER types and as a consequence make it harder to >> encourage adoption. Also, I think multi-part messages are part of what >> makes ZeroMQ incredibly convenient for rapid development of useful >> applications. >> >> Furthermore, the lack of multi-part messages seems entirely inspired by >> the difficulty of creating thread-safe sockets within the C/C++ >> implementation (libzmq). This is because multi-part messages are sent using >> the ZMQ_RCVMORE and ZMQ_SNDMORE flags, making the process a stateful one. >> However, it strikes me that many high level bindings and ports are already >> conceptually covering this statefulness with a stateless abstraction in >> which multi-part messages are contained in a single object and sent with a >> single API call (for example, zmsg in CZMQ contains multiple zframes). In >> such high level bindings and ports, removing multi-part messaging is quite >> an artificial restriction, with no basis in conceptual technical necessity. >> >> Because many such abstractions exist around the concept of multi-part >> messages as a single object, both in bindings/ports and in applications, >> removing support for multi-part messages in new socket types and deprecating >> the old types that supports them creates a cascading effect of rewrites that >> is likely to frustrate many maintainers across the spectrum. So, to be >> sure, modern technologies will always undergo major changes if they wish to >> remain modern, so this is not necessarily a bad thing, but make no mistake >> that this design decision is quite a significant change in the ZeroMQ >> ecosystem, even though it has been executed so far with minimal impact to >> the API of libzmq. >> >> I'd like to take a moment to consider the alternative - what if we could >> make a more significant change to the API of libzmq in order to make a much >> smaller impact on the broader ZeroMQ ecosystem, including the APIs of >> bindings and ports as well as the multitude of applications? What if we can >> have both thread-safe sockets and keep multi-part messages - with the >> requirement that bindings and applications desiring thread-safety use the >> new atomic API for multi-part messages? For existing applications using the >> existing ZMQ_SNDMORE/ZMQ_RCVMORE API, nothing would change - they could >> continue to use this (probably deprecated) API, but not if they wanted to >> use the thread-safe socket types. This is already the case - the new >> ZMQ_CLIENT type will simply drop any messages with the ZMQ_SNDMORE flag >> (https://github.com/zeromq/libzmq/blob/master/tests/test_client_drop_more.cpp#L55), >> meaning that we're no worse off in this way. >> >> I'm arguing that such an API, if done right would be: >> - Backwards-compatible (the existing ZMQ_SNDMORE/ZMQ_RCVMORE API would >> still exist, but probably be deprecated) >> - Less of an impact on the broader ZeroMQ community than the current >> design direction of deprecating multi-part messaging. >> - Useful for making threadsafe implementations of the other patterns such >> as the publish-subscribe pattern. >> >> So far I fail to see any disadvantages that outweigh those advantages. >> >> --- >> >> I'd like this email thread to be about discussing the design of such an >> API. >> >> If we can come up with a coherent plan, I'd be happy to working on the >> implementation, using the ZMQ_CLIENT and ZMQ_SERVER sockets as the guinea >> pigs they already are. >> >> One way to implement such an API that comes to mind, is with an approach >> that mimics the `readv` and `writev` functions used in scatter/gather I/O >> (http://linux.die.net/man/2/writev). Such an API call in ZeroMQ could >> accept a pointer and count for an array of pointers to `zmq_msg_t` objects, >> instead of a pointer and count for an array of pointers to byte buffers. >> Alternatively, a different struct definition other than `zmq_msg_t` could be >> created that holds the pointer and count for an array of pointers to byte >> buffers. I think the first approach has a smaller API impact, but the >> second approach is more elegant. Both approaches would be preferable to >> dropping support for multi-part messages. >> >> Thoughts? >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
