> Am 12.09.2015 um 13:16 schrieb Pieter Hintjens <[email protected]>: > > We've had a discussion on the list, some time ago when I made that > "multipart is dead!" blog posting... of course as many people pointed > out, it's an essential feature. > > Deprecating the request-reply pattern is going to take time. No hurry. > I like the idea of an atomic API scatter/gather send. This also > removes that bogus implication that multipart is somehow a form of > streaming.
Amen. Bury ZMQ_MORE eventually, but not multipart. That'd be throwing out the baby with the bathwater. - Michael > > > > On Sat, Sep 12, 2015 at 1:09 AM, Doron Somech <[email protected]> wrote: >> I think it is a good solution to the problem. We can continue and support >> old API which will not be thread safe and have thread safe methods for >> multipart messages. We then can drop those tests which already problematic. >> Binding will have some work adapting to new safe API. >> >> On Sep 11, 2015 4:46 PM, "Joe McIlvain" <[email protected]> wrote: >>> >>> Solution: Add new libzmq API for sending and receiving multi-part messages >>> atomically/statelessly. >>> >>> --- >>> >>> With this commit, the REQ, REP, DEALER, and ROUTER types are marked as >>> soon-to-be deprecated in favor of the CLIENT and SERVER types on libzmq >>> master: >>> >>> https://github.com/hintjens/libzmq/commit/f3ee8c69dbb50322a931534a55940fef85ad71dd >>> >>> This email is not about fighting the deprecation, but I do think that one >>> specific deficit in the current implementation of the CLIENT and SERVER >>> types: the lack of multi-part messages. >>> >>> I think the lack of multi-part message support will limit the usefulness >>> of the CLIENT and SERVER types and as a consequence make it harder to >>> encourage adoption. Also, I think multi-part messages are part of what >>> makes ZeroMQ incredibly convenient for rapid development of useful >>> applications. >>> >>> Furthermore, the lack of multi-part messages seems entirely inspired by >>> the difficulty of creating thread-safe sockets within the C/C++ >>> implementation (libzmq). This is because multi-part messages are sent using >>> the ZMQ_RCVMORE and ZMQ_SNDMORE flags, making the process a stateful one. >>> However, it strikes me that many high level bindings and ports are already >>> conceptually covering this statefulness with a stateless abstraction in >>> which multi-part messages are contained in a single object and sent with a >>> single API call (for example, zmsg in CZMQ contains multiple zframes). In >>> such high level bindings and ports, removing multi-part messaging is quite >>> an artificial restriction, with no basis in conceptual technical necessity. >>> >>> Because many such abstractions exist around the concept of multi-part >>> messages as a single object, both in bindings/ports and in applications, >>> removing support for multi-part messages in new socket types and deprecating >>> the old types that supports them creates a cascading effect of rewrites that >>> is likely to frustrate many maintainers across the spectrum. So, to be >>> sure, modern technologies will always undergo major changes if they wish to >>> remain modern, so this is not necessarily a bad thing, but make no mistake >>> that this design decision is quite a significant change in the ZeroMQ >>> ecosystem, even though it has been executed so far with minimal impact to >>> the API of libzmq. >>> >>> I'd like to take a moment to consider the alternative - what if we could >>> make a more significant change to the API of libzmq in order to make a much >>> smaller impact on the broader ZeroMQ ecosystem, including the APIs of >>> bindings and ports as well as the multitude of applications? What if we can >>> have both thread-safe sockets and keep multi-part messages - with the >>> requirement that bindings and applications desiring thread-safety use the >>> new atomic API for multi-part messages? For existing applications using the >>> existing ZMQ_SNDMORE/ZMQ_RCVMORE API, nothing would change - they could >>> continue to use this (probably deprecated) API, but not if they wanted to >>> use the thread-safe socket types. This is already the case - the new >>> ZMQ_CLIENT type will simply drop any messages with the ZMQ_SNDMORE flag >>> (https://github.com/zeromq/libzmq/blob/master/tests/test_client_drop_more.cpp#L55), >>> meaning that we're no worse off in this way. >>> >>> I'm arguing that such an API, if done right would be: >>> - Backwards-compatible (the existing ZMQ_SNDMORE/ZMQ_RCVMORE API would >>> still exist, but probably be deprecated) >>> - Less of an impact on the broader ZeroMQ community than the current >>> design direction of deprecating multi-part messaging. >>> - Useful for making threadsafe implementations of the other patterns such >>> as the publish-subscribe pattern. >>> >>> So far I fail to see any disadvantages that outweigh those advantages. >>> >>> --- >>> >>> I'd like this email thread to be about discussing the design of such an >>> API. >>> >>> If we can come up with a coherent plan, I'd be happy to working on the >>> implementation, using the ZMQ_CLIENT and ZMQ_SERVER sockets as the guinea >>> pigs they already are. >>> >>> One way to implement such an API that comes to mind, is with an approach >>> that mimics the `readv` and `writev` functions used in scatter/gather I/O >>> (http://linux.die.net/man/2/writev). Such an API call in ZeroMQ could >>> accept a pointer and count for an array of pointers to `zmq_msg_t` objects, >>> instead of a pointer and count for an array of pointers to byte buffers. >>> Alternatively, a different struct definition other than `zmq_msg_t` could be >>> created that holds the pointer and count for an array of pointers to byte >>> buffers. I think the first approach has a smaller API impact, but the >>> second approach is more elegant. Both approaches would be preferable to >>> dropping support for multi-part messages. >>> >>> Thoughts? >>> >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
