blockquote, div.yahoo_quoted { margin-left: 0 !important; border-left:1px
#715FFA solid !important; padding-left:1ex !important; background-color:white
!important; } what about enhancing zmq_sendiov to do the right thing for these
new sockets and then remove the experimental label when it works?
zmq_sendv(const zmq_msg_t*, int n)
that did the right thing would be nicer for some use cases but I think there
could be some benefits in experimenting with an existing "experimental"
interface first.
On Friday, September 11, 2015, 16:09, Doron Somech <[email protected]> wrote:
I think it is a good solution to the problem. We can continue and support old
API which will not be thread safe and have thread safe methods for multipart
messages. We then can drop those tests which already problematic. Binding will
have some work adapting to new safe API.
On Sep 11, 2015 4:46 PM, "Joe McIlvain" <[email protected]> wrote:
Solution: Add new libzmq API for sending and receiving multi-part messages
atomically/statelessly.
---
With this commit, the REQ, REP, DEALER, and ROUTER types are marked as
soon-to-be deprecated in favor of the CLIENT and SERVER types on libzmq master:
https://github.com/hintjens/libzmq/commit/f3ee8c69dbb50322a931534a55940fef85ad71dd
This email is not about fighting the deprecation, but I do think that one
specific deficit in the current implementation of the CLIENT and SERVER types:
the lack of multi-part messages.
I think the lack of multi-part message support will limit the usefulness of the
CLIENT and SERVER types and as a consequence make it harder to encourage
adoption. Also, I think multi-part messages are part of what makes ZeroMQ
incredibly convenient for rapid development of useful applications.
Furthermore, the lack of multi-part messages seems entirely inspired by the
difficulty of creating thread-safe sockets within the C/C++ implementation
(libzmq). This is because multi-part messages are sent using the ZMQ_RCVMORE
and ZMQ_SNDMORE flags, making the process a stateful one. However, it strikes
me that many high level bindings and ports are already conceptually covering
this statefulness with a stateless abstraction in which multi-part messages are
contained in a single object and sent with a single API call (for example, zmsg
in CZMQ contains multiple zframes). In such high level bindings and ports,
removing multi-part messaging is quite an artificial restriction, with no basis
in conceptual technical necessity.
Because many such abstractions exist around the concept of multi-part messages
as a single object, both in bindings/ports and in applications, removing
support for multi-part messages in new socket types and deprecating the old
types that supports them creates a cascading effect of rewrites that is likely
to frustrate many maintainers across the spectrum. So, to be sure, modern
technologies will always undergo major changes if they wish to remain modern,
so this is not necessarily a bad thing, but make no mistake that this design
decision is quite a significant change in the ZeroMQ ecosystem, even though it
has been executed so far with minimal impact to the API of libzmq.
I'd like to take a moment to consider the alternative - what if we could make a
more significant change to the API of libzmq in order to make a much smaller
impact on the broader ZeroMQ ecosystem, including the APIs of bindings and
ports as well as the multitude of applications? What if we can have both
thread-safe sockets and keep multi-part messages - with the requirement that
bindings and applications desiring thread-safety use the new atomic API for
multi-part messages? For existing applications using the existing
ZMQ_SNDMORE/ZMQ_RCVMORE API, nothing would change - they could continue to use
this (probably deprecated) API, but not if they wanted to use the thread-safe
socket types. This is already the case - the new ZMQ_CLIENT type will simply
drop any messages with the ZMQ_SNDMORE flag
(https://github.com/zeromq/libzmq/blob/master/tests/test_client_drop_more.cpp#L55),
meaning that we're no worse off in this way.
I'm arguing that such an API, if done right would be:
- Backwards-compatible (the existing ZMQ_SNDMORE/ZMQ_RCVMORE API would still
exist, but probably be deprecated)
- Less of an impact on the broader ZeroMQ community than the current design
direction of deprecating multi-part messaging.
- Useful for making threadsafe implementations of the other patterns such as
the publish-subscribe pattern.
So far I fail to see any disadvantages that outweigh those advantages.
---
I'd like this email thread to be about discussing the design of such an API.
If we can come up with a coherent plan, I'd be happy to working on the
implementation, using the ZMQ_CLIENT and ZMQ_SERVER sockets as the guinea pigs
they already are.
One way to implement such an API that comes to mind, is with an approach that
mimics the `readv` and `writev` functions used in scatter/gather I/O
(http://linux.die.net/man/2/writev). Such an API call in ZeroMQ could accept a
pointer and count for an array of pointers to `zmq_msg_t` objects, instead of a
pointer and count for an array of pointers to byte buffers. Alternatively, a
different struct definition other than `zmq_msg_t` could be created that holds
the pointer and count for an array of pointers to byte buffers. I think the
first approach has a smaller API impact, but the second approach is more
elegant. Both approaches would be preferable to dropping support for
multi-part messages.
Thoughts?
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
_______________________________________________
zeromq-dev mailing list
[email protected]
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
