Ironically the original argument for multiple frames was zero-copy performance -- you can send a message without copying, with a prefix frame for routing. In practice multiframe messages appear to punish performance.
On Mon, Feb 2, 2015 at 6:11 PM, Joe McIlvain <[email protected]> wrote: > Doron, > > You're right - there's a cost there I wasn't considering. I'm not yet > entirely convinced that the cost makes the abstraction no longer worth it in > certain applications, but it at least shows that it should not be the most > advocated pattern in general. > > > On Mon, Feb 2, 2015 at 8:55 AM, Doron Somech <[email protected]> wrote: >> >> Joe the problem with the api of zmsg for single frame is that you cannot >> calculate the frame size in advance causing a lot of re allocation and >> copying data around (it's ok for receiving, the problem is sending). >> >> >> >> On Mon, Feb 2, 2015 at 6:35 PM, Joe McIlvain <[email protected]> >> wrote: >>> >>> In my opinion, maintaining some kind of multi-frame abstraction at the >>> API level is in fact very useful to applications, even if they are sent >>> atomically on the implementation level as you describe. >>> >>> For example, in CZMQ It's quite useful in program flow to be able to push >>> or pop a single zframe onto or off of a zmsg, then send that zmsg as an >>> entire message. Rarely do I find myself using zframe_send or zframe_recv >>> with the "send more" and "receive more" flags in a CZMQ application. The >>> CZMQ "picture" messages are nice, but they are not always the easiest API to >>> integrate into an application. Having zmsg and zframe remain (or something >>> like them), while removing zframe_send and zframe_recv could ease >>> applications relying on multi-frame semantics into the transition. The >>> implementation for zmsg would simply serialize the frames in some >>> zmsg-determined way into a single-part zmq_msg to send and deserialize >>> according to the same scheme to receive. This multi-frame serialization >>> essentially be at the application level rather than at the ZMTP level. >>> >>> In short, I think a multi-frame abstraction is important for >>> applications, but if we remove the ability to send one frame at a time at >>> the fundamental level, this abstraction can remain at a higher level >>> (whether in libzmq, or in CZMQ and other wrappers) and coexist happily with >>> the proposed changes. >>> >>> >>> >>> On Mon, Feb 2, 2015 at 7:37 AM, Doron Somech <[email protected]> wrote: >>>> >>>> So the new sockets are in the github master, you can take a loot at the >>>> test to see how to use the new routing id field: >>>> >>>> >>>> https://github.com/zeromq/libzmq/blob/master/tests/test_client_server.cpp >>>> >>>> Few of the reasons I didn't like multi frames: >>>> * I tried in the past to make both zeromq and NetMQ thread safe, think >>>> of sending from multiple threads or receiving from multiple threads. we >>>> cannot do that with Multipart. >>>> * There are a lot of good transport that already implement message >>>> framing (UDP, WebSockets, SCTP and even HTTP), but because zeromq required >>>> it is own framing it was not easy to add them. >>>> * Multipart, router usage (routing id frame) and not being thread safe >>>> make the learning curve of zeromq hard to beginners. Without three of them >>>> zeromq become much simpler. >>>> * At least with NetMQ single part message is much faster than multipart. >>>> * New stacks, multipart is complicated to implement, with the new API it >>>> will much more easier to implement new stacks (like JSMQ) or any native >>>> stack. >>>> >>>> Pieter I'm looking forward to see how you expose the routing id in the >>>> czmq API. >>>> I also like the czmq API for sending mutlipart messages (the picture >>>> feature) so maybe we can use that to generate single frame which is also >>>> compatible with zproto. >>>> >>>> About the implementation, none of new sockets support any option now. >>>> server behave like mandatory router, so when router is not reachable or >>>> highwater mark is reached an error will be returned. >>>> >>>> As ZMTP 3.1 is still in raw status, what do you think of removing the >>>> multipart from it? maybe the 3.1 will only support the new socket types. >>>> >>>> Anyway I really excited about this change. >>>> >>>> >>>> >>>> >>>> >>>> On Mon, Feb 2, 2015 at 4:17 PM, Thomas Rodgers <[email protected]> >>>> wrote: >>>>>> >>>>>> What we really want IMO is per-peer metadata, and an API to get/set >>>>>> this. Using messages is a hack. >>>>> >>>>> >>>>> Currently working on that :) >>>>> >>>>>> Having two layers that both >>>>>> carry per-message data is... wrong IMO. >>>>> >>>>> >>>>> Protocols supporting 'out of band' data aren't exactly uncommon. >>>>> >>>>>> However the key thing is, what's the problem. Then we can discuss >>>>>> solutions to that. >>>>> >>>>> >>>>> I don't have an immediate use-case justifying it. I noted it, mostly >>>>> because it has come up a few times since I started paying attention. >>>>> >>>>> On Mon, Feb 2, 2015 at 7:55 AM, Pieter Hintjens <[email protected]> wrote: >>>>>> >>>>>> > It seems to me that in order to remove multi-part messages and >>>>>> > introduce new >>>>>> > socket types (e.g. SERVER/CLIENT) that would >>>>>> > necessitate a revision of the wire protocol. If we are going to do >>>>>> > that, it >>>>>> > might be worth considering per-message >>>>>> > metadata - >>>>>> >>>>>> We'd have to be very clear about the problem that per-message metadata >>>>>> is aiming for. There is an elegance to delivering blobs and nothing >>>>>> more. Metadata can be added on top using zproto. >>>>>> >>>>>> What we really want IMO is per-peer metadata, and an API to get/set >>>>>> this. Using messages is a hack. If we are sending/receiving data on a >>>>>> per-message basis, that is the message. Having two layers that both >>>>>> carry per-message data is... wrong IMO. >>>>>> >>>>>> However the key thing is, what's the problem. Then we can discuss >>>>>> solutions to that. >>>>>> >>>>>> -Pieter >>>>>> _______________________________________________ >>>>>> zeromq-dev mailing list >>>>>> [email protected] >>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> zeromq-dev mailing list >>>>> [email protected] >>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>> >>>> >>>> >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> [email protected] >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> >>> >>> >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
