The discussion about thread safety was quite short iirc, though that contributor did discuss other things... at length. I merged his "thread safe socket" change rapidly, then we reverted it after a few days, and he disappeared. It was rather brute force and I suspect did not work at all, it simply wrapped all accesses to the socket structure in mutexes. No discussion at the time of multipart data and atomic send/recv.
As for socket safety, I've no strong opinion. I see that many people expect that to work and hit errors when it doesn't. I see that nanomsg has threadsafe sockets and no multipart. I see that sharing sockets across threads would make some actor models simpler, which is nice. On Wed, Feb 4, 2015 at 7:35 PM, Michel Pelletier <[email protected]> wrote: > I think Brian has some good points here, there are numerous unrelated issues > being discussed in this thread. > > A few points that I have: > > Multi part messages have also bothered me. However as a Python programmer i > see Min's points about the expense of buffer creation. To my knowledge > zproto does not (yet) have Python generation support either, or maybe > something like generated cffi or ctypes wrappers around the zproto generated > C code. That being said there are a variety of serialization libraries for > Python. With some ctypes and mmap magic they can also be done "zero copy" > but it's not pretty: > > https://gist.github.com/michelp/7522179 > > Multi part envelops are also how multi-hop routing is done. I don't see how > the new ideas handle that. I don't think we can just say "multi hop routing > is bad" and get rid of it. > > "Thread safe" sockets do not sound appealing to me. We did that, had a long > and contentious discussion with the person championing them, merged it, then > reverted it and that person is now no longer in the community. Pieter was > the most vocal opponent to them then and now he wants them back. Of course, > anyone can change their mind, but the only current argument I hear now for > them though is improving the performance of short lived threads, but that > can be solved, more correctly in my opinion, with thread or connection > pools. If you creating and tearing down threads that rapidly then you have > two problems. > > -Michel > > On Wed, Feb 4, 2015 at 3:37 AM, Brian Knox <[email protected]> wrote: >> >> After catching up on this thread, I feel like at least three problems are >> being conflated into one problem. I'll state what I see being discussed >> from my perspective: >> >> 1. "Using multi part messages as a way to route to clients from a router >> socket is overly complicated and not how new users expect things to work" >> >> 2. "Using multi part messages for message serialization is costly, and >> potentially confusing to others." >> >> 3. "ZeroMQ sockets are not thread safe." >> >> While on an implementation level these three problems may be related, on a >> conceptual level I don't see them as related. I may agree with some of >> these problem statements and not others. >> >> For me, my first priority is to always have the ability to get back a nice >> agnostic blob of bytes from ZeroMQ. This makes it easy to make ZeroMQ >> socket use compatible with standard io interfaces in Go. Structure for what >> is contained in those bytes is a concern of a different layer. Sometimes I >> use zproto for this (which I like), and other times I don't. >> >> As a demonstration that the problems are different problems, I solved #1 >> for myself in goczmq without addressing anything else. >> >> I would assert some of the confusion in this discussion is that we're >> talking about multiple problem statements at the same time. >> >> Cheers - and it was great meeting people this week! >> >> Brian >> >> >> >> >> On Wed, Feb 4, 2015 at 12:50 AM, Pieter Hintjens <[email protected]> wrote: >>> >>> Ironically, in my testing of high message rate), allowing multipart >>> creates significant costs. Multipart is just one way of getting >>> zero-copy, and even then only works on writing, not reading. >>> >>> For high performance brokers like Malamute I'd *really* like to be >>> moving blobs around instead of lists of blobs. >>> >>> >>> On Wed, Feb 4, 2015 at 12:41 AM, Gregg Irwin <[email protected]> >>> wrote: >>> > M> Perhaps it is because I spend my days in a higher level language >>> > M> like Python, but zproto is not an attractive option. >>> > >>> > Same here. I will read in detail about it shortly, but it may not make >>> > it into my toolbox as a multipart replacement. Multipart looked very >>> > cool when I found 0MQ, but I've ended up not using it much. I'm not >>> > doing high performance stuff though. Simplicity and ease of use are >>> > tops on my list. >>> > >>> > -- Gregg >>> > >>> > _______________________________________________ >>> > zeromq-dev mailing list >>> > [email protected] >>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
