Re: [zeromq-dev] thread affinity
Hi Justin, This is an area of some debate. We've had patches to libzmq that made sockets thread safe, and removed those patches again. Sharing sockets between threads for the most part is bad for design and performance. However there are languages where this just essential, for the reasons you describe. My advice would be to solve this in the language binding. Simply create socket access routines (send/recv) that do the necessary locking. -Pieter On Sat, Nov 30, 2013 at 7:30 AM, Justin Karneges jus...@affinix.com wrote: Hi folks, What's the latest on the thread-safety of sockets? I know that normal 0MQ practice suggests not using a socket from multiple threads, but I wonder if this is nonetheless possible, for example by wrapping a mutex around access. The reason I ask is I'm exploring the possibility of using 0MQ in a hybrid event-driven threaded C++ environment (imagine something like Goroutines or .NET tasks). A socket would never be used by two threads simultaneously, but the thread on which a socket is utilized could change depending on how the eventing system dispatches work. For example, a task executing in one thread could go to sleep while it waits for a message and then wake up in a different thread when a message becomes available. I just want to confirm that this is a valid usage of 0MQ, before I do something that might not be possible or might get broken in a future release. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] RFC: zbeacon for dhcp-style discovery
I have a seemingly trivial problem where I currently have tunnel vision: if I send a directed UDP packet to the beacon socket, the s_beacon_recv () callback isnt called except if I force the destination address to broadcast (255.255.255.255; in which case tcpdump does display the packet) if I set the sender address to one of the interface addresses, the packet doesnt go out as per tcpdump and s_beacon_recv() isnt called I wonder if I stepped into some socket behavior which I overlooked? noecho is not the cause btw. - Michael ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] RFC: zbeacon for dhcp-style discovery
On 11/30/2013 03:49 PM, Michael Haberler wrote: I have a seemingly trivial problem where I currently have tunnel vision: if I send a directed UDP packet to the beacon socket, the s_beacon_recv () callback isnt called except if I force the destination address to broadcast (255.255.255.255; in which case tcpdump does display the packet) if I set the sender address to one of the interface addresses, the packet doesnt go out as per tcpdump and s_beacon_recv() isnt called I wonder if I stepped into some socket behavior which I overlooked? noecho is not the cause btw. - Michael Hi Michael, I just tried it with the python zbeacon and the only way I can send data from socat to the socket is by specifying the broadcast option: $ socat - UDP-DATAGRAM:255.255.255.255:5670,broadcast sdkfjlsfjlsdfjls works okay, I reveive an error in zbeacon because the filter doens't match. $ socat - UDP-DATAGRAM:255.255.255.255:5670 sdflsdkf;sfk;slfk;fk;sfk;sd 2013/11/30 17:11:57 socat[31591] E sendto(3, 0x1ad99d0, 28, 0, AF=2 255.255.255.255:5670, 16): Permission denied In netstat I see the socket only listening on the broadcast address: udp0768 255.255.255.255:56700.0.0.0:* 31430/python3 I haven't read the docs but I guess this is normal socket behaviour? Rg, Arnaud -- w: http://www.sphaero.org t: http://twitter.com/sphaero g: http://github.com/sphaero i: freenode: sphaero_z25 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZRE/Zbeacon improvement proposal
On 10/27/2013 09:05 AM, Pieter Hintjens wrote: [snip] Yes, it's a natural pattern to say bye bye when leaving a room or ending a phone call, precisely to save the other person from wasting time wondering what happened. And the fallback is already there. So for leaving, it makes sense to broadcast a bye bye beacon. I was confused because you proposed five states to signal one thing, which seemed a lot. Given this use case, I'd modify the existing protocol in a compatible way. Right now we broadcast the port number to signal availability. it would be natural to broadcast a port number zero to signal unavailability. That's a safe change, minimal, and does what you need. Along with multicast as an option, and we make nice progress. Hi all, I just comitted sending port 0 on exit in the Python implementation of Zyre. I'm not sure this is the right way to do it since it is implemented mostly in zbeacon. Perhaps it would be better to implemented using the api calls but when doing this through a destructor will probably not work because of the asynchronous design? The commit is very simple: https://github.com/zeromq/pyre/commit/d6d83db38094e1a3dece663f5b1e8da36c604680 I've been testing it and it works very nice, on a reliable network ofcourse ;) No more poking to dead nodes. What do you think about how to implement this? Rg, Arnaud -- w: http://www.sphaero.org t: http://twitter.com/sphaero g: http://github.com/sphaero i: freenode: sphaero_z25 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] RFC: zbeacon for dhcp-style discovery
On 11/30/2013 06:30 PM, Michael Haberler wrote: Hi Arnaud, Am 30.11.2013 um 17:16 schrieb Arnaud Loonstra arn...@sphaero.org: On 11/30/2013 03:49 PM, Michael Haberler wrote: I have a seemingly trivial problem where I currently have tunnel vision: if I send a directed UDP packet to the beacon socket, the s_beacon_recv () callback isnt called except if I force the destination address to broadcast (255.255.255.255; in which case tcpdump does display the packet) if I set the sender address to one of the interface addresses, the packet doesnt go out as per tcpdump and s_beacon_recv() isnt called I wonder if I stepped into some socket behavior which I overlooked? noecho is not the cause btw. - Michael Hi Michael, I just tried it with the python zbeacon and the only way I can send data from socat to the socket is by specifying the broadcast option: $ socat - UDP-DATAGRAM:255.255.255.255:5670,broadcast sdkfjlsfjlsdfjls works okay, I reveive an error in zbeacon because the filter doens't match. $ socat - UDP-DATAGRAM:255.255.255.255:5670 sdflsdkf;sfk;slfk;fk;sfk;sd 2013/11/30 17:11:57 socat[31591] E sendto(3, 0x1ad99d0, 28, 0, AF=2 255.255.255.255:5670, 16): Permission denied In netstat I see the socket only listening on the broadcast address: udp0768 255.255.255.255:56700.0.0.0:* 31430/python3 thanks! this makes perfect sense - my implicit assumption so far was that udpsocket would receive unicasts just as well, which likely was the error - in retrospect there's no good reason it should this would suggest that binding udpsocket also for each interface address will result in the desired behavior. I'll try that but I'm sure that was it. is that a change we're willing to accept? I dont see side effects as of now - yet - Michael Well I've been thinking about your proposal but I'm not sure to do this this through udp. If you just don't broadcast all the time and don't expire nodes automatically you have exactly what you need in Zyre. If a node broadcasts other nodes can reply through unicast like done now in Zyre. If you don't want Zyre and want to do it zbeacon you would need to open an extra socket I guess. You could then also do it outside of zbeacon the same way Zyre uses zbeacon now. But anyways if we want this in zbeacon we need to open an extra socket. The only side effect I see is in how we would specify we want this behaviour in zbeacon or not? Only a master node would need to listen for unicast right? Rg, Arnaud -- w: http://www.sphaero.org t: http://twitter.com/sphaero g: http://github.com/sphaero i: freenode: sphaero_z25 ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] thread affinity
On 11/30/2013 10:02 AM, Min RK wrote: On Nov 30, 2013, at 9:30, Justin Karneges jus...@affinix.com wrote: Great, it sounds like the answer to my question is that it is possible to use the same socket from different threads provided I do my own locking. That's perfectly workable. I mainly wanted to be sure there wasn't something in libzmq explicitly preventing this kind of usage. For example, I believe it is impossible to share a socket across threads with pyzmq, but this must be a limitation imposed in the binding rather than in libzmq. Pyzmq imposes no restrictions on how you use sockets, so it is perfectly possible to share them across threads using locks (or not, and just risk segfault). Good to hear. I could have sworn I hit some attempt to use socket from thread it wasn't created in exception when I tried to do it (even with locking) but it's been awhile so I could be mistaking things. Justin ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
