That's *wonderful*, thanks.
On Wed, Mar 16, 2016 at 11:09 AM, MinRK <[email protected]> wrote: > > > On Wed, Mar 16, 2016 at 11:06 AM, Pieter Hintjens <[email protected]> wrote: >> >> Off-topic, but I need to ask, how do you do the syntax highlighting in >> emails? > > > http://markdown-here.com/ > >> >> >> On Mon, Mar 14, 2016 at 4:42 PM, MinRK <[email protected]> wrote: >> > On Sat, Mar 12, 2016 at 9:00 PM, Dave Kuhlman <[email protected]> >> > wrote: >> >> >> >> For those of us who use Python and the pyzmq library for ZeroMQ, I'm >> >> wondering whether the solution to the problem and need for thread >> >> safety is to avoid threads and to use either the tornado ioloop >> >> support or asyncio ioloop. Am I right that doing so would give a >> >> Python developer the capabilities of multiple threads without the >> >> worries about thread safety. >> >> >> >> One reason I'm asking is that I'm the one who added the ioloop >> >> examples to zguide (in examples/Python/tornado_ioloop and >> >> examples/Python/ascyncio_ioloop). A number of those examples use >> >> (what I understand as) multiple concurrent tasks within a single >> >> thread. So, I'm wondering whether that really is thread safe (or >> >> whatever the equivalent is for tasks), as I believe it is. >> > >> > Yup, asyncio/tornado are single-threaded eventloops, so you won’t get >> > threadsafety problems if everything happens in a callback/coroutine on >> > those >> > loops. You can also use these to make threadsafe actions, even when >> > using >> > actual threads. In the tornado case, there is one (and only one) >> > threadsafe >> > method, and it’s IOLoop.add_callback. You can use this to schedule calls >> > to >> > be made on the IOLoop, from other threads. For instance: >> > >> > loop = start_loop_in_background_thread() >> > >> > def threadsafe_send(socket, msg): >> > loop.add_callback(lambda : socket.send_multipart(msg)) >> > >> > In this way, you can schedule sends from any number of threads perfectly >> > safely, because the actual zmq socket is only ever operated on in the IO >> > thread. Threadsafe recv is similarly doable, but a bit more code. >> > >> > On the other hand, you can also write coroutine-unsafe code if you have >> > yields in the middle of sending multipart messages, for instance: >> > >> > @coroutine >> > def bad_coroutine_send(socket, msg): >> > for frame in msg[:-1]: >> > yield socket.send(frame, zmq.SNDMORE) >> > yield socket.send(msg[-1]) >> > >> > Where each yield is an opportunity to take over and do some other >> > operation >> > on the socket. I think it is a lot easier to write coroutine-safe code >> > than >> > thread-safe code, since all context switching is explicit (make sure you >> > have no yield calls in the middle of an operation). >> > >> > -MinRK >> >> >> >> >> >> Any enlightenment will be welcome. >> >> >> >> And, by the way, I suspect that something equivalent could be done >> >> in Node.js (JavaScript), which also uses a single threaded approach >> >> to "concurrency" (psuedo-concurrency, quasi-concurrency, whatever). >> >> In Node.js, you could use callbacks, but there are libraries that >> >> can wrap your code to make is seem more simple and parallel. For >> >> example, see async.js. Here are a few quick examples from the >> >> async.js Web page (https://www.npmjs.com/package/async): >> >> >> >> async.map([`file1`,`file2`,`file3`], fs.stat, function(err, >> >> results){ >> >> // results is now an array of stats for each file >> >> }); >> >> >> >> async.filter([`file1`,`file2`,`file3`], fs.exists, >> >> function(results){ >> >> // results now equals an array of the existing files >> >> }); >> >> >> >> async.parallel([ >> >> function(){ ... }, >> >> function(){ ... } >> >> ], callback); >> >> >> >> async.series([ >> >> function(){ ... }, >> >> function(){ ... } >> >> ]); >> >> >> >> As a bit of justification for this question, the main intension of >> >> the Python ioloop support in tornado and asyncio is to address >> >> problems of network and I/O delays and to enable our code to do >> >> something useful during those delays. That seems like a central >> >> concern for users of ZeroMQ. >> >> >> >> Dave >> >> >> >> >> >> On Sat, Mar 12, 2016 at 03:00:58AM +0200, Doron Somech wrote: >> >> > Hi All, >> >> > During the ZeroMQ hackathon I have added the Radio/Dish pattern >> >> > (thread >> >> > safe version of pubsub) and multicast support (only for Radio/Dish >> >> > sockets) to both libzmq and czmq. >> >> > Radio/Dish is very similar to pub/sub with some differences: >> >> > * Thread safe, you can send and receive messages from multiple >> >> > threads, so for publisher you now don't need internal device >> >> > to >> >> > publish from multiple threads and with subscriber you can use >> >> > it >> >> > as >> >> > load balancer. >> >> > * Only single frame can be sent >> >> > * Radio/Dish matching is exact matching (vs prefix of pubsub) >> >> > * Group (Radio/Dish topic) is string and currently limited to 15 >> >> > chars (might be increased in the future) >> >> > * Group is set as part of the zmq_msg (libzmq) or zframe (czmq). >> >> > * You call join or leave on the socket (with methods instead of >> >> > socket options) >> >> > >> >> > To learn how to use it with czmq take a loot at the following >> >> > gist: >> >> > [1]https://gist.github.com/somdoron/9423196a228775c8f5af >> >> > For libzmq take a loot at the test: >> >> > >> >> > >> >> > [2]https://github.com/zeromq/libzmq/blob/master/tests/test_radio_dish.c >> >> > pp#L90 >> >> > And the most exciting news, with radio-dish you can use multicast >> >> > transport, just replace the address from following gist with udp >> >> > transport and multicast address, following is a valid address for >> >> > example: >> >> > udp://[3]239.0.0.1:55555 >> >> > Regards >> >> > Doron >> >> > >> >> > References >> >> > >> >> > 1. https://gist.github.com/somdoron/9423196a228775c8f5af >> >> > 2. >> >> > >> >> > https://github.com/zeromq/libzmq/blob/master/tests/test_radio_dish.cpp#L90 >> >> > 3. http://239.0.0.1:55555/ >> >> >> >> > _______________________________________________ >> >> > zeromq-dev mailing list >> >> > [email protected] >> >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> >> >> >> >> -- >> >> >> >> Dave Kuhlman >> >> http://www.davekuhlman.org >> >> _______________________________________________ >> >> zeromq-dev mailing list >> >> [email protected] >> >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > >> > >> > _______________________________________________ >> > zeromq-dev mailing list >> > [email protected] >> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> > >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev > > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
