Re: [zeromq-dev] Limit socket queue size
21.03.2012 20:35, Johnny Gozde написал: On Wed, Mar 21, 2012 at 3:56 AM, Boris Gulay bo...@boressoft.ru mailto:bo...@boressoft.ru wrote: I see that latest version differs very match from 2.2.2. If I want to make some patches can I make it for stable version? Yes, you can make patches against the stable version. This will be supported for a long time. If the patches make sense for the unstable version, I'll apply them there as well. https://github.com/zeromq/clrzmq/pull/58 signature.asc Description: OpenPGP digital signature ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] assertion in mailbox.cpp:84
Hi everybody, In my program I've got an assertion in mailbox.cpp:84 (zeromq-2.1.11). Could you explain me why this might hapenning? I use singlethreaded application under Windows 7 OS. The application repeatedly sends data through PUB socket and never receives any messages. -Mikhail Navernyuk ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] REQ/REP for possibly async comm?
When you need asynchronous communication you better use ROUTER/DEALER than REQ/REP, the only time I may want to use a REQ socket is to write a simple client taking to a server using a ROUTER socket. Except that I am still wondering why would anyone use REP/REQ given its limitations for anything serious, I think it takes too many assumptions that everything will work as expected which rarely happens in a real use case. I never tried ROUTER-ROUTER though but that's one of the things I love with zeromq, you can really build interesting topologies without the need to concentrate on the low level implementation. On 28 March 2012 18:27, Andrei Zmievski and...@zmievski.org wrote: Okay, will give that a try. Is there a reason why I couldn't use PUB/SUB sockets for this interaction? -Andrei On Tue, Mar 27, 2012 at 6:37 PM, Michel Pelletier pelletier.mic...@gmail.com wrote: If not getting a timely response is an exceptional condition, then you can throw away your REQ socket, create a new REQ socket, connect it and proceed from there as if it were new. If it's a common occurrence then you can do ROUTER-ROUTER and track the state of requests yourself. REQ and REP enforce a simple state, if you need a more complex state tracking, then you can implement it yourself with ROUTER. The way I think of ROUTER is exactly like the hardware device of the same name, you tell a router's send() where to send the data, when you recv(), it tells you where it came from. given that information you can track your own requests and timeouts for every given connected peer yourself in a zmq_poll() loop. -Michel On Tue, Mar 27, 2012 at 4:35 PM, Andrei Zmievski and...@zmievski.org wrote: Michel, Thanks for the example. However, from what I understand, this pattern does not allow process A to re-send the request in case it has not received the response after a certain amount of time, because REQ sockets don't allow multiple messages? -Andrei You can use non-blocking ROUTER on a poll loop and REQ or REP on the other end, that way you can handle requests and their responses as they come and as you compute them, respectively. Whether to use REQ or REP only matters for whichever end starts the conversation. The LRU pattern documented in the guide has a good implementation of this pattern with REQ-ROUTER-REP. It doesn't sound like you need the actual LRU part of the pattern, just the ROUTER-REQ part. See the code for an example how to use a ROUTER socket in non-blocking mode in a poll loop to handle out of order patterns like A then B requests but you respond B then A. Start here and read it a few times. :) http://zguide.zeromq.org/page:all#Least-Recently-Used-Routing-LRU-Pattern -Michel ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] REQ/REP for possibly async comm?
On Mar 30, 2012, at 7:04 AM, Schmurfy wrote: When you need asynchronous communication you better use ROUTER/DEALER than REQ/REP, the only time I may want to use a REQ socket is to write a simple client taking to a server using a ROUTER socket. Except that I am still wondering why would anyone use REP/REQ given its limitations for anything serious, I think it takes too many assumptions that everything will work as expected which rarely happens in a real use case. I think the main use-case for REQ/REP is to use it to communicate within a single thread (via inproc). It can simplify some communications. As soon as you introduce a second thread (or another process) there is the possibility of failure, so REQ/REP becomes less useful. In those cases I almost always use DEALER/ROUTER. cr ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] assertion in mailbox.cpp:84
On Fri, Mar 30, 2012 at 10:50 AM, Mikhail Navernyuk mike.nav...@gmail.comwrote: The application repeatedly sends data through PUB socket and never receives any messages. -Mikhail Navernyuk Can you create a small programme that repeatably demonstrates the issue? Ideally in C. It'll make it a lot easier to track down issue rather than just looking at the specific assert. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Backport of ZMQ_RCVTIMEO/ZMQ_SNDTIMEO to 2.1.x
Does anyone have any problem if I back port the send/recv time out socket options to 2.1.x? Joshua ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] REQ/REP for possibly async comm?
I suppose you could use PUB/SUB with subscriptions, I don't know enough about your topology or message patterns to be sure. The advantage would be if you needed to do 1 to N transmission or messages or N to 1 reception. -Michel On Wed, Mar 28, 2012 at 9:27 AM, Andrei Zmievski and...@zmievski.org wrote: Okay, will give that a try. Is there a reason why I couldn't use PUB/SUB sockets for this interaction? -Andrei On Tue, Mar 27, 2012 at 6:37 PM, Michel Pelletier pelletier.mic...@gmail.com wrote: If not getting a timely response is an exceptional condition, then you can throw away your REQ socket, create a new REQ socket, connect it and proceed from there as if it were new. If it's a common occurrence then you can do ROUTER-ROUTER and track the state of requests yourself. REQ and REP enforce a simple state, if you need a more complex state tracking, then you can implement it yourself with ROUTER. The way I think of ROUTER is exactly like the hardware device of the same name, you tell a router's send() where to send the data, when you recv(), it tells you where it came from. given that information you can track your own requests and timeouts for every given connected peer yourself in a zmq_poll() loop. -Michel On Tue, Mar 27, 2012 at 4:35 PM, Andrei Zmievski and...@zmievski.org wrote: Michel, Thanks for the example. However, from what I understand, this pattern does not allow process A to re-send the request in case it has not received the response after a certain amount of time, because REQ sockets don't allow multiple messages? -Andrei You can use non-blocking ROUTER on a poll loop and REQ or REP on the other end, that way you can handle requests and their responses as they come and as you compute them, respectively. Whether to use REQ or REP only matters for whichever end starts the conversation. The LRU pattern documented in the guide has a good implementation of this pattern with REQ-ROUTER-REP. It doesn't sound like you need the actual LRU part of the pattern, just the ROUTER-REQ part. See the code for an example how to use a ROUTER socket in non-blocking mode in a poll loop to handle out of order patterns like A then B requests but you respond B then A. Start here and read it a few times. :) http://zguide.zeromq.org/page:all#Least-Recently-Used-Routing-LRU-Pattern -Michel ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] compilation errors
Al, I apologize since this is probably more an autotool/gcc issue rather then anything with zmq but I am hoping somebody run into similar troubles and might offer some help. I am trying to compile zmq on a new box: Windows 7, Cygwin 1.7, gcc (GCC) 4.5.3 and I get errors with the visibility attribute: $ make all-am make[2]: Entering directory `/home/me/Software/Zmq/src' CXXlibzmq_la-address.lo cc1plus: warnings being treated as errors address.cpp: In constructor 'zmq::address_t::address_t(const std::string, const std::string)': address.cpp:34:1: error: visibility attribute not supported in this configuration; ignored address.cpp: At global scope: address.cpp:52:1: error: visibility attribute not supported in this configuration; ignored address.cpp:52:1: error: visibility attribute not supported in this configuration; ignored Makefile:612: recipe for target `libzmq_la-address.lo' failed I was trying to set CPPFLAGS=-march=native -fvisibility=hidden in my environment so the Makefile(s) end up with CPPFLAGS = -pedantic -Werror -Wall -D_GNU_SOURCE -D_REENTRANT -D_THREAD_SAFE -march=native -fvisibility=hidden -DZMQ_FORCE_SELECT but compilation still fails with the dame error. Does anybody have some suggestions here? Thank you very much, -- Bernd ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Limit socket queue size
On Fri, Mar 30, 2012 at 2:41 AM, Boris Gulay bo...@boressoft.ru wrote: Yes, you can make patches against the stable version. This will be supported for a long time. If the patches make sense for the unstable version, I'll apply them there as well. https://github.com/zeromq/clrzmq/pull/58 I merged the pull request since it had been there for a day without comment. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Backport of ZMQ_RCVTIMEO/ZMQ_SNDTIMEO to 2.1.x
On Fri, Mar 30, 2012 at 10:26 AM, Joshua Foster jhaw...@gmail.com wrote: Does anyone have any problem if I back port the send/recv time out socket options to 2.1.x? Sounds like a great idea. We'll have to modify the ABI version. This may impact bindings. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Backport of ZMQ_RCVTIMEO/ZMQ_SNDTIMEO to 2.1.x
Looking through the code, much of it has already been back ported as part of issue 231. Are we still using the https://github.com/zeromq/zeromq2-1 repository for the 2.1.x development? Joshua On Mar 30, 2012, at 11:55 AM, Pieter Hintjens wrote: On Fri, Mar 30, 2012 at 10:26 AM, Joshua Foster jhaw...@gmail.com wrote: Does anyone have any problem if I back port the send/recv time out socket options to 2.1.x? Sounds like a great idea. We'll have to modify the ABI version. This may impact bindings. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] REQ/REP for possibly async comm?
ROUTER-ROUTER seemed to work. I just set the socket identity on process B to something known and then process A uses that to send the messages. The socket in process A is anonymous. -Andrei On Fri, Mar 30, 2012 at 8:27 AM, Michel Pelletier pelletier.mic...@gmail.com wrote: I suppose you could use PUB/SUB with subscriptions, I don't know enough about your topology or message patterns to be sure. The advantage would be if you needed to do 1 to N transmission or messages or N to 1 reception. -Michel On Wed, Mar 28, 2012 at 9:27 AM, Andrei Zmievski and...@zmievski.org wrote: Okay, will give that a try. Is there a reason why I couldn't use PUB/SUB sockets for this interaction? -Andrei On Tue, Mar 27, 2012 at 6:37 PM, Michel Pelletier pelletier.mic...@gmail.com wrote: If not getting a timely response is an exceptional condition, then you can throw away your REQ socket, create a new REQ socket, connect it and proceed from there as if it were new. If it's a common occurrence then you can do ROUTER-ROUTER and track the state of requests yourself. REQ and REP enforce a simple state, if you need a more complex state tracking, then you can implement it yourself with ROUTER. The way I think of ROUTER is exactly like the hardware device of the same name, you tell a router's send() where to send the data, when you recv(), it tells you where it came from. given that information you can track your own requests and timeouts for every given connected peer yourself in a zmq_poll() loop. -Michel On Tue, Mar 27, 2012 at 4:35 PM, Andrei Zmievski and...@zmievski.org wrote: Michel, Thanks for the example. However, from what I understand, this pattern does not allow process A to re-send the request in case it has not received the response after a certain amount of time, because REQ sockets don't allow multiple messages? -Andrei You can use non-blocking ROUTER on a poll loop and REQ or REP on the other end, that way you can handle requests and their responses as they come and as you compute them, respectively. Whether to use REQ or REP only matters for whichever end starts the conversation. The LRU pattern documented in the guide has a good implementation of this pattern with REQ-ROUTER-REP. It doesn't sound like you need the actual LRU part of the pattern, just the ROUTER-REQ part. See the code for an example how to use a ROUTER socket in non-blocking mode in a poll loop to handle out of order patterns like A then B requests but you respond B then A. Start here and read it a few times. :) http://zguide.zeromq.org/page:all#Least-Recently-Used-Routing-LRU-Pattern -Michel ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Backport of ZMQ_RCVTIMEO/ZMQ_SNDTIMEO to 2.1.x
Yes, that's the repo. Note that every change needs to be backed by an issue and a test case. Apart from the can we change the ABI version number question, it seems a good investment in 2.1. So my suggestion is, make an issue, make a test case, then make a pull request (please also update the ABI version number). I'll test it and then merge it to libzmq2-1 master. -Pieter On Fri, Mar 30, 2012 at 11:01 AM, Joshua Foster jhaw...@gmail.com wrote: Looking through the code, much of it has already been back ported as part of issue 231. Are we still using the https://github.com/zeromq/zeromq2-1 repository for the 2.1.x development? Joshua On Mar 30, 2012, at 11:55 AM, Pieter Hintjens wrote: On Fri, Mar 30, 2012 at 10:26 AM, Joshua Foster jhaw...@gmail.com wrote: Does anyone have any problem if I back port the send/recv time out socket options to 2.1.x? Sounds like a great idea. We'll have to modify the ABI version. This may impact bindings. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Backport of ZMQ_RCVTIMEO/ZMQ_SNDTIMEO to 2.1.x
Does adding to the API/ABI really make sense at this late point in the 2.1.x series? This will be the first such change since the first stable 2.1 release, as far as I know. I thought patch releases were only meant to contain bugfixes, etc., not API expansion. It's fine if API/ABI are officially not fixed per major.minor, but that seems to have been practice until this point, even if only by coincidence. -MinRK On Fri, Mar 30, 2012 at 17:21, Pieter Hintjens p...@imatix.com wrote: Yes, that's the repo. Note that every change needs to be backed by an issue and a test case. Apart from the can we change the ABI version number question, it seems a good investment in 2.1. So my suggestion is, make an issue, make a test case, then make a pull request (please also update the ABI version number). I'll test it and then merge it to libzmq2-1 master. -Pieter On Fri, Mar 30, 2012 at 11:01 AM, Joshua Foster jhaw...@gmail.com wrote: Looking through the code, much of it has already been back ported as part of issue 231. Are we still using the https://github.com/zeromq/zeromq2-1 repository for the 2.1.x development? Joshua On Mar 30, 2012, at 11:55 AM, Pieter Hintjens wrote: On Fri, Mar 30, 2012 at 10:26 AM, Joshua Foster jhaw...@gmail.com wrote: Does anyone have any problem if I back port the send/recv time out socket options to 2.1.x? Sounds like a great idea. We'll have to modify the ABI version. This may impact bindings. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] BSON as high performance serialisation
Interesting tidbit from a YouTube presentation, *Serialization formats* - no matter which one you use, they are all expensive. Measure. Don’t use pickle. Not a good choice. Found protocol buffers slow. They wrote their own BSON implementation which is 10-15 time faster than the one you can download. http://highscalability.com/blog/2012/3/26/7-years-of-youtube-scalability-lessons-in-30-minutes.html BSON is an initialism for Binary-JSON, http://en.wikipedia.org/wiki/BSON -- Steve-o ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Some real perfomance numbers from real workloads
I'd be interested in hearing some of the numbers the community has gotten for its real workloads. Right now I work exclusively with customers leveraging 0MQ on Windows and under real workloads there we've experienced anywhere from 300K to 1.5 million messages a second throughput (the 1.5 million was for 40 byte messages). Latency has always been sub ½ second and for the folks I've worked with that's good enough so I have not spent too much time measuring nanosecond latency on Windows. I'd love to hear from Linux folks or even other Windows folks what their numbers look like as well as folks working with the various language bindings... Basically what I'm trying to do is get some idea of what the expectation is from the community when they utilize 0MQ. Of particular interest is anyone that has a large number of concurrent subscribers. I haven't done any 0mq work yet with say 500 concurrent subscribers. I'm not really interested in test harnesses or bench suites etc. I'm looking for actual workload #'s. Thanks!. --CM ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
