Re: [zeromq-dev] Does ZeroMQ pay a lot in terms of marshalling compared to akka and clojure.core.async?
There's also the option to init messages from an existing buffer, which prevents memcpy overhead etc. often associated with marshalling. See : http://api.zeromq.org/3-2:zmq-msg-init-data This works pretty well with garbage collectors in most languages as well. Flag the String or similar object as in use before init, remove object flag in the callback function. On Thu, Oct 10, 2013 at 3:44 PM, Pieter Hintjens p...@imatix.com wrote: First off, don't ever believe anyone's claim of performance if you can try yourself. Second, marshalling can go from very expensive to very cheap but it depends on the data you use, and the marshalling technology. It's got nothing to do with ZeroMQ. However, with ZeroMQ you are free to study the cost of marshalling and choose or build the design that works best for you. Lastly, faster is very slippery. How large are your messages, how many do you send per second, to how many peers? -Pieter On Thu, Oct 10, 2013 at 3:22 PM, crocket crockabisc...@gmail.com wrote: Someone pointed out that ZeroMQ was great at communicaton between two or more languages but that for communications in one language, internal messaging solutions like akka and clojure.core.async would be a lot faster. He said marshalling was expensive. I looked into core.async, and I found it uses queues for asynchronicity. How would one compare core.async and ZeroMQ? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq socket monitor goes wrong?
Hi Ondra, Afaik you'd need to memcpy the data - from the event struct as it can be free'd from within libzmq in nondeterministic fashion and thus https://github.com/tchap/gozmq-utils/blob/master/parse_socket_event.c#L51-L52won't work. See the test cases in libzmq for an example : https://github.com/zeromq/libzmq/blob/master/tests/test_monitor.cpp#L55 On Wed, May 22, 2013 at 11:50 AM, Ondrej Kupka ondra@gmail.com wrote: Hi, I am working on zmq_socket_monitor bindings + some abstractions for Go. I am almost there, but I am getting very weird results when I run my tests (one large test for now to be honest). You can see the output at http://pastebin.com/kqgjLFyt. As you can see, only once out of ten runs of the tests, event addr is filled in properly. I am not sure what is happening, but I am pretty sure that I am getting this from libzmq (gozmq) and I am not breaking it anywhere in my Go. In any case, you can check sources at https://github.com/tchap/gozmq-utils, the C part is at https://github.com/tchap/gozmq-utils/blob/master/parse_socket_event.c If you find any silly mistake of mine, please let me know, otherwise it is probably a libzmq bug… Thanks in advance, Ondra Kupka ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Getting information about connected hosts, hitting high water mark, etc.
Hi Randall, There's events support built into the recent 3.2.x releases : http://api.zeromq.org/3-2:zmq-socket-monitor Currently there's no events bubbling up for HWM thresholds etc. On Tue, May 14, 2013 at 1:54 PM, Randall Nortman rn...@wonderclown.netwrote: While I'm complaining about stuff missing from the current API, let me mention this: It would be nice to be able to get at some information about what peers are currently connected to a socket, and how close each peer is to hitting its high water mark (i.e., how many queued messages). It would also be nice to get connect/disconnect and muted/unmuted notifications somehow. These could be reported via zmq_poll() (either as ZMQ_POLLERR or a new ZMQ_POLLPEERSTATUS or something like that), or if you want to get really fancy, report them back to the app via a special inproc PUB/SUB socket. I like the latter, but it's more like a high-level API while the rest of libzmq is currently more like a low-level socket API. A higher-level library could always build the latter on top of the former. By the way, my complaining is a form of flattery. I wouldn't bother complaining about something I didn't really want to use. ZMQ is pretty cool and has a lot of promise, it just has a little more maturing to do. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] czmq tarballs
Many thanks :-) On Tue, Apr 30, 2013 at 9:25 AM, Pieter Hintjens p...@imatix.com wrote: OK, I've made new tarballs for CZMQ 1.4.0, at http://czmq.zeromq.org/page:get-the-software -Pieter On Mon, Apr 29, 2013 at 11:30 PM, Pieter Hintjens p...@imatix.com wrote: Sounds like a good plan. -Pieter On Mon, Apr 29, 2013 at 10:31 PM, Lourens Naudé lour...@methodmissing.com wrote: Hi guys, We recently changed a binding against zeromq / czmq to prefer official tarball releases in order to drop autotools etc. as dependencies for the end user. I was wondering if we can perhaps push out a 1.4 stable tarball for czmq given there's the zbeacon_* API and a bunch of fixes and improvements since the 1.3.2 release circa December 2012 ... http://czmq.zeromq.org/page:get-the-software - Lourens ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] czmq tarballs
Hi guys, We recently changed a binding against zeromq / czmq to prefer official tarball releases in order to drop autotools etc. as dependencies for the end user. I was wondering if we can perhaps push out a 1.4 stable tarball for czmq given there's the zbeacon_* API and a bunch of fixes and improvements since the 1.3.2 release circa December 2012 ... http://czmq.zeromq.org/page:get-the-software - Lourens ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] Experimental TLS transport
Hi Guys, I synced my a TLS ( OpenSSL ) branch I've been working on ( well mostly off than on due to time constraints ) the last few months as I've hit a wall with some issues and was wondering if there's anyone in the community with experience with OpenSSL, specifically : * Async OpenSSL * OpenSSL + threads ( there's still a nondeterministic issue that can be reproduced with tests, but it's evading me ) * OpenSSL on windows I'm also not sure if any of this would apply moving forward as I noticed Pieter's working on a new auth and security layer and noticed the discussions around a newer ZMTP 3.0 spec ( http://hintjens.com/blog:39 ). Branch : https://github.com/methodmissing/libzmq/tree/tls Diff with current master : https://github.com/methodmissing/libzmq/compare/tls To build ( it's fully backwards compatible with master otherwise ) : ./configure --with-tls Pending items : * Haven't been tested on Windows * Depends on system OpenSSL installation * Support for tls:// in /perf * Default certificates ? * There's still some non-deterministic edge cases on connector shutdown despite the OpenSSL static locks implemented ( triggers100% CPU from within OpenSSL - mostly SSL_free ) Anyways - just wanted to put this out there to see if there's still an interest and if someone more knowledgeable could lend a hand ... - Lourens ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Changes to the monitoring message structure
Hi Guido, Looks good to me. - Lourens On Fri, Mar 8, 2013 at 11:33 AM, guido drop...@a-nugget.de wrote: Hi! Just recently I found the zmq_socket_monitor function and went Hoorray!. Already submitted a patch to simplify the zmq_event_t struct. And while thinking on how to get that feature into the python bindings I discovered that using pointers in messages is far from beeing the best solution for language bindings. I'd like to change the message structure in that a way that makes the the integeration into other languages a lot easier. Namely I'll introduce a second message frame for the variable length address (endpoint) part. In addition I'll make the binary parts fit to well known sizes. The event mask will be a 16-bit value and the error/fd/etc. thingy a 4 byte value. I'll make sure that there is no padding, so the values can be parsed easily. The new message structure will be the following: Frame 0 2-byte event id in native byte order 4-byte event value in native byte order Frame 1 variable length string containing the affected endpoint This change will also eliminate the need for the zmq_free_event callback, further simplifying the whole thing. Pull-Request will arrive today for public inspection :) Cheers Guido PS: Just in case someone misses the point: Please comment! ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] DTLS over ZeroMQ
Hi Pieter, I've spent some downtime working on the tis:// transport again and it's working for most topologies except PUB/SUB with subscription forwarding and PUSH/PULL isn't reliable just yet either. Thoughts on documentation for the transport, noting compliment socket types that's known to work and let folks have a go while leaving it open for feedback and community implementation of the failure cases ? - Lourens On Sat, Mar 2, 2013 at 12:33 PM, Pieter Hintjens p...@imatix.com wrote: On Sat, Mar 2, 2013 at 9:12 AM, Tom Cocagne tom.coca...@gmail.com wrote: I have a use case coming up that requires secure communication between two remote nodes that have an arbitrary number of message relayers sitting in between them. Very nice. I've started looking at the same use case and am making a CurveCP layer (on top of 0MQ, not below it). -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event Identified
Hi Doron, Or you could also use the ZMQ_FD socket option against zmq::socket_base_t::getsockopt (int option_, void *optval_, size_t *optvallen_) and not change the engine interface ... On Sun, Dec 9, 2012 at 11:01 PM, Doron Somech somdo...@gmail.com wrote: Hi, Because of security needs I need to know the ip address of each socket connecting to my system (matching identity to ip address). I'm using dealer-router, the router is the one doing the bind and the dealer connecting, i thought about adding a new event called Event Identified or something that when the identifier message arrived this event is called (maybe only for tcp) with the identifier and socket file descriptor. there a lot of other thing this feature can be used to, you can now deny anonymous identities (close the socket of any anonymous identity), just verify identity or manage a list of socket to identifier. Anyway I thought to implement this in the push_msg method of the session_base, the problem is that i don't have the socket file descriptor in the session_base class. I thought about adding GetFD to the IEngine interface and then the PushMsg will use that method to get the file descriptor (the method will return zero if engine is using socket). . Thoughts? Thanks, Doron ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event Identified
Correct - sorry, my bad. On Sun, Dec 9, 2012 at 11:10 PM, Doron Somech somdo...@gmail.com wrote: Hi, I'm not sure, i think (not sure) the ZMQ_FD is returning the mailbox socket (signaler) and not the tcp socket. On Mon, Dec 10, 2012 at 1:07 AM, Lourens Naudé lour...@methodmissing.comwrote: Hi Doron, Or you could also use the ZMQ_FD socket option against zmq::socket_base_t::getsockopt (int option_, void *optval_, size_t *optvallen_) and not change the engine interface ... On Sun, Dec 9, 2012 at 11:01 PM, Doron Somech somdo...@gmail.com wrote: Hi, Because of security needs I need to know the ip address of each socket connecting to my system (matching identity to ip address). I'm using dealer-router, the router is the one doing the bind and the dealer connecting, i thought about adding a new event called Event Identified or something that when the identifier message arrived this event is called (maybe only for tcp) with the identifier and socket file descriptor. there a lot of other thing this feature can be used to, you can now deny anonymous identities (close the socket of any anonymous identity), just verify identity or manage a list of socket to identifier. Anyway I thought to implement this in the push_msg method of the session_base, the problem is that i don't have the socket file descriptor in the session_base class. I thought about adding GetFD to the IEngine interface and then the PushMsg will use that method to get the file descriptor (the method will return zero if engine is using socket). . Thoughts? Thanks, Doron ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_socket_monitor wrong disconnect address
Fixed in https://github.com/zeromq/libzmq/pull/471 On Wed, Nov 14, 2012 at 12:50 AM, Pieter Hintjens p...@imatix.com wrote: Hi Balazs, This is a known issue I think. The code refers to strings which are freed. I suspect we'll have to store these in the socket object. -Pieter On Wed, Nov 14, 2012 at 7:46 AM, Balázs Varga bb.va...@gmail.com wrote: Hello, zmq_socket_monitor(3) disconnect event sometimes gives back a wrong address. I tried to create a test case, which seems to reproduce this issue, you can find it here: https://gist.github.com/4068920# regards balazs ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_socket_monitor wrong disconnect address
https://zeromq.jira.com/browse/LIBZMQ-384 is most definitely fixed, however I need to take another look @ https://zeromq.jira.com/browse/LIBZMQ-450 at the end of my day. On Wed, Nov 14, 2012 at 10:35 AM, Pieter Hintjens p...@imatix.com wrote: Lourens, Nice. I've backported that to 3.2rc. Can you confirm that it's these two issues: https://zeromq.jira.com/browse/LIBZMQ-450 https://zeromq.jira.com/browse/LIBZMQ-384 ? thanks Piete On Wed, Nov 14, 2012 at 6:52 PM, Lourens Naudé lour...@methodmissing.com wrote: Fixed in https://github.com/zeromq/libzmq/pull/471 On Wed, Nov 14, 2012 at 12:50 AM, Pieter Hintjens p...@imatix.com wrote: Hi Balazs, This is a known issue I think. The code refers to strings which are freed. I suspect we'll have to store these in the socket object. -Pieter On Wed, Nov 14, 2012 at 7:46 AM, Balázs Varga bb.va...@gmail.com wrote: Hello, zmq_socket_monitor(3) disconnect event sometimes gives back a wrong address. I tried to create a test case, which seems to reproduce this issue, you can find it here: https://gist.github.com/4068920# regards balazs ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] inproc: need to bind to an address before connect?
Hi Kahn-Chan, It's a feature - inproc transport doesn't supported unordered bind or connect. A map of endpoints are maintained per context - you'll get that error if and when there's no bound address when trying to connect. - Lourens On Thu, Oct 18, 2012 at 5:55 PM, Kah-Chan Low kahchan...@yahoo.com wrote: Hi, When I do this I will get a connection refused error: socket1.connect(inproc://abc); socket2.bind(inproc://abc); When I reverse the order of the statements everything works fine. Both sockets have been created with the same ZMQ context. Is this a feature or a bug? Thanks! KC ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_monitor
Hi Guys, Took a stab at reworking the monitoring infrastructure to address the issues noted in this thread thus far : https://github.com/methodmissing/libzmq/compare/fix-monitor Thoughts ? - Lourens On Fri, Sep 14, 2012 at 3:44 PM, Bjorn Reese bre...@mail1.stofanet.dkwrote: On 2012-09-13 01:14, Paul Colomiets wrote: So what are use cases for callback? One is to get notified when the exchange of identity has completed on non-blocking router-to-router connections. See the Router to router delay thread, dated 2012-07-28. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zmq_monitor
Hi Guys, There's another thread that touch on some of these points as well. I agree that a callback doesn't make much sense for higher level languages. It could be replaced with an inproc:// PUB socket, with event metadata being multipart messages published to that socket. I can take a stab @ this soon. Thoughts ? - Lourens On Wed, Sep 12, 2012 at 11:16 PM, Paul Colomiets p...@colomiets.namewrote: Hi Benjamin, On Thu, Sep 13, 2012 at 12:48 AM, MinRK benjami...@gmail.com wrote: Hello, We've been investigating adding support for the experimental new monitor functionality in pyzmq, and I have some questions. The callback is per-context, which makes no sense to me. If I wanted to monitor socket events on one socket out of one hundred, I have only two choices: make the callback aware of which socket(s) it cares about, and try to make minimize the disruption when called on the 99 sockets I don't care about (troublesome in Python, as acquiring the GIL from the io_thread is already problematic), or register monitored sockets with one Context, and non-monitored sockets with another. 1. All the conditions, that trigger monitor callback are not at hot places. 2. They are not intended for business logic, so the only thing you should probably do is find a name for the socket and log the message Can someone explain why the monitor is per-context and not per-socket? I saw on the list that it was added to the Context for a cleaner interface than cramming it into setsockopt. Is there any reason that zmq_ctx_set_monitor is preferable to zmq_socket_set_monitor, which would solve the exact same problem, without a fundamental change in how it is meant to work? Not that I support this implementation, but it's ok for me. For python implementation, I think you should provide something along the options: 1. Write monitoring message to another socket (possibly inproc) 2. Write message to the specified logger directly (as you need to hold GIL at the moment of writing, and python logging is quite heavyweight, this option may be implemented in another thread and inproc socket) 3. Collect number of occurrences of each of the event in the socket object, so it can be periodically inspected by main loop and sent to statistics collection software In my opinion it's wrong to provide callback interface for ZMQ_MONITOR in scripting languages. Thoughts? -- Paul ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Reporting make check error
Hi Lorin,
Regarding the monitor error, could you verify with a checkout of current
master of either of these repos (it's recently been fixed there ) ?
* https://github.com/zeromq/libzmq
* https://github.com/zeromq/zeromq3-x
The shutdown stress failure is common on OS X with default ulimit settings
( 256 open fd limit ). As per http://www.zeromq.org/docs:tuning-zeromq ,
run ulimit -n 1200
- Lourens
On Sun, Aug 19, 2012 at 8:01 AM, Sebastian Lauwers
sebastian.lauw...@gmail.com wrote:
Hi Lorin,
On 08/17/2012 04:56 PM, Lorin Sutton wrote:
Running on a MacBook Pro -- Mac OS X - 10.6.8 (4 GB RAM)
$ make check
Assertion failed: (memcmp (data_-accepted.addr, addr, 22)), function
socket_monitor, file test_monitor.cpp, line 44.
/bin/sh: line 1: 18108 Abort trap ${dir}$tst
FAIL: test_monitor
What version (or git commit) are you running? I was under the impression
this had been fixed recently.
test_shutdown_stress running...
Too many open files (signaler.cpp:388)
/bin/sh: line 1: 18147 Abort trap ${dir}$tst
FAIL: test_shutdown_stress
This is probably due to your OS configuration. Could you check the
values of ulimit -n?
-S.
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] FIX to 0mq bridge
Hey, I helped building a FIX 4.x compliant engine before and can help with some input ( we used the QuickFix specs / tests to ensure initial compliance as it covers a large set of edge cases in the FIX protocol ). I'd definitely recommend either parsing very lightweight / specific OR prefer the QuickFix parser and infer message semantics from that. If you're only interested in Market Data, a lightweight parser for the raw result messages could work best. Also note that formats differ depending on the type of data subscription - snapshot ( http://www.onixs.biz/fix-dictionary/4.4/msgType_W_87.html ) VS incremental ( http://www.onixs.biz/fix-dictionary/4.4/msgType_X_88.html ). A snapshot can be translated into incremental updates in the device etc. I'm not sure of the 5.x implementation - need to catch up. Just some thoughts. - Lourens On Fri, Aug 17, 2012 at 3:04 PM, gonzalo diethelm gdieth...@dcv.cl wrote: I have been trying to familiarize myself with the FIX protocol, specifically in relation with its usage to publish stock prices /volumes over a feed. I would think anybody with a “0mq point of view” (you know who you are) would conceive of the following idea: how about writing a dedicated device that swallows FIX messages on its front end and spits out those stock prices / volumes using a PUB socket on the back end. If I were to do this, I would probably use QuickFix to handle FIX connections, formats, etc. ** ** Does this makes sense? Has anybody seen anything like this? Anybody knows of any code (hopefully C++) that could be used as the basis for this beast? I realize this idea could be part of a commercial / closed source package, but just knowing someone else has walked this path before would be a nudge in the right direction. ** ** Thanks in advance and best regards. ** ** -- ** ** Gonzalo Diethelm DCV Chile ** ** -- Declaración de confidencialidad: Este Mensaje esta destinado para el uso de la o las personas o entidades a quien ha sido dirigido y puede contener información reservada y confidencial que no puede ser divulgada, difundida, ni aprovechada en forma alguna. El uso no autorizado de la información contenida en este correo podrá ser sancionado de conformidad con la ley chilena. Si usted ha recibido este correo electrónico por error, le pedimos eliminarlo junto con los archivos adjuntos y avisar inmediatamente al remitente, respondiendo este mensaje. Disclosure: This Message is to be used by the individual, individuals or entities that it is addressed to and may include private and confidential information that may not be disclosed, made public nor used in any way at all. Unauthorized use of the information in this electronic mail message may be subject to the penalties set forth by Chilean law. If you have received this electronic mail message in error, we ask you to destroy the message and its attached file(s) and to immediately notify the sender by answering this message. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event callback questions
Hi Bjorn, I agree with your feedback and think it's worthwhile splitting it out into message passing through a lazy initialized socket + inproc transport. It's trivial to pass pointers as messages as well. Traveling today and tomorrow but will create a defect @ JIRA and a patch from the other side for you to review. Might help to get rid of the va_args cruft also. Sounds like a plan ? Anyone else have any further input ? - Lourens On Tue, Aug 14, 2012 at 2:19 PM, Bjorn Reese bre...@mail1.stofanet.dkwrote: Any comments? On 2012-08-05 14:37, Lourens Naudé wrote: Hi Bjorn, Great feedback - digesting and will look into it. - Lourens On Sun, Aug 5, 2012 at 12:48 PM, Bjorn Reese bre...@mail1.stofanet.dk mailto:bre...@mail1.stofanet.dk wrote: The documentation on zmq_ctx_set_monitor() states that one should be careful with what you're doing in the callback function as excess time spent in the handle will block the socket's application thread. This does not appear to be fully correct. My investigation shows that the callback function can either be called by the application thread, e.g. for ZMQ_EVENT_LISTENING, or from a context thread, e.g. for ZMQ_EVENT_CONNECTED. It may be tempting for users to call zmq_connect() from the callback when we receive ZMQ_EVENT_DISCONNECTED, but this may deadlock as the event is called from the context thread. It might therefore be a good idea to insert a caveat in the documentation that the callback function should not call zmq functions. Alternative, we could construct a different callback mechanism that has a clear separation between application and library threads. Instead of a callback, we could create a monitor socket for the context, that is, an inproc socket to deliver the events. I know that this may sound like overengineering, but in my experience errors often occur when we use callbacks in a multithreaded program. Furthermore, the ZMQ_EVENT_* macros are given values that are a multiple of 2. Does this mean that the event parameter of the callback may contain two or more events in the same call? If not, why aren't the ZMQ_EVENT_* values simply consecutive numbers? Oh, and while I am at it, why is the event parameter an int rather than an unsigned int? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org mailto:zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Event callback questions
Hi Bjorn, Great feedback - digesting and will look into it. - Lourens On Sun, Aug 5, 2012 at 12:48 PM, Bjorn Reese bre...@mail1.stofanet.dkwrote: The documentation on zmq_ctx_set_monitor() states that one should be careful with what you're doing in the callback function as excess time spent in the handle will block the socket's application thread. This does not appear to be fully correct. My investigation shows that the callback function can either be called by the application thread, e.g. for ZMQ_EVENT_LISTENING, or from a context thread, e.g. for ZMQ_EVENT_CONNECTED. It may be tempting for users to call zmq_connect() from the callback when we receive ZMQ_EVENT_DISCONNECTED, but this may deadlock as the event is called from the context thread. It might therefore be a good idea to insert a caveat in the documentation that the callback function should not call zmq functions. Alternative, we could construct a different callback mechanism that has a clear separation between application and library threads. Instead of a callback, we could create a monitor socket for the context, that is, an inproc socket to deliver the events. I know that this may sound like overengineering, but in my experience errors often occur when we use callbacks in a multithreaded program. Furthermore, the ZMQ_EVENT_* macros are given values that are a multiple of 2. Does this mean that the event parameter of the callback may contain two or more events in the same call? If not, why aren't the ZMQ_EVENT_* values simply consecutive numbers? Oh, and while I am at it, why is the event parameter an int rather than an unsigned int? ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] zeromq 3.2 tests fails
Hi Guys, Please have a look at : * https://github.com/zeromq/zeromq3-x/pull/24 * https://zeromq.jira.com/browse/LIBZMQ-399#comment-15912 Works for me on OS X, but don't have access to a Linux ( or other OS ) from my current location. Mind taking it for a spin ? - Lourens On Fri, Aug 3, 2012 at 8:40 PM, Sebastian Lauwers sebastian.lauw...@gmail.com wrote: On 31 July 2012 09:00, Sebastian Lauwers sebastian.lauw...@gmail.com wrote: If this research is correct, then I guess two things could be done (again, feel free to add other possibilities if you see them, I'm shooting in the dark here): - Find a way to properly create the addr object before creating the session object, or, - Not use the session to get the address in string format. Thoughts? Anyone mind chipping in? I don't mind doing the development/hacking away at this, I'd rather not do anything which goes other people's mindset or isn't likely to be merged/accepted, though. -S. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] time for a 3.2 final?
Hi guys, Took at stab @ fixing LIBZMQ-409 with https://github.com/zeromq/zeromq3-x/pull/26 with a backport from Crossroads I/O (with some additions) . There's more context added as a comment to https://zeromq.jira.com/browse/LIBZMQ-409 . - Lourens On Sat, Aug 4, 2012 at 2:03 AM, Pieter Hintjens p...@imatix.com wrote: On Sat, Aug 4, 2012 at 4:32 AM, Stefan de Konink ste...@konink.de wrote: On Fri, 3 Aug 2012, Chuck Remes wrote: Judging by the critical issues on the wiki site, most of them are associated with 2.2. None of the listed tickets are for 3.2. Is it time to push 3.2rc1 to 3.2 final? I still want to provide a test for multipacket pubsub in 3.2. We are still unable to get any packets through when using an envelope + payload. There's also https://zeromq.jira.com/browse/LIBZMQ-409, which I think we need to address in 3.2 final. I'll cut an rc2, seems about time for that. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] time for a 3.2 final?
Also, zmq_poll() semantics changed from 2.x - 3.x for when the poll set's empty. Yes it's a non-sensical use case, but affect's czmq's zloop implementation and I'm sure there's other cases in the wild where users would get an error status now for what used to be a no-op / early return. https://github.com/zeromq/libzmq/pull/403 Thoughts ? On Sun, Aug 5, 2012 at 1:13 AM, Lourens Naudé lour...@methodmissing.comwrote: Hi guys, Took at stab @ fixing LIBZMQ-409 with https://github.com/zeromq/zeromq3-x/pull/26 with a backport from Crossroads I/O (with some additions) . There's more context added as a comment to https://zeromq.jira.com/browse/LIBZMQ-409 . - Lourens On Sat, Aug 4, 2012 at 2:03 AM, Pieter Hintjens p...@imatix.com wrote: On Sat, Aug 4, 2012 at 4:32 AM, Stefan de Konink ste...@konink.de wrote: On Fri, 3 Aug 2012, Chuck Remes wrote: Judging by the critical issues on the wiki site, most of them are associated with 2.2. None of the listed tickets are for 3.2. Is it time to push 3.2rc1 to 3.2 final? I still want to provide a test for multipacket pubsub in 3.2. We are still unable to get any packets through when using an envelope + payload. There's also https://zeromq.jira.com/browse/LIBZMQ-409, which I think we need to address in 3.2 final. I'll cut an rc2, seems about time for that. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] ZeroMQ 3.2.0 stable RC1 is now available
Hi Guys, Are there any interest in having some of the monitoring features backported to 2.x as well, given that most bindings are still dependent on that release ? - Lourens -- Forwarded message -- From: Pieter Hintjens piet...@gmail.com Date: Tue, Jun 5, 2012 at 10:18 AM Subject: [zeromq-dev] ZeroMQ 3.2.0 stable RC1 is now available To: zeromq-dev@lists.zeromq.org Hi all, We're pleased to announce that ZeroMQ 3.2.0 stable RC1 is now available for download at: * http://download.zeromq.org/zeromq-3.2.0.tar.gz (UNIX line endings) * http://download.zeromq.org/zeromq-3.2.0.zip (Windows line endings) This release is already in production and is considered highly stable. However, as usual before upgrading to a new version with many changes, test your apps carefully and under realistic conditions. A full list of changes is below. This release covers an astounding 354 commits. Thanks to the following, who helped make this happen: AJ Lewis, Ben Gray, Boris Gulay, Chuck Remes, Daniel Norberg, Douglas Young, Emmanuel Taurel, Ian Barber, Ivan Pechorin, Ivo Danihelka, Jos Decoster, Jtzl Forshizl, Lourens Naudé, Martin Hurton, Martin Lucina, Martin Sustrik, Mikko Koppanen, Patrick Trantham, Paul Colomiets, Pieter Hintjens, Ricardo Catalinas Jiménez, Rob Gagnon, Sergey Hripchenko, Max Skaller, Staffan Gimåker, Elliot Saba, and Steven McCoy. -Pieter Hintjens 0MQ version 3.2.0 (RC1), released on 2012/06/05 === Bug fixes - * Fixed issue 264 - Potential bug with linger, messages dropped during socket close. * Fixed issue 293 - libzmq doesn't follow the ZMTP/1.0 spec (did not set reserved bits to 0). * Fixed issue 303 - Assertion failure in pgm_sender.cpp:102. * Fixed issue 320 - Assertion failure in connect_session.cpp:96 when connecting epgm to an invalid endpoint. * Fixed issue 325 - Assertion failure in xrep.cpp:93, when two sockets connect using the same identity. * Fixed issue 327 - Assertion failure in mtrie.cpp:246, when unsubscribing from channel. * Fixed issue 346 - Assertion failure in signaler.cpp:155, when using a closed socket. * Fixed issue 328 - unsubscribe wrongly clears multiple subscriptions. * Fixed issue 330 - IPC listener does not remove unix domain stream file when terminated. * Fixed issue 334 - Memory leak in session_base.cpp:59. * Fixed issue 369 - ROUTER cannot close/reopen while DEALER connected. Operating systems - * Fixed issue 301 - HPUX 11iv2 - build fails, CLOCK_MONOTONIC undefined. * Fixed issue 324 - OS/X - build fails, ECANTROUTE undefined. * Fixed issue 368 - Solaris / Sun C++ - build fails, no insert method in multimap classes. * Fixed issue 366 - Windows - ports not freed after crash. * Fixed issue 355 - Windows - build fails, MSVC solution file is out of date. * Fixed issue 331 - FreeBSD 8 and 9 - getaddrinfo fails with EAI_BADFLAGS on AI_V4MAPPED flag. * Fixed issue xxx - Added support for WinCE. Performance --- * Fixed issue xxx - Implemented atomic operations for ARMv7a (runs 15-20% faster). API changes --- * Fixed issue 337 - Cleaned-up context API: zmq_ctx_new() - create new context (will deprecate zmq_init) zmq_ctx_destroy() - destroy context (will deprecate zmq_term) zmq_ctx_set() - set context property zmq_ctx_get() - get context property * Fixed issue xxx - Cleaned-up message API: zmq_msg_send() - send a message (will deprecate zmq_sendmsg) zmq_msg_recv() - receive a message (will deprecate zmq_recvmsg) zmq_msg_more() - indicate whether this is final part of message zmq_msg_get() - get message property zmq_msg_set() - set message property * Fixed issue xxx - Added context monitoring API: zmq_ctx_set_monitor() - configure monitor callback. * Fixed issue xxx - Added unbind/disconnect API: zmq_unbind() - unbind socket. zmq_disconnect() - disconnect socket. * Fixed issue xxx - Added ZMQ_TCP_ACCEPT_FILTER setsockopt() for listening TCP sockets. * Fixed issue 336 - Removed sys: transport. * Fixed issue 333 - Added zmq_device function back to API (was removed in 3.0). * Fixed issue 340 - Add support for MAX_SOCKETS to new context API. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] make check failed
It references my last commit - I'll debug and spawn a pull request to fix. On Thu, May 31, 2012 at 12:53 PM, Pieter Hintjens p...@imatix.com wrote: On Thu, May 31, 2012 at 1:18 PM, Ben Gray b...@benjamg.com wrote: I get the same thing under ubuntu 12.04 with g++ 4.6.3 using the libzmq master branch. Under gdb I get: ... OK, can someone log this as a critical issue, and we'll look at it? Thanks Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] make check failed
Hi Guys, References a race condition ( it's non-deterministic, sometimes succeeds, sometimes doesn't ) during socket and context termination for the shutdown stress test case. I gathered a lot of context from an Ubuntu instance and a fix is in the works. I think it'll make sense to log the critical issue as this is a hairy one from which we can all benefit from through collective knowledge transfer. Also noted by Martin in http://www.aosabook.org/en/zeromq.html - shutdown. Is. Hard. - Lourens On Thu, May 31, 2012 at 1:35 PM, Lourens Naudé lour...@methodmissing.comwrote: It references my last commit - I'll debug and spawn a pull request to fix. On Thu, May 31, 2012 at 12:53 PM, Pieter Hintjens p...@imatix.com wrote: On Thu, May 31, 2012 at 1:18 PM, Ben Gray b...@benjamg.com wrote: I get the same thing under ubuntu 12.04 with g++ 4.6.3 using the libzmq master branch. Under gdb I get: ... OK, can someone log this as a critical issue, and we'll look at it? Thanks Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] make check failed
Hi Brian, strace doesn't help much here as it traces system calls between the user process and the Kernel. The segfault's in user space and thus a stack trace is more helpful in tracking it down ( although system calls leading up to the segfault may help as well ). Note that tests are wrapper scripts and needs to be invoked with (g)libtool execute mode to run with gdb. On OS X : glibtool --mode=execute gdb test_shutdown_stress In your environment most likely : libtool --mode=execute gdb test_shutdown_stress This is what I got ( exhausted file descriptors, normal on Darwin ) : https://gist.github.com/2831352 - Lourens On Tue, May 29, 2012 at 11:46 PM, Brian Knox bri...@talksum.com wrote: Definitely still segfaulting (sorry, got swamped with work!) Did a rebuild, 1/17 tests failed, and manually run segfaults: $ ./test_shutdown_stress test_shutdown_stress running... [1]11992 segmentation fault ./test_shutdown_stress I've attached the output of strace -f ./test_shutdown_stress - not at the end it appears to exit with 0 again - but maybe the fuller trace will be helpful to something. Let me know if there's anything else I can do as far as gathering information! Brian On Tue, May 29, 2012 at 2:52 PM, Brian Knox bri...@talksum.com wrote: gah! that implies it's failing except when I strace it give me a bit. On Tue, May 29, 2012 at 1:42 PM, Pieter Hintjens p...@imatix.com wrote: On Tue, May 29, 2012 at 12:26 PM, Brian Knox bri...@talksum.com wrote: Pieter - just a heads up that I've verified a failing test on zmq head, on arch linux: I've attached a strace. Not sure how to read the trace, it looks fine and ends with exit (0). Doesn't fail on my 64-bit Ubuntu 11.10... :/ Could you post a normal stack trace? -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
[zeromq-dev] ZMQ_MONITOR socket option
Hey guys, I took a stab at implementing a socket option ( ZMQ_MONITOR ) that exposes changes in session state to a callback function. This is driven partly by requests for such a feature on the list as well as upcoming operations requirements for a commercial project. From the docs : ZMQ_MONITOR: Registers a callback for socket state changes ~~ Registers a callback function / event sink for changes in underlying socket state. Expected signature is `void (zmq_monitor_fn) (void *s, int event, zmq_event_data_t *data)` To remove the callback function call `zmq_setsockopt(socket, ZMQ_MONITOR, NULL, 0)` The default value of `NULL` means no monitor callback function. Supported events are : * 'ZMQ_EVENT_CONNECTED' - connection established * 'ZMQ_EVENT_CONNECT_DELAYED' - connection could not be established synchronously, it's being polled * 'ZMQ_EVENT_CONNECT_RETRIED' - asynchronous connect / reconnection attempt * 'ZMQ_EVENT_LISTENING' - socket bound to an address, ready to accept connections * 'ZMQ_EVENT_BIND_FAILED' - socket couldn't bind to an address * 'ZMQ_EVENT_ACCEPTED' - connection accepted to bound interface * 'ZMQ_EVENT_ACCEPT_FAILED' - could not accept client connection * 'ZMQ_EVENT_CLOSED' - connection closed * 'ZMQ_EVENT_CLOSE_FAILED' - connection couldn't be closed * 'ZMQ_EVENT_DISCONNECTED' - broken session See `zmq_event_data_t` and `ZMQ_EVENT_*` constants in zmq.h for event specific data (third argument to callback). Please note that both events and their context data aren't stable contracts. The 'ZMQ_MONITOR' socket option is intended for monitoring infrastructure / operations concerns only - NOT BUSINESS LOGIC. An event is a representation of something that happened - you cannot change the past, but only react to them. The implementation also only concerned with a single session. No state of peers, other sessions etc. are tracked - this will only pollute internals and is the responsibility of application authors to either implement or correlate in another datastore. Monitor events are exceptional conditions and are thus not directly in the messaging critical path. However, still be careful with what you're doing in the callback function as severe latency there will block the socket's application thread. Only tcp and ipc specific transport events are supported in this initial implementation. [horizontal] Option value type:: zmq_monitor_fn Option value unit:: N/A Default value:: no callback function Applicable socket types:: all Caveats : * Session state changes for tcp and ipc transport only ( those are much more volatile than inproc ) * No HWM or other mailbox events - they're tricky with distributor sockets ( PUSH, PUB ) * Callbacks are per socket and there's no effort towards any correlation and topology mapping - this can be done elsewhere. * Events and event arguments aren't stable contracts yet ( eg. exposing the underlying socket descriptor for some events are dangerous ) At this point I'm soliciting feedback / requirements from other environments and not spawning a formal pull request. Please review at https://github.com/methodmissing/libzmq/compare/master...events and circle back if and when there's a moment. - Lourens ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] req-rep with max delay, in Ruby
https://github.com/methodmissing/rbczmq/blob/master/test/test_socket.rb#L403-409( values are in milliseconds ) On Fri, Apr 20, 2012 at 1:51 PM, Joshua Foster jhaw...@gmail.com wrote: If the binding is able to use 2.2.0, you can set the recvTimeOut socket option instead of using the poller. Joshua On Apr 20, 2012, at 3:53 AM, Raphael Bauduin wrote: Hi, I need a process to send a request to a server, and wait a maximum of (let's say) 300 milliseconds. If no reply comes in this time span, a default action should be taken. Is using a poller the only solution to this? I'm using the ruby bindings (rbzmq), and didn't see any reference to pollers in the rdoc. Thanks Raph ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Message-on-expiration hacked into Redis
Hi Steffen, I've got similar experiments, but for all state changes, at https://github.com/methodmissing/redis/commits/zmq Here's some usage examples : https://gist.github.com/a767acd4ca8df0e9f528 - Lourens On Thu, Apr 5, 2012 at 6:59 AM, Steffen Mueller zer...@steffen-mueller.netwrote: Dear all, I thought you'd find this entertaining or possibly even useful as a fun-with-0MQ example: I hacked support for dispatching 0MQ PUB messages from a Redis server whenever a Redis key (string only for now) expires. It's quite limited functionality, should probably be changed to support all Redis types, and likely stands no chance of being incorporated upstream, though. It does what I need it to do (extremely high read/write load session cache which expires old sessions into a cold storage, a fat mysql, from where they can be promoted back into the hot cache if necessary). https://github.com/tsee/redis/tree/msg_on_expire One detail that I am still wondering about is the type of socket to use: - There will be many Redis instances on many servers (it's single-threaded and did I say there's extremely high read/write load?). - Eventhough the hot cache is not write-through (yay!), I expect a rather high insert load on mysql, so expect to require more than one worker that inserts expired sessions into the database. - All the Redis instances have slaves for resiliency. - This needs to be able to fail over elegantly without losing too much data. Thus = I will probably switch to PUSH in the expiry hook with multiple insert workers PULLing. = I could have a worker on the same machine as Redis, one worker per Redis instance, but while that would provide more than enough insert throughput, it doesn't allow batching of inserts as much for improved database load. = How can the 0MQ mesh of connections recover if a Redis instance fails over to the slave? == Redis should reconnect. But of a worker is added/removed, the worker needs to reconnect. Who should be binding instead of connecting? What's the general strategy for resilience and failover in such setups with 0MQ? BTW: Did I miss a bit of the 0MQ API that allows checking for dropped connections to trigger reconnects? --Steffen ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] BSON as high performance serialisation
I think it's more difficult to draw comparisons when one factors in the binding ecosystem as well - a large part of the community use libzmq through some higher level binding. Most serialization wrappers tend to create additional heap cruft that stresses the GC in some languages. Here's an interesting case study : * Deets : http://www.ohler.com/software/thoughts/Blog/Entries/2012/3/13_Need_for_Speed.html * Implementation : https://github.com/ohler55/oj So in summary, watch out for edges where micro benches of an implementation is fast, yet introduce a hidden cost in GC pressure relative to message volume which can destroy any soft realtime guarantees and overall system throughput / performance. I also think the topic should perhaps be taken off the list since libzmq does not impose message structure, BUT it's also important to keep tabs on this ( recommendations or real production feedback etc. ) somewhere on the wiki or docs for reference. - Lourens On Sat, Mar 31, 2012 at 11:27 PM, Wolfgang Richter w...@cs.cmu.edu wrote: On Sat, Mar 31, 2012 at 5:33 PM, Rick Olson technowee...@gmail.com wrote: How's BSON compare to msgpack? I've started using that in places. In my mind, it seems like the performance of BSON and msgpack could be comparable. msgpack's specification is more generic than BSON's (no MongoDB-specifics), and it seems to be a bit more well specified. In addition, msgpack doesn't require a string 'key' per message, and it's format seems to be more compact (space-efficient) than BSON. This might imply quicker encoding/decoding, although that could also be implementation-specific. msgpack looks really nice :-) Both seem simple enough to implement on your own (no external dependencies introduced which can be nice). However, msgpack seems nice because an ecosystem of software including RPC is built around it (although reinventing the communication layers which could be managed by 0MQ...). BSON's canonical implementation is the one included in MongoDB. I'd expect things like msgpack to win in the long run (unless the BSON spec is changed to be simpler+more generic; it's already simpler than msgpack though) because they are more generic. ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] polling for messages
Hi Alex, That'll be fine - the fd events are edge-triggered and would work with any reactor implementation. Just remember to read until completion as the notification can be for more than one message (zmq protocol is framed) as well. Failure to do so will result in further notifications not being triggered. - Lourens On Monday, March 26, 2012, Pieter Hintjens piet...@gmail.com wrote: Yes, it should work fine. On Mar 26, 2012 11:44 AM, Alex Zeffertt azeffe...@cambridgesys.com wrote: Hi list, I'm considering using czmq in my single threaded application. However, the application is already using an event driven framework in which a file descriptor has to be registered along with a callback that is invoked when it becomes readable. My question is: can I use the file descriptor returned by zsockopt_fd() in this way, or would this break things when I issue zframe_recv_nowait()? Thanks in advance, Alex ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Request for co-maintainer(s) for CZMQ
Hi Pieter, I am the author of a Ruby binding against czmq ( https://github.com/methodmissing/rbczmq ), am very familiar with the API and have an ongoing interest in it being compatible with libzmq and libxs releases. I would be open to maintain / review and manage process around contribution. - Lourens On Tue, Mar 20, 2012 at 6:36 PM, Pieter Hintjens p...@imatix.com wrote: Hi all, This is a request for one or more people to volunteer as co-maintainers of CZMQ. Note that the job requires just the ability to merge pull requests and respect process. You don't actually have to write or review code. The goal is to get CZMQ conformant with the C4 process (http://rfc.zeromq.org/spec:16) which requires that a project have 2+ maintainers. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] Request for co-maintainer(s) for CZMQ
Hi Pieter, Github handle: methodmissing - Lourens On Tue, Mar 20, 2012 at 10:20 PM, Pieter Hintjens p...@imatix.com wrote: Lourens, Great, what's your github login? -Pieter On Tue, Mar 20, 2012 at 4:35 PM, Lourens Naudé lour...@methodmissing.com wrote: Hi Pieter, I am the author of a Ruby binding against czmq ( https://github.com/methodmissing/rbczmq ), am very familiar with the API and have an ongoing interest in it being compatible with libzmq and libxs releases. I would be open to maintain / review and manage process around contribution. - Lourens On Tue, Mar 20, 2012 at 6:36 PM, Pieter Hintjens p...@imatix.com wrote: Hi all, This is a request for one or more people to volunteer as co-maintainers of CZMQ. Note that the job requires just the ability to merge pull requests and respect process. You don't actually have to write or review code. The goal is to get CZMQ conformant with the C4 process (http://rfc.zeromq.org/spec:16) which requires that a project have 2+ maintainers. -Pieter ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
Re: [zeromq-dev] xpub.cpp line 82
Hi Guys, I've been using Travis CI ( http://travis-ci.org/ ) for a Ruby binding that wraps czmq ( https://github.com/methodmissing/rbczmq ) and also bundles both libzmq and czmq as vendor'ed tarballs. Both frameworks build without any issues https://github.com/methodmissing/rbczmq/blob/master/ext/rbczmq/extconf.rb#L45-57 . Their VMs are Ubuntu based, with various build and notification hooks supported. I can get pull requests up for Travis CI integration against both zeromq and czmq repositories if there's enough votes in favor of using it as a CI environment. A local environment can easily be bootstrapped with the following tool and Chef cookbooks as well : * https://github.com/travis-ci/travis-cookbooks/tree/master/ci_environment * https://github.com/michaelklishin/sous-chef - Lourens On 2012/02/03, at 16:48, Pieter Hintjens wrote: We don't need to... Jenkins builds daily and failures are rapidly fixed. -Pieter On Feb 3, 2012 2:42 PM, Ian Barber ian.bar...@gmail.com wrote: On Fri, Feb 3, 2012 at 1:01 PM, Martin Lucina mar...@lucina.net wrote: Huh? How did that happen? -mato As part of the work by Daniel Norberg to free received sub messages. Looking at the rest of the patch it seems OK to me, but I guess it wasn't built as is. Not sure how we enforce that kind of thing on pull reqs without a heavier process though. Ian ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev ___ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
