Re: [zeromq-dev] ZeroMQ 4.2 release, planning

[Luca Boccassi]

On Tue, 2016-09-27 at 09:41 +0300, Doron Somech wrote:
> Sorry for the late response, increasing the msg_t structure will be
> great, however this will require changing a lot of binding.

I think I remember we need it for the new socket types, is that correct?

There is a large performance penalty (intuitively due to not fitting
into a single cache line anymore, but haven't ran perf/cachegrind), and
the throughput with vsm type messages goes down by 4% (min) and 20%
(max) for TCP, and 36% (min) 38 (max) for inproc, which is quite a lot,
so we need to be sure it's worth it.

Regarding the bindings, after a quick search on the Github org, I could
only see:

https://github.com/zeromq/lzmq/blob/master/src/lua/lzmq/ffi/api.lua#L144
https://github.com/zeromq/clrzmq4/blob/master/lib/zmq.cs#L28
https://github.com/zeromq/pyczmq/blob/master/pyczmq/zmq.py#L177

Other bindings just import zmq.h. Did I miss any?

> Sorry for disappearing, baby and full time job is a lot :-), hopefully
> I'm back...

No worries, perfectly understandable :-)

> On Mon, Aug 29, 2016 at 6:46 PM, Luca Boccassi  
> wrote:
> > Sorry, I meant if we go with (1), not (2), we might bump the size as
> > well, since we are already doing another ABI-breaking change.
> >
> > I agree on the solution as well.
> >
> > On Mon, 2016-08-29 at 17:12 +0200, Pieter Hintjens wrote:
> >> I'm confused between the (1) and (2) choices, and can't see where
> >> bumping the message size fits.
> >>
> >> Nonetheless, I think bumping the size, fixing the alignment issues,
> >> and bumping the ABI version is the best solution here.
> >>
> >> On Fri, Aug 26, 2016 at 12:33 PM, Luca Boccassi  
> >> wrote:
> >> > I've given some more thoughts and testing to the alignment issue. I can
> >> > reproduce the problem by enabling alignment checks on x86 too.
> >> >
> >> > But most importantly, I think we cannot get away from bumping the ABI
> >> > with this fix, however we rearrange it, simply because applications need
> >> > to be rebuilt against the new header to be fixed. A simple rebuild of
> >> > the libzmq.so is not enough. And the way to do this is to bump the ABI
> >> > so that distros can schedule transitions and rebuilds and so on.
> >> >
> >> > So the choice list is now restricted to:
> >> >
> >> > 1) Bump ABI
> >> > 2) Revert the fix and leave everything broken on sparc64 and some
> >> > aarch64 (rpi3 seems not to be affected, must depend on the SoC flavour)
> >> >
> >> > If we go with 2, we might as well get 2 birds with one stone and bump
> >> > the zmq_msg_t size to 128 as we have talked about in the past.
> >> >
> >> > Doron, this would help with the new UDP based socket types right?
> >> >
> >> > Pros of bumping msg size:
> >> >
> >> > - we can get rid of the malloc() in the lmsg type case as all the data
> >> > will fit
> >> >
> >> > Cons:
> >> >
> >> > - for the vsm/cmsg type cases (for most architectures anyway) it won't
> >> > fit anymore into a single cacheline
> >> >
> >> > Given all this, I'd say we should go for it.
> >> >
> >> > Opinions?
> >> >
> >> > On Sat, 2016-08-13 at 16:59 +0100, Luca Boccassi wrote:
> >> >> Hello,
> >> >>
> >> >> Trying to give some thoughts again on the libzmq 4.2 release. It's
> >> >> really long overdue!
> >> >>
> >> >> The main issue from my point of view is this change:
> >> >>
> >> >> https://github.com/zeromq/libzmq/commit/d9fb1d36ff2008966af538f722a1f4ab158dbf64
> >> >>
> >> >> -typedef struct zmq_msg_t {unsigned char _ [64];} zmq_msg_t;
> >> >>  +/* union here ensures correct alignment on architectures that require
> >> >> it, e.g.
> >> >>  + * SPARC
> >> >>  + */
> >> >>  +typedef union zmq_msg_t {unsigned char _ [64]; void *p; } zmq_msg_t;
> >> >>
> >> >>
> >> >> This is flagged by the common ABI checkers tools as an ABI breakage
> >> >> (see: http://abi-laboratory.pro/tracker/timeline/zeromq/ ). And it makes
> >> >> sense from this point of view: if some applications on some
> >> >> architectures are broken due to wrong alignment, they would need to be
> >> >> rebuilt, and the way to ensure that is to bump the ABI "current" digit
> >> >> to make sure maintainers do a rebuild.
> >> >>
> >> >> On the other hand, signaling an ABI breakage is a pain, and a cause of
> >> >> major churn for packagers and maintainers. It means for example a new
> >> >> package has to be created (eg: libzmq5 -> libzmq6), and a transition has
> >> >> to be started and all reverse dependencies need to be rebuilt. And if
> >> >> this is pointless for all save a few corner cases (eg SPARC64 as for
> >> >> above) it's all quite frustrating.
> >> >>
> >> >> So we have a choice to make before we release 4.2, four possibilities as
> >> >> far as I can see:
> >> >>
> >> >> 1) Ignore the ABI checkers and get yelled at by maintainers and
> >> >> packagers. Also the SPARC64 users will most likely NOT get their bug
> >> >> fixed
> >> >> 2) Bump ABI revision to 6 and get yelled at by maintainers and 

Re: [zeromq-dev] Connect to UDP Multicast CoT Server

[Doron Somech]

Udp only work with radio-dish. You can use pgm for pubsub.

On Fri, Sep 30, 2016, 16:53 Craig Stutts ARA/SED  wrote:

> Hey guys,
>
> I'm new to network programming and I am having trouble connecting to a
> Cursor on Target multicast UDP server using ZMQ 4.2.0 (master). I'm getting
> a "The protocol is not compatible with the socket type" error in
> zmq_connect. The manual page for zmq_connect doesn't mention UDP, but the
> below link suggests that it should be possible using the ZMQ_RADIO socket
> type. This type appears to no longer exist in this version, so I'm using
> ZMQ_SUB. Below is my code.
>
> UDP Link
> https://github.com/zeromq/libzmq/blob/master/doc/zmq_udp.txt
>
> void *ctx = zmq_ctx_new();
> void *sub = zmq_socket(ctx, ZMQ_SUB);
> int rc = zmq_connect(sub, "udp://239.2.3.1:6968");
> cout << zmq_strerror(errno) << endl;
>
> char buf[256];
> rc = zmq_recv(sub, buf, 256, 0);
> cout << "Return Message: " << buf << endl;
> zmq_close(sub);
> zmq_ctx_destroy(ctx);
>
> I have also tried int rc = zmq_connect(sub, "epgm://192.168.3.150;
> 239.2.3.1:6968"); which includes the Wireless IPv4 network that the CoT
> device is on. This command returns a "Protocol not supported" error.
>
> Thanks,
>
> Craig
>
> -Original Message-
> From: zeromq-dev [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf
> Of zeromq-dev-requ...@lists.zeromq.org
> Sent: Friday, September 30, 2016 6:00 AM
> To: zeromq-dev@lists.zeromq.org
> Subject: zeromq-dev Digest, Vol 6, Issue 16
>
> Send zeromq-dev mailing list submissions to
> zeromq-dev@lists.zeromq.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
> or, via email, send a message with subject or body 'help' to
> zeromq-dev-requ...@lists.zeromq.org
>
> You can reach the person managing the list at
> zeromq-dev-ow...@lists.zeromq.org
>
> When replying, please edit your Subject line so it is more specific than
> "Re: Contents of zeromq-dev digest..."
>
>
> Today's Topics:
>
>1. Re: ZeroMQ 4.2 release, planning (Pieter Hintjens)
>2. Re: ZeroMQ 4.2 release, planning (Benjamin Henrion)
>3. czmq + draft mode (Wes Young)
>4. Re: czmq + draft mode (Wes Young)
>5. Re: czmq + draft mode (Kevin Sapper)
>6. Re: czmq + draft mode (Wes Young)
>7. Question about context and/or socket creation (James Chapman)
>
>
> --
>
> Message: 1
> Date: Thu, 29 Sep 2016 12:13:28 +0200
> From: Pieter Hintjens 
> To: ZeroMQ development list 
> Cc: Luca Boccassi 
> Subject: Re: [zeromq-dev] ZeroMQ 4.2 release, planning
> Message-ID:
> 

Re: [zeromq-dev] recv() on a ZMQ_REQ socket is slow

[Doron Somech]

REQ-REP pattern is slow because zeromq is not tuned for one message at a
time but for batch of messages. I suggest to use the dealer-router or
better the client-server.  You can use uuid or just sequnce number for
request identifier. However if your use case is one message at the time so
the benefit of zeromq will be throughput at the server side.

On Thu, Sep 29, 2016, 01:30 Dimos Stamatakis  wrote:

> Hi everyone,
>
> I have been testing the ZeroMQ library combined with Thrift to compare the
> performance. I have a simple ZeroMQ example where I create ZMQ_REP and
> ZMQ_REQ sockets for a simple client-server communication. In the example,
> client simply sends a message to the server and waits for a reply. The only
> slow part I measured is the client receive, which makes sense. But it is
> almost three times slower than a native TCP socket.
> So the actual benchmark is the Thrift with ZeroMQ, as officially released
> by apache:
>
> https://github.com/apache/thrift/tree/master/contrib/zeromq
>
> I measured the overhead of using ZeroMQ for Thrift's communication,
> instead of a native TCP socket.
> The client response is much higher when using ZeroMQ and I was expecting
> to be caused by the copying that must be done from Thrift data structures
> to ZeroMQ data structures (zmq::message_t).
> However the copying is negligible. The actual overhead is the recv() of
> the client, waiting for the server to reply. Is it normal for a ZeroMQ
> REQ-REP socket to be almost three times slower than a native TCP socket? Is
> there something I can do to optimize it? I am using the default send and
> recv with no flags.
>
> I thought that maybe something is wrong with the way Thrift-ZeroMQ is
> implemented so I also ran the hello example:
> http://zguide.zeromq.org/cpp:hwserver
> This also takes more time in the recv().
>
>
> Many thanks in advance,
> Dimos
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev