Re: [zeromq-dev] Newbie questions re HWM and Async/Sync io

[Luca Boccassi]

On 21 March 2016 at 08:15, Aaron Koolen-Bourke  wrote:
> Hi all.

> Thing is, from what I can tell I then need to move to polling and not
> blocking receives, which not only keeps the thread busy looping, but seems
> to be quite expensive (There seems to be a sleep timer in there or some
> such). This just kills performance. Is there any support for a completely
> event driven system in ZMQ/CZMQ?

Hi Aaron,

CZMQ uses libzmq's poll APIs, which depending on your system will pick
out the best option to poll on file descriptors. On Linux for example
it will use epoll, on Windows it will use select and so on. So there's
no busy looping, so don't worry about performances.

Sorry but I'm not sure about the HWM issue, hopefully someone else can
give their 2c on that.

Kind regards,
Luca Boccassi
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Newbie questions re HWM and Async/Sync io

[Aaron Koolen-Bourke]

Hi all.

Slowly getting my ZMQ evaluation application together but I've discovered a
couple of things I'm hoping have some as-yet undiscovered (to me) answers.

I was noticing that my message feeder was blocking at the HWM and latency
was taking a hit so I've been experimenting with different HWM values along
the pipeline. 2000, (workers*1000) and various things like that. However I
have noticed that sometimes I'll use a value at one point in the chain and
the whole program hangs. Everyone is blocked on waiting for messages. This
doesn't make obvious sense to me.  NOTE: I'm waiting for a start signal
before any messages are sent so I'm not setting the HWM "mid action".

Is there any info on why it might hang and how to avoid it? I'm worried
that it's not deterministic with value and could just happen in a real
application.

Secondly I've noticed that when I try and write something like a worker in
the "recommended way" by using ZMQ messages for all communication -
messages, start, stop etc. I get to the point where I need to both receive
some data on a socket, and get a command that signals I should cleanup and
exit my thread (I'm using CZMQ actors).

Thing is, from what I can tell I then need to move to polling and not
blocking receives, which not only keeps the thread busy looping, but seems
to be quite expensive (There seems to be a sleep timer in there or some
such). This just kills performance. Is there any support for a completely
event driven system in ZMQ/CZMQ?

Many thanks
Aaron

CZMQ, ZMQ 4.1.4, Windows
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Newbie questions regarding performance

[Kevin Sapper]

Hi Aaron,

without your actual code it's hard to tell. Please create a Github gist.
Thanks!

//Kevin

2016-03-18 11:49 GMT+01:00 Aaron Koolen-Bourke :

> Hi all. I'm currently tasked with building a small test application to
> test 0MQ's performance for a possible replacement of our legacy messaging.
>
> The performance tests that come with libzmq obviously show great
> performance (8us latency on my box) but when I build a slightly different
> test of my own I'm getting comparatively terrible performance (2000us). Now
> I am sure this is my fault but I'm banging my head against a wall now and
> need some help.
>
> First, my setup - Windows platform, release mode builds using VS2015.
> Using CZMQ and 0MQ version 4.1.4.
>
> I built what I thought was a simple pipeline as suggested in the guide.
>
> PUSH->PULL->PULL (calculates entire time of message processing)
>
> The steps I do.
>
> Create producer zactor with a ZMQ_PUSH socket for "tasks". Then send
> 1, 4 byte messages in a tight loop using zsock_bsend(socket, "4",
> count);
>
> Create another actor that is a ZMQ_PULL for receiving messages to indicate
> the task as been completely processed by a worker. I did this so I could
> measure the entire round trip.
>
> Create worker actor with a ZMQ_PULL to the producer. In tight loop I call
> zsock_brecv to receive those 4 bytes. When I do, I calculate the time since
> it was sent and post that as a message to the reply actor. That actor
> stores the value in a pre-allocated vector.
>
> So, to a newbie like myself it seems that I'm doing 2 sends and 2
> receives, so I would expect something in the ballpark of the lib tests.
>
> Hope that was clear.
>
> Thanks
> Aaron
>
>
> ___
> zeromq-dev mailing list
> zeromq-dev@lists.zeromq.org
> http://lists.zeromq.org/mailman/listinfo/zeromq-dev
>
>
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Newbie questions regarding performance

[Aaron Koolen-Bourke]

Hi all. I'm currently tasked with building a small test application to test
0MQ's performance for a possible replacement of our legacy messaging.

The performance tests that come with libzmq obviously show great
performance (8us latency on my box) but when I build a slightly different
test of my own I'm getting comparatively terrible performance (2000us). Now
I am sure this is my fault but I'm banging my head against a wall now and
need some help.

First, my setup - Windows platform, release mode builds using VS2015. Using
CZMQ and 0MQ version 4.1.4.

I built what I thought was a simple pipeline as suggested in the guide.

PUSH->PULL->PULL (calculates entire time of message processing)

The steps I do.

Create producer zactor with a ZMQ_PUSH socket for "tasks". Then send 1,
4 byte messages in a tight loop using zsock_bsend(socket, "4", count);

Create another actor that is a ZMQ_PULL for receiving messages to indicate
the task as been completely processed by a worker. I did this so I could
measure the entire round trip.

Create worker actor with a ZMQ_PULL to the producer. In tight loop I call
zsock_brecv to receive those 4 bytes. When I do, I calculate the time since
it was sent and post that as a message to the reply actor. That actor
stores the value in a pre-allocated vector.

So, to a newbie like myself it seems that I'm doing 2 sends and 2 receives,
so I would expect something in the ballpark of the lib tests.

Hope that was clear.

Thanks
Aaron
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

[zeromq-dev] Newbie questions

[Alex du Plessis]

Hello list,

I'm a complete newbie in the use and application of Zeromq so I am sure 
my questions will sound quite inane and downright stupid.  I apologise 
in advance.  I stumbled on ZeroMQ early this week while googling for a 
decent message system that is'nt implemented in Java. I use 
FPC(FreePascal) and Lazarus (both Open Source products) as my 
programming platform  and intend to write a binding as well as 
component/s to enable the use of ZeroMQ with FPC.  Unfortunately, I am 
not very C proficient, so I am also doing a crash course in C/C++ at the 
moment.  I have already started the port and I'm ironing out the 
wrinkles in my interface unit .So much for the intro, here's my 
question:


From the api documentation on the web it seems that the variable ffn of 
type zmq_free_fn as defined in the parameters of *zmq_msg_init_data* is 
a user supplied function and one needs to pass a pointer to a function 
that will free the message data.  Is this correct?


Regards


___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Newbie questions

[Hoelzlwimmer Andreas]

That should be how it's used, yes. The parameter is optional, so if you don't 
submit a method/function there, you will have to take care of deleting yourself.

From: zeromq-dev-boun...@lists.zeromq.org 
[mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Alex du Plessis
Sent: Freitag, 01. April 2011 15:53
To: ZeroMQ Mailing list
Subject: [zeromq-dev] Newbie questions

Hello list,

I'm a complete newbie in the use and application of Zeromq so I am sure my 
questions will sound quite inane and downright stupid.  I apologise in advance. 
 I stumbled on ZeroMQ early this week while googling for a decent message 
system that is'nt implemented in Java. I use FPC(FreePascal) and Lazarus (both 
Open Source products) as my programming platform  and intend to write a binding 
as well as component/s to enable the use of ZeroMQ with FPC.  Unfortunately, I 
am not very C proficient, so I am also doing a crash course in C/C++ at the 
moment.  I have already started the port and I'm ironing out the wrinkles in my 
interface unit .So much for the intro, here's my question:

From the api documentation on the web it seems that the variable ffn of type 
zmq_free_fn as defined in the parameters of zmq_msg_init_data is a user 
supplied function and one needs to pass a pointer to a function that will free 
the message data.  Is this correct?

Regards

___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Newbie questions

[Steven McCoy]

On 1 April 2011 21:53, Alex du Plessis alexdup.m...@gmail.com wrote:

 I use FPC(FreePascal) and Lazarus (both Open Source products) as my
 programming platform  and intend to write a binding as well as component/s
 to enable the use of ZeroMQ with FPC.


Dare I suggest looking at the Delphi, or Pascal on OpenVMS bindings?  The
Delphi bindings appear to be for the previous version though:

http://www.danieleteti.it/?page_id=131
https://github.com/zeromq/zeromq1/tree/master/windows/paszmq/

http://zeromqonopenvms.blogspot.com/

-- 
Steve-o
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Newbie questions

[Martin Sustrik]

Alex,

  From the api documentation on the web it seems that the variable ffn of
 type zmq_free_fn as defined in the parameters of *zmq_msg_init_data* is
 a user supplied function and one needs to pass a pointer to a function
 that will free the message data. Is this correct?

Most bindings don't even use zmq_msg_init_data. It's intended for 
zero-copy transfer of user-allocated buffers, which you probably won't 
be able to do in Pascal anyway.

Just stick to zmq_msg_init_size.

Martin
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev

Re: [zeromq-dev] Newbie questions

[Shonari Knibbs]

  It is dangerous to not use the callback (at least with sub/pub; I have not
tested this with the other models).

It is safer to free the message in the callback than at some arbitrary time
in the future: it is possible that you wind up freeing the message before it
is used by 0mq and your message will not be sent.
--
From: Hoelzlwimmer Andreas
Sent: 01 April 2011 10:01
To: ZeroMQ development list
Subject: Re: [zeromq-dev] Newbie questions

That should be how it’s used, yes. The parameter is optional, so if you
don’t submit a method/function there, you will have to take care of deleting
yourself.



*From:* zeromq-dev-boun...@lists.zeromq.org [mailto:
zeromq-dev-boun...@lists.zeromq.org] *On Behalf Of *Alex du Plessis
*Sent:* Freitag, 01. April 2011 15:53
*To:* ZeroMQ Mailing list
*Subject:* [zeromq-dev] Newbie questions



Hello list,

I'm a complete newbie in the use and application of Zeromq so I am sure my
questions will sound quite inane and downright stupid.  I apologise in
advance.  I stumbled on ZeroMQ early this week while googling for a decent
message system that is'nt implemented in Java. I use FPC(FreePascal) and
Lazarus (both Open Source products) as my programming platform  and intend
to write a binding as well as component/s to enable the use of ZeroMQ with
FPC.  Unfortunately, I am not very C proficient, so I am also doing a crash
course in C/C++ at the moment.  I have already started the port and I'm
ironing out the wrinkles in my interface unit .So much for the intro,
here's my question:

From the api documentation on the web it seems that the variable ffn of type
zmq_free_fn as defined in the parameters of *zmq_msg_init_data* is a user
supplied function and one needs to pass a pointer to a function that will
free the message data.  Is this correct?

Regards
___
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
http://lists.zeromq.org/mailman/listinfo/zeromq-dev