Btw,
I realized that I actually already hit this "phenomenon" of poll() being
called so fast!!!
This is the link of the thread where I raised the issue:
   https://lists.zeromq.org/pipermail/zeromq-dev/2017-October/031974.html

At that time I solved by simply using the ZMQ_RCVTIMEO option as suggested
by Luca... this time it's slightly different because I'm already using that
option but now I have 2 zmq sockets to dequeue so that "inbound poll rate"
issue is doubled... I will check if changing that zmq non-configurable
internal parameter makes a performance difference or not!

Francesco



2018-04-12 22:47 GMT+02:00 Francesco <francesco.monto...@gmail.com>:

> [sorry I hit "send" too early! here's the complete email]
>
> Hi Luca, Hi Bill,
> thanks for the answers.
> Actually I verified that the configure script of zmq seems to be using
> epoll:
>
>    configure: Choosing polling system from 'kqueue epoll devpoll pollset
> poll select'...
>    configure: Using 'epoll' polling system with CLOEXEC
>
> and indeed inside platform.hpp I found:
>
>   /* Use 'epoll' polling system */
>   #define ZMQ_USE_EPOLL 1
>
>   /* Use 'epoll' polling system with CLOEXEC */
>   #define ZMQ_USE_EPOLL_CLOEXEC 1
>
> I also read the links you sent me Bill: actually since I'm monitoring only
> 2 file descriptors (2 zmq sockets) I think that poll() and epoll()
> difference is hardly measurable.
> My feeling is just that whatever code ends up calling poll() so often will
> pay a cost in performance due to the syscall overhead:
>
>    http://arkanis.de/weblog/2017-01-05-measurements-of-system-
> call-performance-and-overhead
>    http://man7.org/linux/man-pages/man7/vdso.7.html
>
> So my question is actually the following one:
>    In a ZMQ application, in which thread context are the "real" TCP socket
> file descriptors actually polled?
>    Is that polling happening in ZMQ background threads?
>    Or rather it happens inside the thread that calls the zmq_msg_recv() /
> zmq_poll() /  zmq_poller_wait_all()  ?
>
> Because from looking briefly at zmq code it looks like zmq_msg_recv()  is
> just talking with the zmq background thread via some kind of inter-thread
> signaler.
> So in my view I _guess_ that the real polling of the TCP socket should
> happen in zmq background threads and the application threads just dequeues
> some kind of queue that lives in those background threads. However that
> seem to be proved wrong by a simple test: I tried to put a breakpoint on
> the syscall poll()... the stacktrace I get is:
>
> #0  poll () at ../sysdeps/unix/syscall-template.S:84
> #1  0x00007ffff63db0ba in zmq::signaler_t::wait(int) ()
> #2  0x00007ffff63bb045 in zmq::mailbox_t::recv(zmq::command_t*, int) ()
> #3  0x00007ffff63dba37 in zmq::socket_base_t::process_commands(int, bool)
> #4  0x00007ffff63ddbca in zmq::socket_base_t::recv(zmq::msg_t*, int) ()
> #5  0x00007ffff6400620 in zmq_msg_recv ()
> [my code stack frames]
>
> so it seems like it's the application threads that ends up calling that
> poll().
> Note that I set the RX timeout on my socket equal to 0 (ZMQ_DONTWAIT)...
>
>
> Thanks,
> Francesco
>
>
>
>
>
> 2018-04-12 22:39 GMT+02:00 Francesco <francesco.monto...@gmail.com>:
>
>> Hi Luca, Hi Bill,
>> thanks for the answers.
>> Actually I verified that the configure script of zmq seems to be using
>> epoll:
>>
>>    configure: Choosing polling system from 'kqueue epoll devpoll pollset
>> poll select'...
>>    configure: Using 'epoll' polling system with CLOEXEC
>>
>> and indeed inside platform.hpp I found:
>>
>>   /* Use 'epoll' polling system */
>>   #define ZMQ_USE_EPOLL 1
>>
>>   /* Use 'epoll' polling system with CLOEXEC */
>>   #define ZMQ_USE_EPOLL_CLOEXEC 1
>>
>> I also read the links you sent me Bill: actually since I'm monitoring
>> only 2 file descriptors (2 zmq sockets) I think that poll() and epoll()
>> difference is hardly measurable.
>> My feeling is just that whatever code ends up calling poll() so often
>> will pay a lot of
>>
>>    http://arkanis.de/weblog/2017-01-05-measurements-of-system-
>> call-performance-and-overhead
>>
>>
>>
>>
>>
>>
>> 2018-04-11 21:05 GMT+02:00 Bill Torpey <wallstp...@gmail.com>:
>>
>>> Well, a little googling found this, which is a pretty good writeup:
>>> https://jvns.ca/blog/2017/06/03/async-io-on-linux--
>>> select--poll--and-epoll/
>>>
>>>
>>> On Apr 11, 2018, at 2:42 PM, Bill Torpey <wallstp...@gmail.com> wrote:
>>>
>>> So, are there any benchmark tests that can be used to quantify the
>>> overhead of zmq_poll?  It seems like this question keeps coming up, and it
>>> would certainly be nice to have some real numbers (and the code used to
>>> generate them).
>>>
>>> Having said that, there are several mechanisms that zmq_poll can use,
>>> and there are apparently significant performance differences between them.
>>> My understanding of the conventional wisdom is that epoll is preferable to
>>> poll, which is preferable to select — but I don’t have any data to back
>>> that up.
>>>
>>> In any event, you can examine the output of the libzmq build to see
>>> which mechanism is being used:
>>>
>>> -- Looking for kqueue
>>> -- Looking for kqueue - not found
>>> -- Looking for epoll_create
>>> -- Looking for epoll_create - found
>>> -- Looking for epoll_create1
>>> -- Looking for epoll_create1 - found
>>> *-- Detected epoll polling method*
>>>
>>> I’ve found one (very old!) post that discusses these differences:
>>> https://cs.uwaterloo.ca/~brecht/servers/epoll/.  If anyone can suggest
>>> additional sources, please do.
>>>
>>>
>>> On Apr 11, 2018, at 1:46 PM, Luca Boccassi <luca.bocca...@gmail.com>
>>> wrote:
>>>
>>> On Wed, 2018-04-11 at 18:47 +0200, Francesco wrote:
>>>
>>> Hi all,
>>>
>>> I'm using zmq_poller_wait_all() API in one of my threads because I
>>> need to
>>> poll over 2 ZMQ sockets.
>>> I'm receiving a lot of traffic on both these sockets.
>>> I think the performance of my software is bad and that may be due
>>> IMHO to
>>> the huge amount of poll() syscalls that my thread does... I think the
>>> overhead of doing the system call is what is killing me...
>>>
>>> so my question is: when zmq_poller_wait_all() polls zmq FDs it is
>>> actually
>>> polling the real socket FD or rather some ZMQ internal structure?
>>> If the latter is true, do I have a way to poll more than 1 zmq socket
>>> without doing a system call?
>>>
>>>
>>> THanks,
>>> Francesco
>>>
>>>
>>> I'm not familiar with the zmq_poller API so someone else might help -
>>> in general, a common optimisation is to drain the socket using non-
>>> blocking receives when poll wakes up (taking care of using heuristic to
>>> avoid starving the other sockets), so that you can process multiple
>>> messages per poll
>>>
>>> --
>>> Kind regards,
>>> Luca Boccassi_______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev@lists.zeromq.org
>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> zeromq-dev mailing list
>>> zeromq-dev@lists.zeromq.org
>>> https://lists.zeromq.org/mailman/listinfo/zeromq-dev
>>>
>>>
>>
>
_______________________________________________
zeromq-dev mailing list
zeromq-dev@lists.zeromq.org
https://lists.zeromq.org/mailman/listinfo/zeromq-dev

Reply via email to