Hi, On 12/12/2014 5:00 PM, Arnaud Kapp wrote: > Hey, > > I did not check for performance improvement, I simply tested that my > program continued to work. > The thing is, my poller only has 1 zmq socket and 1 file descriptor > (that relies on POLLPRI -- its a GPIO pin). So in my case I could'nt > really check > for perf improvement. Also I am running this on a Raspberry Pi. Of course with only two descriptors the performance improvement will be negligible. > > I used it for about 1hour before sending the previous mail. Gonna work > with this patch applied next week, making sure it keeps working. Thank you very much for testing it, at least it doesn't break anything at first glance. Cheers, Francis
> > On Thu, Dec 11, 2014 at 2:39 PM, Francis Le Bourse > <[email protected]> wrote: >> Hello, >> On 12/10/2014 6:54 PM, Arnaud Kapp wrote: >>> Hello, >>> >>> Sorry it took me a while, but I finally go to test your patch. >>> My setup that use POLLPRI seems to work properly with your patch applied >>> :). >> Good. Do you see a performance improvement ? How long have you been using it >> ? >>> >>> Did you submit a PR to get it merged into libzmq master yet? >> No, not yet. I was waiting for feedback before. And I had another issue with >> a memory hog in libzmq. >> Cheers, >> Francis >> >> >>> On Fri, Nov 28, 2014 at 11:35 AM, Francis Le Bourse >>> <[email protected]> wrote: >>>> On 11/24/2014 8:08 PM, Arnaud Kapp wrote: >>>>>> Currently, the patch is written for 3.2.4. I'll wait to put it in >>>>>> libzmq >>>>>> master >>>> The first patch for 3.2.4 had an issue in zmq_poll(), I had tried a >>>> little >>>> too aggressive optimization by bypassing the "first_pass" processing. It >>>> is >>>> fixed in the current one. >>>> The patch for the current head is clean. >>>> Cheers, >>>> Francis >>>> >>>> >>>>> Oh okay. This is the commit that added the flag: >>>>> >>>>> >>>>> https://github.com/zeromq/libzmq/commit/779c37abc433cb6595ddeedaf86b280317656bdd >>>>> >>>>> libzmq was 4.1 at the time I believe. >>>>> >>>>> I'll probably look at it this week-end then :) >>>>> >>>>> On Mon, Nov 24, 2014 at 12:10 PM, Francis Le Bourse >>>>> <[email protected]> wrote: >>>>>> Hi, >>>>>> On 11/24/2014 11:35 AM, Arnaud Kapp wrote: >>>>>>> Hello, >>>>>>> >>>>>>> I recently added support for POLLPRI flag. >>>>>>> It looks like it's not handled in your patch >>>>>> No, it isn't handled. In which version do you have added this flag ? >>>>>> Currently, the patch is written for 3.2.4. I'll wait to put it in >>>>>> libzmq >>>>>> master. >>>>>>> and that it needs custom >>>>>>> support. Since there is no test related to this flags you wouldn't >>>>>>> notice. >>>>>>> >>>>>>> I can give it a look if you want. >>>>>> That would be nice. >>>>>> >>>>>> Cheers, >>>>>> Francis >>>>>> >>>>>> >>>>>>> On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens <[email protected]> >>>>>>> wrote: >>>>>>>> I suggest you send the patch to libzmq master, and ensure all test >>>>>>>> cases pass. Then we can get this into the next version. >>>>>>>> >>>>>>>> On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse >>>>>>>> <[email protected]> wrote: >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> On 11/6/2014 3:18 PM, Pieter Hintjens wrote: >>>>>>>>>> Oh, ok. Sounds like you have a good candidate for some before/after >>>>>>>>>> measurement and optimization. Are you going to try to make a patch >>>>>>>>>> for >>>>>>>>>> this? >>>>>>>>> I have a patch candidate for this optimization, the performance >>>>>>>>> improvement >>>>>>>>> is very good and it doesn't seem to introduce any new instability. >>>>>>>>> What is modified: >>>>>>>>> - zmq_poll(), there is only one poll() now, >>>>>>>>> - and epoll() from epoll.cpp >>>>>>>>> Other calls to poll() and select() are left unmodified. >>>>>>>>> >>>>>>>>> I woulld be happy to have any feedback. >>>>>>>>> >>>>>>>>> >>>>>>>>> Cheers, >>>>>>>>> Francis >>>>>>>>>> >>>>>>>>>> On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse >>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>> On 11/6/2014 11:47 AM, Pieter Hintjens wrote: >>>>>>>>>>>> A simple optimization is, when you are polling sockets for input, >>>>>>>>>>>> to >>>>>>>>>>>> continue reading from an active socket using a non-blocking read. >>>>>>>>>>>> So >>>>>>>>>>>> you process all waiting messages on a socket and then only switch >>>>>>>>>>>> back >>>>>>>>>>>> to poll when needed. >>>>>>>>>>> Thank you for you quick reply. >>>>>>>>>>> >>>>>>>>>>> Yes, but the question was more about the zmq_poll() internals. >>>>>>>>>>> For 600+ file descriptors, zmq_poll() calls poll() a huge number >>>>>>>>>>> of >>>>>>>>>>> times >>>>>>>>>>> for only a few that will trigger a POLLIN and the relevant >>>>>>>>>>> information >>>>>>>>>>> is >>>>>>>>>>> already known / present in the pollfds array. The performance hit >>>>>>>>>>> is >>>>>>>>>>> there. >>>>>>>>>>> >>>>>>>>>>> Cheers, >>>>>>>>>>> Francis >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>> On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse >>>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>>> Hi, >>>>>>>>>>>>> >>>>>>>>>>>>> I am looking at a performance issue in zmq, when the number of >>>>>>>>>>>>> zsockets >>>>>>>>>>>>> / >>>>>>>>>>>>> file descriptors becomes large. >>>>>>>>>>>>> The relevant calls are: >>>>>>>>>>>>> poll+0x57 >>>>>>>>>>>>> zmq_poll+0x2e3 >>>>>>>>>>>>> zloop_start+0x1e8 >>>>>>>>>>>>> main+0xb40 >>>>>>>>>>>>> __libc_start_main+0xfd >>>>>>>>>>>>> immediately followed by a loop of >>>>>>>>>>>>> poll+0x57 >>>>>>>>>>>>> zmq::signaler_t::wait(int)+0x33 >>>>>>>>>>>>> zmq::mailbox_t::recv(zmq::command_t*, int)+0x78 >>>>>>>>>>>>> zmq::socket_base_t::process_commands(int, bool)+0xbe >>>>>>>>>>>>> zmq::socket_base_t::getsockopt(int, void*, unsigned >>>>>>>>>>>>> long*)+0x135 >>>>>>>>>>>>> zmq_getsockopt+0x75 >>>>>>>>>>>>> zmq_poll+0x3da >>>>>>>>>>>>> zloop_start+0x1e8 >>>>>>>>>>>>> main+0xb40 >>>>>>>>>>>>> __libc_start_main+0xfd >>>>>>>>>>>>> >>>>>>>>>>>>> The code in the loop is executed once per file descriptor in the >>>>>>>>>>>>> initial >>>>>>>>>>>>> pollarray, redoing a poll() system call each time. >>>>>>>>>>>>> Is there a reason to proceed that way ? >>>>>>>>>>>>> Would be possible to reuse the results of the first poll() in >>>>>>>>>>>>> order >>>>>>>>>>>>> to >>>>>>>>>>>>> bypass the second set of system calls ? >>>>>>>>>>>>> >>>>>>>>>>>>> Cheers, >>>>>>>>>>>>> Francis >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>> zeromq-dev mailing list >>>>>>>>>>>>> [email protected] >>>>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> zeromq-dev mailing list >>>>>>>> [email protected] >>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>> >>>>>>> >>> >> > > _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
