Hey, I did not check for performance improvement, I simply tested that my program continued to work. The thing is, my poller only has 1 zmq socket and 1 file descriptor (that relies on POLLPRI -- its a GPIO pin). So in my case I could'nt really check for perf improvement. Also I am running this on a Raspberry Pi.
I used it for about 1hour before sending the previous mail. Gonna work with this patch applied next week, making sure it keeps working. On Thu, Dec 11, 2014 at 2:39 PM, Francis Le Bourse <[email protected]> wrote: > Hello, > On 12/10/2014 6:54 PM, Arnaud Kapp wrote: >> >> Hello, >> >> Sorry it took me a while, but I finally go to test your patch. >> My setup that use POLLPRI seems to work properly with your patch applied >> :). > > Good. Do you see a performance improvement ? How long have you been using it > ? >> >> >> Did you submit a PR to get it merged into libzmq master yet? > > No, not yet. I was waiting for feedback before. And I had another issue with > a memory hog in libzmq. > Cheers, > Francis > > >> >> On Fri, Nov 28, 2014 at 11:35 AM, Francis Le Bourse >> <[email protected]> wrote: >>> >>> On 11/24/2014 8:08 PM, Arnaud Kapp wrote: >>>>> >>>>> Currently, the patch is written for 3.2.4. I'll wait to put it in >>>>> libzmq >>>>> master >>> >>> The first patch for 3.2.4 had an issue in zmq_poll(), I had tried a >>> little >>> too aggressive optimization by bypassing the "first_pass" processing. It >>> is >>> fixed in the current one. >>> The patch for the current head is clean. >>> Cheers, >>> Francis >>> >>> >>>> Oh okay. This is the commit that added the flag: >>>> >>>> >>>> https://github.com/zeromq/libzmq/commit/779c37abc433cb6595ddeedaf86b280317656bdd >>>> >>>> libzmq was 4.1 at the time I believe. >>>> >>>> I'll probably look at it this week-end then :) >>>> >>>> On Mon, Nov 24, 2014 at 12:10 PM, Francis Le Bourse >>>> <[email protected]> wrote: >>>>> >>>>> Hi, >>>>> On 11/24/2014 11:35 AM, Arnaud Kapp wrote: >>>>>> >>>>>> Hello, >>>>>> >>>>>> I recently added support for POLLPRI flag. >>>>>> It looks like it's not handled in your patch >>>>> >>>>> No, it isn't handled. In which version do you have added this flag ? >>>>> Currently, the patch is written for 3.2.4. I'll wait to put it in >>>>> libzmq >>>>> master. >>>>>> >>>>>> and that it needs custom >>>>>> support. Since there is no test related to this flags you wouldn't >>>>>> notice. >>>>>> >>>>>> I can give it a look if you want. >>>>> >>>>> That would be nice. >>>>> >>>>> Cheers, >>>>> Francis >>>>> >>>>> >>>>>> On Sat, Nov 22, 2014 at 2:16 PM, Pieter Hintjens <[email protected]> >>>>>> wrote: >>>>>>> >>>>>>> I suggest you send the patch to libzmq master, and ensure all test >>>>>>> cases pass. Then we can get this into the next version. >>>>>>> >>>>>>> On Fri, Nov 21, 2014 at 2:50 PM, Francis Le Bourse >>>>>>> <[email protected]> wrote: >>>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> On 11/6/2014 3:18 PM, Pieter Hintjens wrote: >>>>>>>>> >>>>>>>>> Oh, ok. Sounds like you have a good candidate for some before/after >>>>>>>>> measurement and optimization. Are you going to try to make a patch >>>>>>>>> for >>>>>>>>> this? >>>>>>>> >>>>>>>> I have a patch candidate for this optimization, the performance >>>>>>>> improvement >>>>>>>> is very good and it doesn't seem to introduce any new instability. >>>>>>>> What is modified: >>>>>>>> - zmq_poll(), there is only one poll() now, >>>>>>>> - and epoll() from epoll.cpp >>>>>>>> Other calls to poll() and select() are left unmodified. >>>>>>>> >>>>>>>> I woulld be happy to have any feedback. >>>>>>>> >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Francis >>>>>>>>> >>>>>>>>> >>>>>>>>> On Thu, Nov 6, 2014 at 2:09 PM, Francis Le Bourse >>>>>>>>> <[email protected]> wrote: >>>>>>>>>> >>>>>>>>>> On 11/6/2014 11:47 AM, Pieter Hintjens wrote: >>>>>>>>>>> >>>>>>>>>>> A simple optimization is, when you are polling sockets for input, >>>>>>>>>>> to >>>>>>>>>>> continue reading from an active socket using a non-blocking read. >>>>>>>>>>> So >>>>>>>>>>> you process all waiting messages on a socket and then only switch >>>>>>>>>>> back >>>>>>>>>>> to poll when needed. >>>>>>>>>> >>>>>>>>>> Thank you for you quick reply. >>>>>>>>>> >>>>>>>>>> Yes, but the question was more about the zmq_poll() internals. >>>>>>>>>> For 600+ file descriptors, zmq_poll() calls poll() a huge number >>>>>>>>>> of >>>>>>>>>> times >>>>>>>>>> for only a few that will trigger a POLLIN and the relevant >>>>>>>>>> information >>>>>>>>>> is >>>>>>>>>> already known / present in the pollfds array. The performance hit >>>>>>>>>> is >>>>>>>>>> there. >>>>>>>>>> >>>>>>>>>> Cheers, >>>>>>>>>> Francis >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Thu, Nov 6, 2014 at 11:28 AM, Francis Le Bourse >>>>>>>>>>> <[email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> I am looking at a performance issue in zmq, when the number of >>>>>>>>>>>> zsockets >>>>>>>>>>>> / >>>>>>>>>>>> file descriptors becomes large. >>>>>>>>>>>> The relevant calls are: >>>>>>>>>>>> poll+0x57 >>>>>>>>>>>> zmq_poll+0x2e3 >>>>>>>>>>>> zloop_start+0x1e8 >>>>>>>>>>>> main+0xb40 >>>>>>>>>>>> __libc_start_main+0xfd >>>>>>>>>>>> immediately followed by a loop of >>>>>>>>>>>> poll+0x57 >>>>>>>>>>>> zmq::signaler_t::wait(int)+0x33 >>>>>>>>>>>> zmq::mailbox_t::recv(zmq::command_t*, int)+0x78 >>>>>>>>>>>> zmq::socket_base_t::process_commands(int, bool)+0xbe >>>>>>>>>>>> zmq::socket_base_t::getsockopt(int, void*, unsigned >>>>>>>>>>>> long*)+0x135 >>>>>>>>>>>> zmq_getsockopt+0x75 >>>>>>>>>>>> zmq_poll+0x3da >>>>>>>>>>>> zloop_start+0x1e8 >>>>>>>>>>>> main+0xb40 >>>>>>>>>>>> __libc_start_main+0xfd >>>>>>>>>>>> >>>>>>>>>>>> The code in the loop is executed once per file descriptor in the >>>>>>>>>>>> initial >>>>>>>>>>>> pollarray, redoing a poll() system call each time. >>>>>>>>>>>> Is there a reason to proceed that way ? >>>>>>>>>>>> Would be possible to reuse the results of the first poll() in >>>>>>>>>>>> order >>>>>>>>>>>> to >>>>>>>>>>>> bypass the second set of system calls ? >>>>>>>>>>>> >>>>>>>>>>>> Cheers, >>>>>>>>>>>> Francis >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>> zeromq-dev mailing list >>>>>>>>>>>> [email protected] >>>>>>>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>>>>>>>> >>>>>>> _______________________________________________ >>>>>>> zeromq-dev mailing list >>>>>>> [email protected] >>>>>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>>>> >>>>>> >>>>>> >>>> >> >> > > -- Kapp Arnaud - Xaqq _______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
