Min - I merged the poll request on my little laptop and gave it a spin. Before your pull request:
mps: 28066 mps: 28116 mps: 29745 mps: 28912 mps: 28679 after your pull request: mps: 78138 mps: 78364 mps: 78411 mps: 78065 mps: 78175 Excellent. By the way, just for fun I imported zmq.green instead of zmq without changing anything else mps: 24738 mps: 24779 mps: 24759 mps: 24630 mps: 24702 And it's performance is well below straight zmq Poller. Thanks for the quick patch! Brian On Tue, Jun 25, 2013 at 8:45 AM, Brian Knox <[email protected]> wrote: > Thanks! The 4x degradation is within an acceptable range for what we're > doing and will allow me to keep some things in python for awhile longer. > I'll patch locally and take it for a test drive today. > > Brian > > > On Mon, Jun 24, 2013 at 11:09 PM, MinRK <[email protected]> wrote: > >> From my tests this evening, it looks like zmq_poll does have an >> appreciable cost (plus the overhead of being wrapped in Python), and will >> be notably slower than a simple `while True: s.recv()`. But I did find >> some inefficiencies in the Poller implementation, which should be addressed >> by PR #381 <https://github.com/zeromq/pyzmq/pull/381>, bringing the >> difference closer to 4x from 10x. >> >> Your tests on my laptop: >> >> pure recv: >> mps: 241121 >> mps: 419644 >> mps: 413874 >> mps: 434421 >> mps: 428623 >> mps: 404847 >> >> poll (master): >> mps: 48544 >> mps: 54143 >> mps: 51642 >> mps: 48888 >> mps: 47591 >> mps: 54218 >> >> poll (after 381): >> mps: 110436 >> mps: 110876 >> mps: 104220 >> mps: 110690 >> mps: 100544 >> mps: 110922 >> >> -MinRK >> >> >> >> On Mon, Jun 24, 2013 at 12:18 PM, Brian Knox <[email protected]> wrote: >> >>> Thanks Min - additionally if I'm incorrect in my assumption that the >>> poll look in my example code should perform better, or if there's a better >>> way to do what I'm doing just let me know. >>> >>> The example code polls only one socket (which isn't that useful, hah) >>> but I wanted to keep the test case dirt simple. The actual thing I was >>> working on has one socket that should receive data at a relatively high >>> rate (say 50k to 100k msgs/s) , and one socket that receives data at a >>> much lower rate (say 1 msg/s) (for heartbeating, command and control, etc). >>> >>> >>> Brian >>> >>> >>> On Mon, Jun 24, 2013 at 12:12 PM, Min RK <[email protected]> wrote: >>> >>>> Thanks for the report, I will look into whether I have introduced a >>>> performance degradation in the last few iterations. >>>> >>>> -MinRK >>>> >>>> On Jun 24, 2013, at 11:27, Brian Knox <[email protected]> wrote: >>>> >>>> > It's been awhile since I've used pyzmq, and I'm running into a >>>> performance issue using Poller.poll(). >>>> > >>>> > With a simple blocking recv() in a while True loop, I get ~ 300k >>>> messages a second. >>>> > >>>> > With the simplest case of the same code using a poll on the socket, I >>>> get ~ 30k messages a second. >>>> > >>>> > (pyzeromq)taotetek@Moya:~/src/performance_example$ python >>>> ./receiver.py >>>> > mps: 226049 >>>> > mps: 348582 >>>> > mps: 271728 >>>> > mps: 300389 >>>> > mps: 411059 >>>> > mps: 276749 >>>> > >>>> > (pyzeromq)taotetek@Moya:~/src/performance_example$ python >>>> ./poll_receiver.py >>>> > mps: 28066 >>>> > mps: 28116 >>>> > mps: 29745 >>>> > mps: 28912 >>>> > mps: 28679 >>>> > >>>> > I don't remember a 10x performance degradation last time I used a >>>> poller - but perhaps I'm remembering something wrong, or making a painfully >>>> obvious error in my code .. I've been using czmq from C mostly for the last >>>> while. >>>> > >>>> > I'm using pyzmq from git master, and the latest Cython (0.19.1) with >>>> libzmq 3.2.3. >>>> > >>>> > I've attached the test cases I'm using. >>>> > >>>> > <performance_example.tar> >>>> > _______________________________________________ >>>> > zeromq-dev mailing list >>>> > [email protected] >>>> > http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> _______________________________________________ >>>> zeromq-dev mailing list >>>> [email protected] >>>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>>> >>> >>> >>> _______________________________________________ >>> zeromq-dev mailing list >>> [email protected] >>> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >>> >>> >> >> _______________________________________________ >> zeromq-dev mailing list >> [email protected] >> http://lists.zeromq.org/mailman/listinfo/zeromq-dev >> >> > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
