Julie, it is a little exasperating that you keep posting these numbers (and related questions) but, to date, have not shown the CODE used to get them. It is not possible to give a meaningful answer to your questions without looking at the EXACT code you are using. Furthermore, it would be very useful to be able to RUN the same code in one's machine, to ascertain whether the behavior is the same as you are reporting, and maybe fix something in 0MQ.
Best regards, -- Gonzalo Diethelm DCV Chile From: zeromq-dev-boun...@lists.zeromq.org [mailto:zeromq-dev-boun...@lists.zeromq.org] On Behalf Of Julie Anderson Sent: Wednesday, August 29, 2012 1:19 PM To: ZeroMQ development list Subject: Re: [zeromq-dev] Too much ZeroMQ overhead versus plain TCP Java NIO Epoll (with measurements) New numbers (fun!). Firstly, to make sure I was comparing apples with apples, I modified my tests to compute one-way trip instead of round-trip. I can't paste code, but I am simply using a Java NIO (non-blocking I/O) optimized with busy spinning to send and receive tcp data. This is *standard* Java NIO code, nothing too fancy. You can google around for Java NIO. I found this link<http://www.cordinc.com/blog/2010/08/java-nio-server-example.html> that shows the basics. You can also do the same thing in C as you can see here<http://stackoverflow.com/questions/27247/could-you-recommend-some-guides-about-epoll-on-linux/6150841#6150841>. My test now consists of: - JVM A sends a message which consist of the ascii representation of a timestamp in nanos. - JVM B receives this message, parses the long, computer the one-way latency and echoes back the message to JVM A. - JVM A receives the echo, parses the ascii long and makes sure that it matches the one it sent out. - Loop back and send the next message. So now I have both times: one-way and round-trip. I ran my test for 1 million messages over loopback. For ZeroMQ I am using the local_lat and remote_lat programs included with latest zeromq from here: git://github.com/zeromq/libzmq.git<http://github.com/zeromq/libzmq.git> The results: - ZeroMQ: ./local_lat tcp://lo:5555 13 1000000 ./remote_lat tcp://127.0.0.1:5555<http://127.0.0.1:5555> 13 1000000 message size: 13 [B] roundtrip count: 1000000 average latency: 19.674 [us] <==== this is one-way - Java NIO: (EPoll with busy spinning) Round-trip: Iterations: 1,000,000 | Avg Time: 16552.15 nanos | Min Time: 12515 nanos | Max Time: 129816 nanos | 75%: 16290 nanos | 90%: 16369 nanos | 99%: 16489 nanos | 99.999%: 16551 nanos One-way trip: Iterations: 1,110,000 | Avg Time: 8100.12 nanos | Min Time: 6150 nanos | Max Time: 118035 nanos | 75%: 7966 nanos | 90%: 8010 nanos | 99%: 8060 nanos | 99.999%: 8099 nanos Conclusions: That's 19.674 versus 8.100 so ZeroMQ overhead on top of TCP is 142% or 11.574 nanoseconds !!! That's excessive. I would expect 1 microsecond overhead there. So questions remain: 1) What does ZeroMQ do under the rood that justifies so many extra clock cycles? (I am really curious to know) 2) Do people agree that 11 microseconds are just too much? My rough guess: ZeroMQ uses threads? (the beauty of NIO is that it is single-threaded, so there is always only one thread reading and writing to the network) -Julie On Wed, Aug 29, 2012 at 10:24 AM, Chuck Remes <li...@chuckremes.com<mailto:li...@chuckremes.com>> wrote: On Aug 29, 2012, at 10:13 AM, Julie Anderson wrote: Just tested ZeroMQ and Java NIO in the same machine. The results: - ZeroMQ: message size: 13 [B] roundtrip count: 100000 average latency: 19.620 [us] <====== ONE-WAY LATENCY - Java NIO Selector: (EPoll) Average RTT (round-trip time) latency of a 13-byte message: 15.342 [us] Min Time: 11.664 [us] 99.999% percentile: 15.340 [us] <====== RTT LATENCY Conclusion: That's 39.240 versus 15.340 so ZeroMQ overhead on top of TCP is 156% or 23.900 nanoseconds !!! That's excessive. I would expect 1 or 2 microseconds there. So my questions are: 1) What does ZeroMQ do under the rood that justifies so many extra clock cycles? (I am really curious to know) 2) Do people agree that 23 microseconds are just too much? As a favor to me, please rerun the tests so that at least 1 million (10 million is better) messages are sent. This shouldn't take more than a few minutes to run. Thanks. Secondly, are you using the local_lat and remote_lat programs that are included with zeromq or did you write your own? If you wrote your own, please share the code. Thirdly, a pastie containing the code for both tests so others could independently reproduce your results would be very handy. cr _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org<mailto:zeromq-dev@lists.zeromq.org> http://lists.zeromq.org/mailman/listinfo/zeromq-dev ----------------------------------------- Declaración de confidencialidad: Este Mensaje esta destinado para el uso de la o las personas o entidades a quien ha sido dirigido y puede contener información reservada y confidencial que no puede ser divulgada, difundida, ni aprovechada en forma alguna. El uso no autorizado de la información contenida en este correo podrá ser sancionado de conformidad con la ley chilena. Si usted ha recibido este correo electrónico por error, le pedimos eliminarlo junto con los archivos adjuntos y avisar inmediatamente al remitente, respondiendo este mensaje. "Before printing this e-mail think if is really necesary". Disclosure: This Message is to be used by the individual, individuals or entities that it is addressed to and may include private and confidential information that may not be disclosed, made public nor used in any way at all. Unauthorized use of the information in this electronic mail message may be subject to the penalties set forth by Chilean law. If you have received this electronic mail message in error, we ask you to destroy the message and its attached file(s) and to immediately notify the sender by answering this message.
_______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
