New numbers (fun!). Firstly, to make sure I was comparing apples with apples, I modified my tests to compute one-way trip instead of round-trip. I can't paste code, but I am simply using a Java NIO (non-blocking I/O) optimized with busy spinning to send and receive tcp data. This is *standard* Java NIO code, nothing too fancy. You can google around for Java NIO. I found this link<http://www.cordinc.com/blog/2010/08/java-nio-server-example.html>that shows the basics. You can also do the same thing in C as you can see here<http://stackoverflow.com/questions/27247/could-you-recommend-some-guides-about-epoll-on-linux/6150841#6150841> .
My test now consists of: - JVM A sends a message which consist of the ascii representation of a timestamp in nanos. - JVM B receives this message, parses the long, computer the one-way latency and echoes back the message to JVM A. - JVM A receives the echo, parses the ascii long and makes sure that it matches the one it sent out. - Loop back and send the next message. So now I have both times: one-way and round-trip. I ran my test for 1 million messages over loopback. For ZeroMQ I am using the local_lat and remote_lat programs included with latest zeromq from here: git://github.com/zeromq/libzmq.git The results: *- ZeroMQ:* ./local_lat tcp://lo:5555 13 1000000 ./remote_lat tcp://127.0.0.1:5555 13 1000000 message size: 13 [B] roundtrip count: 1000000 average latency: *19.674* [us] *<==== this is one-way* *- Java NIO:* (EPoll with busy spinning) Round-trip: Iterations: 1,000,000 | Avg Time: *16552.15 nanos* | Min Time: 12515 nanos | Max Time: 129816 nanos | 75%: 16290 nanos | 90%: 16369 nanos | 99%: 16489 nanos | 99.999%: *16551 nanos* One-way trip: Iterations: 1,110,000 | Avg Time: *8100.12 nanos* | Min Time: 6150 nanos | Max Time: 118035 nanos | 75%: 7966 nanos | 90%: 8010 nanos | 99%: 8060 nanos | 99.999%: *8099 nanos* *Conclusions:* That's *19.674 versus 8.100* so ZeroMQ overhead on top of TCP is *142%* or *11.574 nanoseconds* !!! That's excessive. I would expect 1 microsecond overhead there. So questions remain: 1) What does ZeroMQ do under the rood that justifies so many extra clock cycles? (I am really curious to know) 2) Do people agree that 11 microseconds are just too much? My rough guess: ZeroMQ uses threads? (the beauty of NIO is that it is single-threaded, so there is always only one thread reading and writing to the network) -Julie On Wed, Aug 29, 2012 at 10:24 AM, Chuck Remes <[email protected]> wrote: > > On Aug 29, 2012, at 10:13 AM, Julie Anderson wrote: > > Just tested ZeroMQ and Java NIO in the same machine. > > The results: > * > - ZeroMQ:* > > message size: 13 [B] > roundtrip count: 100000 > average latency: *19.620* [us] *<====== ONE-WAY LATENCY* > > *- Java NIO Selector:* (EPoll) > > Average RTT (round-trip time) latency of a 13-byte message: 15.342 [us] > Min Time: 11.664 [us] > 99.999% percentile: *15.340* [us] *<====== RTT LATENCY* > > *Conclusion:* That's *39.240 versus 15.340* so ZeroMQ overhead on top of > TCP is *156%* or *23.900 nanoseconds* !!! That's excessive. I would > expect 1 or 2 microseconds there. > > So my questions are: > > 1) What does ZeroMQ do under the rood that justifies so many extra clock > cycles? (I am really curious to know) > > 2) Do people agree that 23 microseconds are just too much? > > > As a favor to me, please rerun the tests so that at least 1 million (10 > million is better) messages are sent. This shouldn't take more than a few > minutes to run. Thanks. > > Secondly, are you using the local_lat and remote_lat programs that are > included with zeromq or did you write your own? If you wrote your own, > please share the code. > > Thirdly, a pastie containing the code for both tests so others could > independently reproduce your results would be very handy. > > cr > > > _______________________________________________ > zeromq-dev mailing list > [email protected] > http://lists.zeromq.org/mailman/listinfo/zeromq-dev > >
_______________________________________________ zeromq-dev mailing list [email protected] http://lists.zeromq.org/mailman/listinfo/zeromq-dev
