On Wednesday 29, Julie Anderson wrote: > See my comments below: > > On Wed, Aug 29, 2012 at 4:06 PM, Robert G. Jakabosky > > <bo...@sharedrealm.com>wrote: > > On Wednesday 29, Julie Anderson wrote: > > > So questions remain: > > > > > > 1) What does ZeroMQ do under the rood that justifies so many extra > > > clock cycles? (I am really curious to know) > > > > ZeroMQ is using background IO threads to do the sending/receiving. So > > the extra latency is do to passing the messages between the application > > thread and > > the IO thread. > > This kind of thread architecture sucks for latency sensitive applications. > That's why non-blocking I/O exists. That's my humble opinion and the > numbers support it.
If low-latency is the most important thing for your application, then use a custom protocol & highly tuned network code. ZeroMQ is not a low-level networking library it provide some high-level features that are not available with raw sockets. If you are planing on doing high-frequency trading, then you will need to write your own networking code (or FPGA logic) to squeeze out every last micro/nanosecond. ZeroMQ is not going to be the right solution to every use- case. > > > 2) Do people agree that 11 microseconds are just too much? > > > > No. A simple IO event loop using epoll is fine for a IO (network) bound > > application, but if you need to do complex work (cpu bound) mixed with > > non- blocking IO, then ZeroMQ can make it easy to scale. > > Totally agree, but that has nothing to do with a financial application. > Financial applications do not need to do complex CPU bound analysis like a > image processing application would need. Financial application only cares > about LATENCY and network I/O. Not all Financial application care only about latency. For some system it is important to scale out to very large number of subscribers and large volume of messages. When comparing ZeroMQ to raw network IO for one connection, ZeroMQ will have more latency overhead. Try your test with many thousands of connections with subscriptions to lots of different topics, then ZeroMQ will start to come out ahead. > > Also try comparing the latency of Java NIO using TCP/UDP against ZeroMQ > > using > > the inproc transport using two threads in the same JVM instance. > > What is the problem with inproc? Just use a method call in the same JVM or > shared memory for different JVMs. If you want inter-thread communication > there are blazing-fast solutions in Java for that too. For example, I would > be surprised if ZeroMQ can come close to Disruptor for inter-thread > communication. ZeroMQ's inproc transport can be used in an event loop along side the TCP and IPC transports. With ZeroMQ you can mix-and-match transports as needed. If you can do all that with custom code with lower latency, then do it. ZeroMQ is for people who don't have the experience to do that kind of thread-safe programming, or just want to scale out there application. > > With ZeroMQ it is easy to do thread to thread, process to process, and/or > > server to server communication all at the same time using the same > > interface. > > This generic API is cool, but it is solving a problem financial systems do > not have and creating a bigger problem by adding latency. ZeroMQ is not adding latency for no reason. If you think that the latency can be eliminated, then go ahead and change the core code to not use IO threads. > > Basically ZeroMQ has different use-case then a simple IO event loop. > > I thought ZeroMQ flagship customers were financial institutions. Then maybe > I was wrong. ZeroMQ is competing with other Message-oriented middleware, like RabbitMQ, SwiftMQ, JMS, or other Message queuing systems. These systems are popular with financial institutions. -- Robert G. Jakabosky _______________________________________________ zeromq-dev mailing list zeromq-dev@lists.zeromq.org http://lists.zeromq.org/mailman/listinfo/zeromq-dev
