Hmm...Looks like Kafka is written in Scala. There is this thing called reactive streams where a slow consumer can apply back pressure if they are consuming slow. Even with Java this is possible with a Library called RxJava and these ideas will be incorporated in Java 9 as well. I still don't see why they would pick poll just to solve this one problem and compensating on others. Poll just don't sound realtime. I heard from some people that they would set poll to 100ms. Well 1) that is a lot of time. 2) Financial applications requires micro second latency. Kafka from what I understand looks like has a very high latency and here is the article. http://bravenewgeek.com/dissecting-message-queues/ I usually don't go by articles but I ran my own experiments on different queues and my numbers are very close to this article so I would say whoever wrote this article has done a good Job. 3) poll does generate unnecessary traffic in case if the data isn't available. Finally still not sure why they would pick poll() ? or do they plan on introducing reactive streams?Thanks,kant
On Sat, Sep 17, 2016 5:14 AM, Radoslaw Gruchalski ra...@gruchalski.com wrote: I'm only guessing here regarding if this is the reason: Pull is much more sensible when a lot of data is pushed through. It allows consumers consuming at their own pace, slow consumers do not slow the complete system down. -- Best regards, Rad On Sat, Sep 17, 2016 at 11:18 AM +0200, "kant kodali" <kanth...@gmail.com> wrote: why did Kafka choose pull instead of push for a consumer? push sounds like it is more realtime to me than poll and also wouldn't poll just keeps polling even when they are no messages in the broker causing more traffic? please enlighten me
