There are two distinct questions...
1. Regarding reactive streams, Akka has an implementation for Kafka:
https://github.com/akka/reactive-kafka
2. Kafka is not a queue. For example, it does not implement "dequeue"
operation.
All the message management / retention is not based on whether a message
was consumed or not. A topic may have a one month retention, so messages
will be deleted after one month, even if you consumed them after five
seconds. Which makes sense, as multiple (unlimited) number of consumer may
read the same messages, and each may decide to move backwards or forwards
in the log based on their need, etc etc.
So, Kafka separates producing messages, consuming messages and managing the
logs.

Ofir Manor

Co-Founder & CTO | Equalum

Mobile: +972-54-7801286 | Email: ofir.ma...@equalum.io

On Sat, Sep 17, 2016 at 10:49 PM, kant kodali <kanth...@gmail.com> wrote:

> Still it should be possible to implement using reactive streams right.
> Could you please enlighten me on what are the some major differences you
> see
> between a commit log and a message queue? I see them being different only
> in the
> implementation but not functionality wise so I would be glad to hear your
> thoughts.
>
>
>
>
>
>
> On Sat, Sep 17, 2016 12:39 PM, Radoslaw Gruchalski ra...@gruchalski.com
> wrote:
> Kafka is not a queue. It’s a distributed commit log.
>
>
>
>
> –
>
> Best regards,
>
> Radek Gruchalski
>
> ra...@gruchalski.com
>
>
>
>
>
>
>
> On September 17, 2016 at 9:23:09 PM, kant kodali (kanth...@gmail.com)
> wrote:
>
>
>
>
> Hmm...Looks like Kafka is written in Scala. There is this thing called
>
> reactive
>
> streams where a slow consumer can apply back pressure if they are consuming
>
> slow. Even with Java this is possible with a Library called RxJava and
>
> these
>
> ideas will be incorporated in Java 9 as well.
>
> I still don't see why they would pick poll just to solve this one problem
>
> and
>
> compensating on others. Poll just don't sound realtime. I heard from some
>
> people
>
> that they would set poll to 100ms. Well 1) that is a lot of time. 2)
>
> Financial
>
> applications requires micro second latency. Kafka from what I understand
>
> looks
>
> like has a very high latency and here is the article.
>
> http://bravenewgeek.com/dissecting-message-queues/ I usually don't go by
>
> articles but I ran my own experiments on different queues and my numbers
>
> are
>
> very close to this article so I would say whoever wrote this article has
>
> done a
>
> good Job. 3) poll does generate unnecessary traffic in case if the data
>
> isn't
>
> available.
>
> Finally still not sure why they would pick poll() ? or do they plan on
>
> introducing reactive streams?Thanks,kant
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Sep 17, 2016 5:14 AM, Radoslaw Gruchalski ra...@gruchalski.com
>
> wrote:
>
> I'm only guessing here regarding if this is the reason:
>
>
>
>
> Pull is much more sensible when a lot of data is pushed through. It allows
>
> consumers consuming at their own pace, slow consumers do not slow the
>
> complete
>
> system down.
>
>
>
>
>
>
>
>
>
>
>
>
>
> --
>
>
>
>
> Best regards,
>
>
>
>
> Rad
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> On Sat, Sep 17, 2016 at 11:18 AM +0200, "kant kodali" <kanth...@gmail.com>
>
> wrote:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> why did Kafka choose pull instead of push for a consumer? push sounds like
>
> it
>
>
>
>
> is more realtime to me than poll and also wouldn't poll just keeps polling
>
> even
>
>
>
>
> when they are no messages in the broker causing more traffic? please
>
> enlighten
>
>
>
>
> me
>

Reply via email to