Hi Jun,
I modified the UpdateOffsetsInZK so that it could take a custom offset and
set it not sure of the consequences though.
The ZKConsumer can now get the messages from the desired offset , So Its
working as of now need to test more though .
Modified method
--
Kafka-users,
I'm looking to use kafka in a pub-sub model where a consumer reads from Kafka
and does some processing on the message. How would you recommend a commit to
Zookeeper / setting the last message consumed location if processing one of the
messages in the pipe is more unreliable than t
Are there any good solutions to scaling with lots of small topics in Kafka?
I'm talking about say 100,000 unique, but low-volume topics with dozens of
subscribers mixed in with say 100 high volume topics?
Thanks
Vito
** Resending due to outlook.com corrupting the original message. Sorry :) **
Kafka-users,
I'm looking to use kafka in a pub-sub model where a consumer reads
from Kafka and does some processing on the message. How would you
recommend a commit to Zookeeper / setting the last message consumed
locat
In this case, you have to wait until 9 is processed before you commit the
offset.
Thanks,
Jun
On Thu, Sep 13, 2012 at 8:52 AM, Vito Sabella wrote:
> Kafka-users,
>
> I'm looking to use kafka in a pub-sub model where a consumer reads from
> Kafka and does some processing on the message. How wo
In 0.7, each topic exists on every broker. So it's not scalable. In 0.8, a
topic only exists on a small number of brokers (replication factor). To
support more topics, you can just add more brokers.
Thanks,
Jun
On Thu, Sep 13, 2012 at 8:59 AM, Vito Sabella wrote:
> Are there any good solutions
