I could go either way. I think that if ZK is up, then Kafka's going to go
crazy trying to figure out who's the master of what, but maybe I'm not thinking
the problem through clearly.
That does beg the issue: it seems like it'd be good to have something
written down somewhere to say how
That's not true Shaninder. Assuming you are using the high level consumer,
if all the consumers have different group id, each of them will get the all
the messages.
Also, another way to think about this a little bit is that maybe you can
produce the same data to many different topics each have
You are looking for a K-V store here, so the general answer is no. But
Kafka does have an internal K-V store but only for consumer offsets. So
there are some tricks we can play:
If your processing node is consuming from Kafka and the offset of a
snapshot is the offset of a Kafka partition, you
Well, you're right. But that's not what I meant. I meant using consumers
with the same consumer id. With different IDs, obviously the problem (and
solution) changes. But then I think the OP is just trying to twist his
problem to fit in the kafka way of things.
--
Sharninder
On Sat, Aug 8, 2015
Thanks Scott Monika.
I just followed the Kafka documentation to start the zookeeper brokers. They
did not mention of this zookeeper id maintenance.
One more observation is, it works fine after several retries. Sometimes the
zookeepers could not recognize the brokers etc.
-Original
