I am trying to understand and document how producers consumers
will/should behave in case of node failures in 0.8. I know there are
various other threads that discuss this but I wanted to bring all the
information together in one post. This should help people building
producers consumers in
Yes. And during retries, the producer and consumer refetch metadata.
Thanks,
Neha
On Oct 24, 2013 3:09 AM, Aniket Bhatnagar aniket.bhatna...@gmail.com
wrote:
I am trying to understand and document how producers consumers
will/should behave in case of node failures in 0.8. I know there are
Thanks Neha
On 24 October 2013 18:11, Neha Narkhede neha.narkh...@gmail.com wrote:
Yes. And during retries, the producer and consumer refetch metadata.
Thanks,
Neha
On Oct 24, 2013 3:09 AM, Aniket Bhatnagar aniket.bhatna...@gmail.com
wrote:
I am trying to understand and document how
publishing to and consumption from the partition will halt
and will not resume until the faulty leader node recovers
Can you confirm that's the case? I think they won't wait until leader
recovered and will try to elect new leader from existing non-ISR replicas?
And in case if they wait, and
Yes, when a leader dies, the preference is to pick a leader from the ISR.
If not, the leader is picked from any other available replica. But if no
replicas are alive, the partition goes offline and all production and
consumption halts, until at least one replica is brought online.
Thanks,
Neha
Thank you for posting these guidelines. I'm wondering if anyone out there
that is using the Kafka spout (for Storm) knows whether or not the Kafka
spout takes care of these types of details ?
regards
-chris
On Thu, Oct 24, 2013 at 2:05 PM, Neha Narkhede neha.narkh...@gmail.comwrote:
Yes,
I am planning to use kafka 0.8 spout and after studing the source code
found that it doesnt handle errors. There is a fork that adds try catch
over using fetchResponse but my guess is this will lead to spout attempting
the same partition infinitely until the leader is elected/comes back
online. I
