Appreciate if anyone could shed some light to this issue. Thank you.
On Sun, Mar 17, 2019, 12:06 PM Devang Shah <devangsha...@gmail.com> wrote: > Dear Kafka Users, > > We have recently discovered *NotLeaderForPartition *in our Kafka producer. > > *Below are the versions we are using,* > Kafka Version (server) - 2.11.0-0.10.2.0 > Kafka Client - 0.11.0.1 > Spring Kafka - 1.3.3 > Camel Kafka - 2.20.2 > > *Background* > We have 4 brokers on Kafka cluster. We have one topic with four partitions > and replication count of one. We have one producer which sends to specific > partition (one of the four partitions) based on key settings and four > consumers (one for each partition). On the incident day, two of the brokers > were down (was unnoticed). The other two brokers were running fine but > suddenly we started receiving *NotLeaderForPartition *in the producer > logs. I observed certain logs on kafka server which said "Replication > thread shutdown successfully" and it started the truncation process. > > *Temporary issue resolution* > The Producer/Consumer were bounced for our application to start processing > the messages. The Kafka cluster was not bounced and the other two brokers > were still down. > > *Questions/Queries,* > 1. Does the Truncation of logs on Kafka brokers affect the > application process? Would it make the kafka cluster offline or the > truncation happens in the background? > 2. We have the the property *log.retention.check.interval.ms > <http://log.retention.check.interval.ms>* set to *300000 (every 5 mins)*. > I think this is set to default and we can make a daily/weekly activity to > reduce load on the kafka brokers and maximize Kafka brokers dedication to > application processing. > 3. Is there any known bug in *kafka-client-0.11.0.1* with respect to > re-connection after a network glitch. > > Any pointers to resolve the above will be helpful. Thank you. > > Thanks & Regards, > Devang >
