Hi
We were on Kafka 0.10.2.1. While upgrading to 1.1, we bring down all the 3
kafka brokers, and make the change in the config file as shown below which
is recommend in http://kafka.apache.org/11/documentation.html#upgrade and
restart the brokers:
*inter.broker.protocol.version=1.1*
*log.message.format.version=0.10.2*
The symptoms that we are seeing is here:
1. There seems to be leader issue post upgrade. For example, if we describe
the topic, we see content like below where Leader is -1 and Isr is empty.
Topic: topic-5aeea219497d4f408504d1af Partition: 0 Leader: -1
Replicas: 3,2 Isr:
Topic: topic-5aeea219497d4f408504d1af Partition: 1 Leader: 1
Replicas: 1,3 Isr: 3,1
Topic: topic-5aeea219497d4f408504d1af Partition: 2 Leader: -1
Replicas: 2,1 Isr:
Topic: topic-5aeea219497d4f408504d1af Partition: 3 Leader: 3
Replicas: 3,1 Isr: 3,1
Topic: topic-5aeea219497d4f408504d1af Partition: 4 Leader: -1
Replicas: 1,2 Isr:
Topic: topic-5aeea219497d4f408504d1af Partition: 5 Leader: -1
Replicas: 2,3 Isr:
Topic: topic-5aeea219497d4f408504d1af Partition: 6 Leader: -1
Replicas: 3,2 Isr:
2. We also saw zookeeper log with messages like there:
2018-05-09_07:05:45.94181 2018-05-09 07:05:45,940 [myid:1] - WARN
[WorkerSender[myid=1]:QuorumCnxManager@400] - Cannot open channel to 3 at
election address kafka-3/1.1.1.144:3888
2018-05-09_07:05:45.94182 java.net.ConnectException: Connection refused
Does anyone know any known caveats or gotchas while upgrading Kafka version
?
Thanks.
--Darshan
