Hi All,
We recently performed a partition reassignment on one of our Kafka cluster.
(0.8.1.1, all topics configured to have 2 replicas.)
After the completion of reassignment, we have noticed that some of the
partitions has 3 replicas and 3 Isrs instead of 2.
for example:
Topic: topic1 Partition: 7 Leader: 5 Replicas: 5,2 Isr: 5,2
(correct number of replicas and Isr)
Topic: topic1 Partition: 8 Leader: 6 Replicas: 6,4,2 Isr: 6,4,2
(has more replica and Isr than configured)
Topic: topic1 Partition: 32 Leader: 5 Replicas: 5,4,2 Isr: 5,4,2
Previously, we have seen 3 replicas during the reassignment, however, the Isr
count was always 2, and replica count will drop back to 2 after the completion
of reassignment.
Just wonder if someone has encountered similar issue. What would be a good way
to fix it? I am thinking about triggering a partial partition reassignment with
a manually created partition assignment config that would bring the replicas
back to 2. Is this the right way to fix such issue?
Thanks,
Ye
P.S.
Here are the step we did for the reassignment:
1. /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper
<zookeeper_host_name>:<port> --broker-list "<broker_list>" --generate
--topics-to-move-json-file <topics-to-move_json_file>
2. From the console output, copy the second part and put it into a text file
assignment_json_file.
3. /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper
<zookeeper_host_name>:<port> --broker-list "<broker_list>" --execute
--topics-to-move-json-file <topics-to-move_json_file> --reassignment-json-file
<assignment_json_file>
