Manish Sharma created KAFKA-2471:
------------------------------------
Summary: Replicas Order and Leader out of sync
Key: KAFKA-2471
URL: https://issues.apache.org/jira/browse/KAFKA-2471
Project: Kafka
Issue Type: Bug
Affects Versions: 0.8.2.1
Reporter: Manish Sharma
Our 2 kafka brokers ( 1 & 5) were rebooted due to hypervisor going down and I
think we encountered a similar
issue that was discussed in thread "Problem with node after restart no
partitions?". The resulting JIRA is closed without conclusions or
recovery steps.
Our Brokers 5 and 1 were also running zookeeper of our cluster (along with
broker 2),
we are running kafka version 0.8.2.1
After doing a controlled restarts over all brokers a few times our cluster
seems ok now.
But there are a some topics that have replicas out of sync with Leaders.
Partition 2 below has Leader 5 and replicas order should be 5,1
{code}
Topic:2015-01-12 PartitionCount:3 ReplicationFactor:2 Configs:
Topic: 2015-01-12 Partition: 0 Leader: 4 Replicas: 4,3
Isr: 3,4
Topic: 2015-01-12 Partition: 1 Leader: 0 Replicas: 0,4
Isr: 0,4
Topic: 2015-01-12 Partition: 2 Leader: 5 Replicas: 1,5
Isr: 5
{code}
I tried reassigning partition 2 replicas to broker 5 (leader) and broker : 0
Now partition reassignment is stuck for more than a day.
%) /usr/local/kafka/bin/kafka-reassign-partitions.sh --zookeeper
kafka-trgt05:2182 --reassignment-json-file 2015-01-12_2.json --verify
Status of partition reassignment:
Reassignment of partition [2015-01-12,2] is still in progress
And In zookeeper, reassign_partitions is empty..
[zk: kafka-trgt05:2182(CONNECTED) 2] ls /admin/reassign_partitions
[]
This seems like a bug being triggered, that leaves the cluster in unhealthy
state.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
