Re: Question Kafka Reassign partitions tool

2017-12-26 Thread Sagar
Hi Todd, That worked. Basically I used a python library to read the leaders of a topic/partition from the following path on zookeeper: /brokers/topics/[topic]/partitions/[partitionId]/state. Then I was able to get the correct leader and assign the replica set so that the current leader remains

Re: Question Kafka Reassign partitions tool

2017-12-22 Thread Todd Palino
Yes, the replicas are stored in Zookeeper, so you can iterate over the information there to build a view of the cluster that you can use. If you want an example for this, take a look at the code for kafka-assigner in https://github.com/linkedin/kafka-tools. Or you can just use that tool to adjust

Re: Question Kafka Reassign partitions tool

2017-12-22 Thread Sagar
Hi Todd, Thanks for the reply. Problem is I have about 160 topics(5 partitions for each) for which I need to increase the replication factors for. So, I would have to find the current leader for each of the partitions and hand code the json which would become tedious. The partition leader info

Re: Question Kafka Reassign partitions tool

2017-12-22 Thread Todd Palino
Preferred replica election is naive. It will always follow the order of the replicas as they are set. So if you want to set the default leader, just make it the first replica in the list for the partition. We build the JASON this way all the time. -Todd On Dec 22, 2017 6:46 AM, "Sagar"

Question Kafka Reassign partitions tool

2017-12-22 Thread Sagar
Hi, Had a question on Kafka reassign partitions tool. We have a 3 node cluster but our replication factor is set to 1. So we have been looking to increase it to 3 for HA. I tried the tool on a couple of topics and it increases the replication factor alright. Also it doesn't change the leader as