For the 1st question, I don't quite get the answer. Say I have broker 1,2,3,4,5,6. And the topic A has three partitions on broker 1,2,3. If I add another 3 partitions for A. They will occupy 4, 5, 6?
One more question. Does the partition addition tool work only with zookeeper? Can I keep the zk cluster running and shut down kafka processes before I run the partition addition tool? I am asking this question because we try to avoid any message loss when adding a partition in a production environment. Thanks. Regards, Libo -----Original Message----- From: Neha Narkhede [mailto:[email protected]] Sent: Monday, December 16, 2013 12:13 PM To: [email protected] Subject: Re: cluster expansion If we have six brokers, and a topic has three partitions on broker 1, 2, 3. Now if I add another three partitions. Will they be evenly distributed to 1,2,3 or 4,5,6? Evenly distributed on all six brokers. If I use reassignment tool in 0.81 with 0.8 broker, will that work and get around the bugs? Your broker also needs to be on 0.8.1 for it to work correctly. On Mon, Dec 16, 2013 at 9:06 AM, Yu, Libo <[email protected]> wrote: > If we have six brokers, and a topic has three partitions on broker 1, 2, 3. > Now if I add another three partitions. Will they be evenly distributed > to > 1,2,3 or 4,5,6? > > One more question. > If I use reassignment tool in 0.81 with 0.8 broker, will that work and > get around the bugs? > > Regards, > > Libo > > > -----Original Message----- > From: Neha Narkhede [mailto:[email protected]] > Sent: Monday, December 16, 2013 10:26 AM > To: [email protected] > Subject: RE: cluster expansion > > They will be evenly distributed across the nodes in the cluster. > > Thanks, > Neha > On Dec 16, 2013 6:42 AM, "Yu, Libo" <[email protected]> wrote: > > > Assume we have three brokers and a topic already has three partitions. > > Now if I add another three partitions to this topic. Where will the > > three partitions be located after running the script? > > > > Regards, > > > > Libo > > > > > > -----Original Message----- > > From: Jun Rao [mailto:[email protected]] > > Sent: Sunday, December 15, 2013 11:43 PM > > To: [email protected] > > Subject: Re: cluster expansion > > > > Yes, kafka-add-partitons.sh should be stable in 0.8.0. > > > > Thanks, > > > > Jun > > > > > > On Sat, Dec 14, 2013 at 1:22 AM, Robert Turner <[email protected]> > wrote: > > > > > Is the kafka-add-partitons.sh tool stable in 0.8.0? > > > > > > > > > On 13 December 2013 19:21, Neha Narkhede <[email protected]> > > wrote: > > > > > > > Partition movement is not an automatic operation in Kafka yet. > > > > You need > > > to > > > > use the partition reassignment tool - > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/Replication+tool > > > s# Re plicationtools-6.ReassignPartitionsTool > > > > . > > > > > > > > > > > > Also, that feature is stable in 0.8.1. > > > > > > > > Thanks, > > > > Neha > > > > > > > > > > > > On Fri, Dec 13, 2013 at 6:48 AM, Robert Turner > > > > <[email protected]> > > > wrote: > > > > > > > > > No the 6 partitions for each topic will remain on the original > > brokers. > > > > You > > > > > could either reassign some partitions from all topics to the > > > > > new > > > brokers > > > > or > > > > > you could add partitions to the new brokers for each topic. In > > > > > 0.8.0 > > > > there > > > > > is now an add-partitions tool. > > > > > > > > > > Cheers > > > > > Rob Turner. > > > > > > > > > > > > > > > On 13 December 2013 14:42, Yu, Libo <[email protected]> wrote: > > > > > > > > > > > Hi folks, > > > > > > > > > > > > There are three brokers running 0.8-beta1 in our cluster > currently. > > > > > Assume > > > > > > all the topics have six partitions. > > > > > > I am going to add another three brokers to the cluster and > > > > > > upgrade > > > all > > > > of > > > > > > them to 0.8. My question is after the cluster is up, will > > > > > > the partition be evenly distributed to all brokers? Thanks. > > > > > > > > > > > > Regards, > > > > > > > > > > > > Libo > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Cheers > > > > > Rob. > > > > > > > > > > > > > > > > > > > > > -- > > > Cheers > > > Rob. > > > > > >
