Jun, appreciate your prompt response.
need some more inputs in understanding this better. Is the following understanding appropriate. 1. When a new broker comes in in the cluster, it would register in broker registry but won't be subscribed to any topics. 2. re-balance triggered by consumers will get some partitions of existing topics to this broker. 3. If we are using a VIP to front brokers, how do they get all the information from ZK about what topics/partitions should go to which broker for appropriate routing. Thanks, - Inder Yahoo India R&D. On Mon, Oct 17, 2011 at 8:21 PM, Jun Rao <jun...@gmail.com> wrote: > Inder, > > 1. Existing data doesn't get moved to the new broker. Some of the new data > will get routed to the new broker once the producer recognizes the new > broker (either through ZK-based producer or vip). > > 2. Yes. Once the new broker gets data for a new topic, consumers will be > notified and start rebalance. > > Thanks, > > Jun > > On Sun, Oct 16, 2011 at 10:29 PM, Inder Pall <inder.p...@gmail.com> wrote: > > > Hi, > > > > I am trying to understand kafka by going through documents/wiki's for > > usage. > > Please pardon if the dev mailing list isn't appropriate to be in the TO > > for > > this query. > > > > Couple of queries. > > > > 1. If you add new broker boxes, how will existing topics and their > > partitions be redistributed on the new broker. > > 2. Does [1] trigger a re-balance process via watchers in which case if > you > > wanna redistribute only certain event types on the new broker, can it be > > achieved dynamically. > > > > > > -- Inder > > > -- -- Inder
