Some more details about my use-case
When i add the new Kafka Broker, it will not be included in the ISR of the Kafka Topic automatically. We'll need to do Kafka rebalancing - so the load gets balanced across all the Brokers, including the newly added Broker. Can this be done while the Producer is pushing data into the Kafka Topic ? What are the implications of this ? Pls. advise. On Tue, Jun 13, 2017 at 4:29 PM, karan alang <[email protected]> wrote: > > Hello, clarifying the question -> > > Within the same Kafka cluster, if i want to add Broker & rebalance topic > (say topic t1, so it starts using newly added Broker) - can i do that w/o > stopping the existing load on topic t1 ? > > I guess - if i can do that, i should be able to use MirrorMaker to > replicate Topic t1 on cluster c1 => Topic t1 on Cluster t2 > > Is that the best way of doing it ? > > > > > On Tue, Jun 13, 2017 at 3:14 PM, Mohammed Manna <[email protected]> > wrote: > >> You should plan this ahead and make sure that your topic partition and >> replication factors are updated correctly using kafka topics admin or a >> batch job setup for Kafka Mirror Maker. Apache wiki explains how Mirror >> maker can guarantee a failsafe mirroring. I cannot quite remember how deep >> the mirroring goes e.g. Consumer offsets etc. But my understanding is that >> it should do it correctly. >> >> Regards, >> >> On Tue, 13 Jun 2017 at 10:58 pm, karan alang <[email protected]> >> wrote: >> >> > Hello All, >> > >> > I've streaming data flowing into Kafka topics. >> > What are the best practices wrt. adding or removing Kafka Brokers while >> > data is flowing into Kafka ? >> > >> > for eg. Can I add Broker & re-balance the topics while the data flows >> into >> > Kafka topic ? >> > >> > Pls. advise. >> > >> > Thanks! >> > >> > >
