Most folks strip the IP and use that as the broker.id. KAFKA-1070 does not yet accommodate for that very widely used method. I think it would be bad if KAFKA-1070 only worked for new installations because that is how people use Kafka today (per https://issues.apache.org/jira/browse/KAFKA-1070?focusedCommentId=14085808&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14085808 )
On Mon, Nov 3, 2014 at 2:12 PM, Joel Koshy <[email protected]> wrote: > KAFKA-1070 will help with this and is pending a review. > > On Mon, Nov 03, 2014 at 05:03:20PM -0500, Otis Gospodnetic wrote: > > Hi, > > > > How do people handle situations, and specifically the broker.id > property, > > where the Kafka (broker) cluster is not fully defined right away? > > > > Here's the use case we have at Sematext: > > * Our software ships as a VM > > * All components run in this single VM, including 1 Kafka broker > > * Of course, this is just for a nice OOTB experience, but to scale one > > needs to have more instances of this VM, including more Kafka brokers > > * *One can clone our VM and launch N instances of it, but because we > have a > > single Kafka broker config with a single broker.id <http://broker.id> in > > it, one can't just launch more of these VMs and expect to see more Kafka > > brokers join the cluster. One would have to change the broker.id > > <http://broker.id> on each new VM instance.* > > > > How do others handle this in a software that is packages and ships to > user > > and is not in your direct control to allow you to edit configs? > > > > Would it be best to have a script that connect to ZooKeeper to get the > list > > of all existing brokers and their IDs and then generate a new distinct > ID + > > config for the new Kafka broker? > > > > Or are there slicker ways to do this that people use? > > > > Thanks, > > Otis > > -- > > Monitoring * Alerting * Anomaly Detection * Centralized Log Management > > Solr & Elasticsearch Support * http://sematext.com/ > >
