Wondering what is the reason behind decoupling the node id with its physical host(port)? If we found that for example, node 1 is not owning any partitions, how would we know which physical machine is this node then?
Guozhang On Wed, Oct 2, 2013 at 9:07 AM, Jay Kreps <jay.kr...@gmail.com> wrote: > I'm in favor of doing this if someone is willing to work on it! I agree it > would really help with easy provisioning. > > I filed a bug to discuss and track: > https://issues.apache.org/jira/browse/KAFKA-1070 > > Some comments: > 1. I'm not in favor of having a pluggable strategy, unless we are really > really sure this is an area where people are going to get a lot of value by > writing lots of plugins. I am not at all sure why you would want to retain > the current behavior if you had a good strategy for automatically > generating ids. Basically plugins are an evil we only want to accept when > either we don't understand the problem or the solutions have such extreme > tradeoffs that there is no single "good approach". Plugins cause problems > for upgrades, testing, documentation, user understandability, code > understandability, etc. > 2. The node id can't be the host or port or anything tied to the physical > machine or its location on the network because you need to be able to > change these things. I recommend we just keep an integer. > > -Jay > > > On Tue, Oct 1, 2013 at 7:08 AM, Aniket Bhatnagar < > aniket.bhatna...@gmail.com > > wrote: > > > Right. It is currently java integer. However, as per previous thread, it > > seems possible to change it to a string. In that case, we can use > instance > > IDs, IP addresses, custom ID generators, etc. > > How are you currently generating broker IDs from IP address? Chef script > or > > custom shell script? > > > > > > On 1 October 2013 18:34, Maxime Brugidou <maxime.brugi...@gmail.com> > > wrote: > > > > > I think it currently is a java (signed) integer or maybe this was > > > zookeeper? > > > We are generating the id from IP address for now but this is not ideal > > (and > > > can cause integer overflow with java signed ints) > > > On Oct 1, 2013 12:52 PM, "Aniket Bhatnagar" < > aniket.bhatna...@gmail.com> > > > wrote: > > > > > > > I would like to revive an older thread around auto generating broker > > ID. > > > As > > > > a AWS user, I would like Kafka to just use the instance's ID or > > > instance's > > > > IP or instance's internal domain (whichever is easier). This would > > mean I > > > > can easily clone from a AMI to launch kafka instances without having > to > > > > worry about setting a unique broker ID. This also alows me to setup > > auto > > > > scaling. > > > > > > > > I realize 1 size may not fit all in this case. Other strategies that > > may > > > > work for other cloud providers are generate the UUID and persist it > on > > a > > > > disk, etc. > > > > > > > > What I propose is a way to define a a broker ID generation strategy > in > > > the > > > > configuration file which points to a class file that is responsible > for > > > > generating the ID. Is this something being already worked upon? > > > > > > > > > > -- -- Guozhang
