Hi Colin,
What we want to is to preserve the broker.id so that we can do an
offline rebuild of a broker. In our cases going through online Kafka
replication to bring up, a failed node will put producer latencies at risk
given the new broker will put all the other leaders busy with its replication
requests. For an offline rebuild, we do not need to do rebalance as long as we
can recover the broker.id
Overall, irrespective of this use case we still want an ability to
retrieve a broker.id for an existing host. This will make swapping in new hosts
with failed hosts by keeping the existing hostname easier.
Thanks,
Harsha
On Wed, Feb 27, 2019, at 11:53 AM, Colin McCabe wrote:
> Hi Li,
>
> > The mechanism simplifies deployment because the same configuration can be
> > used across all brokers, however, in a large system where disk failure is
> > a norm, the meta file could often get lost, causing a new broker id being
> > allocated. This is problematic because new broker id has no partition
> > assigned to it so it can’t do anything, while partitions assigned to the
> > old one lose one replica
>
> If all of the disks have failed, then the partitions will lose their
> replicas no matter what, right? If any of the disks is still around,
> then there will be a meta file on the disk which contains the previous
> broker ID. So I'm not sure that we need to change anything here.
>
> best,
> Colin
>
>
> On Tue, Feb 5, 2019, at 14:38, Li Kan wrote:
> > Hi, I have KIP-426, which is a small change on automatically determining
> > broker id when starting up. I am new to Kafka so there are a bunch of
> > design trade-offs that I might be missing or hard to decide, so I'd like to
> > get some suggestions on it. I'd expect (and open) to modify (or even
> > totally rewrite) the KIP based on suggestions. Thanks.
> >
> > --
> > Best,
> > Kan
> >
>
