Hi Colin, I just wanted to make sure I’m on the right track, so I’m sharing my understanding of the current process for adding a controller to the voter set.
Here's what I think it looks like: Step 1: Use kafka-storage.sh format to format the controller. Step 2: Start up the controller and ensure it starts up successfully. Step 3: Provide the controller config and use the kafka-metadata-quorum.sh add-controller command to add the controller to the voter set. The KIP aims to simplify "Step 3". It removes the need to provide the controller config, as all the information that add-controller requires—like the directory UUID and controller endpoint— can be gathered through Admin#describeMetadataQuorum and Admin#describeConfigs. Since the controller is already in the cluster by Step 2, this information should be available. So, users would still need to complete "Step 1" and "Step 2". Otherwise, as you mentioned, we won’t be able to retrieve the controller UUID and its corresponding endpoints for Step 3. I’m not sure if I’ve misunderstood anything, so please let me know if there’s anything I’ve missed. Thanks so much for reviewing my KIP! Best regards, Kuan-Po Tseng On Tue, Apr 29, 2025 at 8:50 AM Colin McCabe <cmcc...@apache.org> wrote: > So to expand a bit on what I wrote below: > > - If the goal is to make it possible to have adding the controller node > happen in one place, and formatting the controller node happen in another, > those two places still need to communicate. What they need to communicate > is the new directory ID. This is basically a random UUID that must never be > reused. > > - The most obvious way to do that is to format the new controller node > first, and then grab that ID. This is what the current system does, by > reading that local file. > > - Another way to do it would be to generate the UUID somewhere else, and > then format using --initial-controllers. This currently requires spelling > out the full set of controllers and matching directory ids in the cluster. > One thing that is a bit subtle is that the tool will actually look at what > directory ID you gave, and use that directory ID when formatting your > directory. This should be documented a bit more prominently if we want > people to use this mechanism. > > best, > Colin > > > On Mon, Apr 28, 2025, at 13:21, Colin McCabe wrote: > > Hi Kuan Po Tseng, > > > > It is not possible to add a new controller purely by ID. You need > > controller ID, directory ID, and controller endpoints. > > > > These cannot be obtained through describeMetadataQuorum since that only > > describes the existing quorum, not what you would like the new one to > > be. Also, obviously, we never want to reuse an existing directory ID. > > > > Therefore I will vote -1 on this pending further discussion. > > > > best, > > Colin > > > > On Tue, Apr 22, 2025, at 08:56, Kuan Po Tseng wrote: > >> Hello everyone, > >> > >> I would like to start a vote on KIP-1141, thank you! > >> > >> KIP: https://cwiki.apache.org/confluence/x/PZFEF > >> Discussion thread: > >> https://lists.apache.org/thread/mq5n8j8w0ycg9fhd1m4zcc2ctrgfd0d4 > >> > >> Best, > >> Kuan-Po Tseng >
