Hi all,
the current official documentation differentiates between formatting
storage for multiple controllers when starting a new cluster versus adding
a new controller during scale-up operations.
For a new cluster, the "initial controllers" must be provided (-I flag in
the storage tool), which creates a checkpoint file that controllers use to
discover each other and form the quorum.
For scale-up, the documentation indicates formatting should be done without
providing initial controllers (using the -N flag instead).

What I've discovered is that using --initial-controllers during scale-up
also works correctly. For example:
- Start with 3 controllers (0, 1, 2) formatted with --initial-controllers
0,1,2
- When scaling to 4 controllers, format controller 3 with
--initial-controllers 0,1,2,3
- The new controller successfully discovers the quorum by fetching metadata
(the local checkpoint file is created but not used for discovery)
- The add-controller operation via admin API is still required

As you can see, the new controller starts, fetches the metadata and becomes
an observer. It doesn't join the quorum automatically until the
add-controller operation.

This approach provides consistency, especially valuable in automated
deployment environments, as there's no need to distinguish between initial
cluster setup and scale-up operations.

I've opened PR #21507 <https://github.com/apache/kafka/pull/21507> with a
test demonstrating this works as expected.

Are there any thoughts about this or concerns from the community to
consider this procedure still valid?

Thanks,
Paolo Patierno

-- 
Paolo Patierno

*Senior Principal Software Engineer @ IBM**CNCF Ambassador*

Twitter : @ppatierno <http://twitter.com/ppatierno>
Linkedin : paolopatierno <http://it.linkedin.com/in/paolopatierno>
GitHub : ppatierno <https://github.com/ppatierno>

Reply via email to