Hi, Chia-Ping, Thanks for the proposal. It could work. Do you want to flesh that out as a separate KIP?
What's your suggestion for https://issues.apache.org/jira/browse/KAFKA-20107? Should we implement it before or after your proposal is in place? Thanks, Jun On Thu, Feb 19, 2026 at 12:03 AM Chia-Ping Tsai <[email protected]> wrote: > A rough idea is that we could add a tagged field to the broker > registration to contain the static properties that have validators in their > config definition. The goal is to empower the controller to do more > validation before approving the upgrade > > On 2026/02/17 19:28:08 Jun Rao via dev wrote: > > Hi, Chia-Ping, > > > > Thanks for the reply. > > > > I am not sure about the MV approach. For example, a user could set > > `segment.bytes` through a static config file. How will MV work in that > case? > > > > Jun > > > > On Sun, Feb 15, 2026 at 12:17 AM Chia-Ping Tsai <[email protected]> > wrote: > > > > > Hi, Jun > > > > > > I prefer to ensure users are aware of what happens with their > > > configuration. Clamping seems to hide the actual behaviour from users, > > > which may lead to potential issues. > > > > > > Here is a crazy idea: what if the validation is bound to the MV? > > > > > > For example, `segment.bytes` could have different lower bounds for 4.0 > and > > > 4.1. This allows users to safely keep the smaller value while > upgrading the > > > binary files. However, they must update the configurations to meet the > new > > > requirement before bumping the MV. Otherwise, they will receive an > error > > > when calling `updateFeatures` > > > > > > This approach not only offers a smooth upgrade path bug also ensures > the > > > final cluster state adheres to the strict validation rules once the MV > is > > > updated. > > > > > > WDYT? > > > > > > Best, > > > Chia-Ping > > > > > > On 2026/02/09 17:08:19 Jun Rao via dev wrote: > > > > Hi, Chia-Ping, > > > > > > > > Thanks for the reply. Do you agree that it's better to use the > clamping > > > > approach for segment.bytes ? > > > > > > > > Jun > > > > > > > > On Tue, Feb 3, 2026 at 11:15 AM Chia-Ping Tsai <[email protected]> > > > wrote: > > > > > > > > > hi Jun > > > > > > > > > > I agree that we should apply reasonable strategies for different > > > `breaking > > > > > changes`. This makes ensuring that the initialization process sees > the > > > > > 'latest' configs even more critical. It is otherwise hard to > guarantee > > > that > > > > > the broker fails at the appropriate time when a `fail-fast` policy > is > > > > > intended > > > > > > > > > > Best, > > > > > Chia-Ping > > > > > > > > > > On 2026/01/30 22:13:00 Jun Rao via dev wrote: > > > > > > Hi, Chia-Ping, > > > > > > > > > > > > Thanks for following up on this. > > > > > > > > > > > > To me, there are two categories of changes. The first type > involves > > > > > changes > > > > > > that always benefit the user. An example is segment.bytes since > > > there is > > > > > no > > > > > > good reason for a user to ever set it to less than 1MB. In this > > > case, one > > > > > > option is to automatically clamp the value to the new lower bound > > > without > > > > > > failing. This is probably best for users because they get better > > > > > > configuration and compatibility. This is essentially the approach > > > that we > > > > > > followed in KIP-1161 for handling duplicates in the List config. > The > > > > > second > > > > > > category includes changes where disallowed values may impact a > user's > > > > > > intention. An example is segment.ms (change not implemented > yet, but > > > > > > targeted for 5.0). A user may intentionally set it to a really > small > > > > > value > > > > > > to control the retention time. In this case, it's probably > better to > > > fail > > > > > > the broker with an invalid config error so that the users know > about > > > it. > > > > > > > > > > > > Jun > > > > > > > > > > > > > > > > > > On Wed, Nov 19, 2025 at 11:18 AM Chia-Ping Tsai < > [email protected] > > > > > > > > > wrote: > > > > > > > > > > > > > hi all > > > > > > > > > > > > > > There was a long discussion about compatibility after > increasing > > > the > > > > > lower > > > > > > > bound ( > > > > > https://github.com/apache/kafka/pull/20334#discussion_r2538011530 > ). > > > > > > > > > > > > > > In summary, increasing the lower bound results in the following > > > issues: > > > > > > > > > > > > > > 1) Static configurations created before KIP-1030 cause the > broker > > > to > > > > > fail. > > > > > > > For example, setting log.segment.bytes=1000 will now encounter > a > > > > > validation > > > > > > > error due to the new lower bound when starting the updated > broker. > > > > > > > > > > > > > > 2) Dynamic configurations created before KIP-1030 cannot be > applied > > > > > due to > > > > > > > validation errors. > > > > > > > > > > > > > > Personally, I believe users should be explicitly aware of > breaking > > > > > > > changes. Therefore, I suggest forcing the broker to fail when > > > > > initializing > > > > > > > the metadata publisher if the dynamic configuration cannot be > > > applied. > > > > > > > > > > > > > > A softer approach is to use warnings instead of fatal errors. > The > > > > > broker > > > > > > > would proceed smoothly, but users might be unaware of the > breaking > > > > > changes, > > > > > > > and the broker would run with unexpected configurations. > > > > > > > > > > > > > > Any feedback is welcome. > > > > > > > > > > > > > > Best, > > > > > > > Chia-Ping > > > > > > > > > > > > > > On 2024/11/18 10:13:41 Divij Vaidya wrote: > > > > > > > > Hey folks > > > > > > > > > > > > > > > > With 4.0, we have an opportunity to reset the default values > and > > > add > > > > > > > > constraints in the configurations based on our learnings > since > > > 3.0. > > > > > > > > > > > > > > > > Here's a KIP which modifies defaults for some properties and > > > > > modifies the > > > > > > > > constraints for a few others. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-1030%3A+Change+constraints+and+default+values+for+various+configurations > > > > > > > > > > > > > > > > > > > > > > > > Looking forward for your feedback. > > > > > > > > > > > > > > > > (Previous discussion thread on this topic - > > > > > > > > > https://lists.apache.org/thread/3dx9mdmsqf8pko9xdmhks80k96g650zp > > > ) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
