1fanwang opened a new pull request, #23304: URL: https://github.com/apache/kafka/pull/23304
A broker config rolled out safely as 1, then 2, then 4 comes back as the static value 1 after a KRaft broker restarts without a current local metadata snapshot. The broker still reports 4: the dynamic cache and the cluster-default DescribeConfigs response both say 4 while the running broker uses 1. Re-applying 4 is then rejected as an invalid jump from 1, leaving an operator with a value they can neither see nor restore. Startup builds the quota managers and the other runtime components straight from the static config and installs the metadata publishers much later, so persisted broker configs arrive after everything that reads them has been constructed. A current local snapshot hides the gap, which is why this surfaces only on a restart with a missing or stale one. The broker now loads the cluster-default and per-broker configs together from the metadata loader's first complete image, before those components are built, through a short-lived publisher removed once startup continues. Live updates validate their reconfigurables before applying, and a rejected update restores the previous maps. A same-instance restart also clears values removed while the broker was offline. Fixes https://issues.apache.org/jira/browse/KAFKA-20890 ## Testing A parameterized restart regression covers all three snapshot states. | snapshot at restart | trunk | with this change | |---|---|---| | none | fails, active value 1 | passes | | stale | fails, active value 1 | passes | | current | passes | passes | <details><summary>Raw output</summary> ```text $ ./gradlew :core:test --tests 'kafka.server.DynamicBrokerReconfigurationTest.testRestoreDynamicConfigFromMetadataLog' # trunk f5e01c7b0a, with only the new test applied snapshot=none FAILED org.opentest4j.AssertionFailedError: expected: <4> but was: <1> snapshot=stale FAILED org.opentest4j.AssertionFailedError: expected: <4> but was: <1> snapshot=current PASSED 3 tests completed, 2 failed BUILD FAILED in 1m 6s # with this change snapshot=none PASSED snapshot=stale PASSED snapshot=current PASSED BUILD SUCCESSFUL in 42s $ ./gradlew :core:test --tests 'kafka.server.DynamicBrokerConfigTest' \ --tests 'kafka.server.metadata.BrokerMetadataPublisherTest' DynamicBrokerConfigTest > testConfigUpdateWithReconfigurableValidationFailure() PASSED BrokerMetadataPublisherTest > testInitialDynamicConfigFailureKeepsPublisherRemovable() PASSED BUILD SUCCESSFUL in 2m 29s ``` </details> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
