goutamadwant opened a new pull request, #19686: URL: https://github.com/apache/pinot/pull/19686
## Issue Fixes #19579. ## Description Pinot currently applies different precedence rules when the same configuration is present in both an instance config and the ZooKeeper cluster config: - Startup configuration uses the instance value. - Dynamically applied configuration uses the cluster value. As a result, an instance cannot reliably override a dynamically managed cluster configuration, and removing a cluster value can leave listeners with an incorrect fallback. This change makes the precedence consistent: `instance config > cluster config > default` The implementation: - Snapshots each component's instance configuration before cluster configuration is folded into it. - Passes that snapshot to `DefaultClusterConfigChangeHandler` for brokers, controllers, servers, and minions. - Builds an immutable effective configuration view in the shared handler. - Applies Pinot's relaxed key normalization when matching instance and cluster configuration keys. - Diffs the effective view instead of the raw cluster view, so changes to a cluster value hidden by an instance override do not produce misleading change notifications. - Sends the effective configuration to listeners registered after initialization. - Updates the cluster configuration provider and listener documentation to describe the effective view. - Preserves the current query-option mode when a dynamically supplied value is invalid. - Adds regression coverage for instance overrides, normalized key spellings, unshadowed dynamic changes, cluster-value removal, late listener registration, and invalid query-option values. Unpinned configuration keys remain dynamically controlled by the cluster config as before. ## Testing Validated with JDK 25: - Focused `DefaultClusterConfigChangeHandlerTest` and `QueryOptionConfigListenerTest`: 5 tests passed. - Full `pinot-common` reactor: 3,630 tests passed, 6 skipped, 0 failures or errors. - Clean test compilation across the affected 32-module reactor succeeded. - Shared Kafka ordered suite: 23 tests passed, 2 skipped. - `S3PinotFSTest`: 45 tests passed. - `BrokerQueryLimitTest`: 2 tests passed. - Spark, Flink, and Java compatibility connector/verifier reactor completed successfully. - Checkstyle, Spotless, license validation, and `git diff --check` passed. - Independent correctness and security reviews found no actionable issues. A full binary-distribution build was also attempted. It did not complete in one uninterrupted invocation because integration-test artifacts exhausted the available local disk. The affected reactor, interrupted relevant suites, and remaining connector/verifier modules were subsequently validated separately as listed above. ## Release notes When a dynamic configuration is set in both an instance config and the cluster config, the instance value now takes precedence. Cluster-level changes to that key remain hidden on the pinned instance until its instance override is removed and the process is restarted. Suggested labels: `release-notes`, `bugfix`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
