Dear Pulsar Community, I'd like to propose PIP-428, which addresses critical consistency issues in the TopicPoliciesService interface and updating topic policies in Pulsar.
Problem: The current topic policies implementation has production-impacting issues: - Data corruption when multiple policies are updated rapidly - Race conditions in the get-modify-set pattern causing overwrites - Thread safety issues with mutable cached objects - Developers forced to add artificial delays between updates These issues (#24393, #21303) affect automated policy management in production deployments. PIP-428 introduces a thread-safe functional update API similar to the namespace policies fix (#9711): - Immutable TopicPolicies objects with proper cloning - Functional updates using Consumer<TopicPolicies> instead of get-modify-set - Sequential processing with update sequencer per topic - Read-your-writes consistency guarantees Benefits: - Eliminates data corruption and race conditions - Removes need for artificial delays between updates - Maintains backward compatibility (internal broker changes only) - Follows proven pattern from namespace policies fix The key API change adds a new updateTopicPoliciesAsync method to TopicPoliciesService that accepts a functional updater, ensuring atomic updates on cloned policy objects. The full proposal can be found at: https://github.com/apache/pulsar/pull/24428 I welcome your feedback and discussion on this proposal. Please share your thoughts, concerns, or suggestions. -Lari
