poorbarcode opened a new pull request, #22695: URL: https://github.com/apache/pulsar/pull/22695
### Motivation **Background**: - [PIP-136: Sync Pulsar policies across multiple clouds](https://github.com/apache/pulsar/pull/16425) defines two topics below: - `metadataSyncEventTopic`: monitors local metadata store changes - `configurationMetadataSyncEventTopic `: monitors local metadata store changes - Local metadata store and Configuration metadata store share the same object in memory when their URLs are the same. **Issue 1** Since the event synchronizer is bound to the `metadata store` object in memory, the synchronizer receives all the events about the Local metadata store and Configuration metadata store when the two metadata stores are the same object in memory, the data in the two topics got mixed up. **Issue 2** The internal producer of the synchronizer relies on the `SyncEventTopic`; this topic relies on the namespace local policies; the operation of writing namespace local policies to ZK relies on the internal producer. A deadlock occurs. See the following flow: - Try to start the internal producer of the synchronizer. - Try to load the topic named `metadataSyncEventTopic` up. - Try to write data to the Local Metadata Store. - Try to send events to `metadataSyncEventTopic` before writing data to the Local Metadata Store. - The internal producer is starting now. - Stuck..... You can reproduce this issue by the test `SyncConfigStore1ZKPerClusterTest. testDynamicEnableConfigurationMetadataSyncEventTopic`. This PR fixed the issue that the synchronizer got stuck due to two metadata stores relying on it. I will write a separate PR that skips syncing data that relies on the synchronizer itself. ### Modifications - Correct the behavior: `metadataSyncEventTopic` only receives the event about the local metadata store and `configurationMetadataSyncEventTopic` only receives the event about the configuration metadata store. - If the Broker has initialized itself with one metadata store, reject the dynamic config changes. - Add an optional choice `mayEnableMetadataSynchronizer` to let the Broker initialize itself with a separate configuration metadata store. ### Documentation <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> - [ ] `doc` <!-- Your PR contains doc changes. --> - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later --> - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-complete` <!-- Docs have been already added --> ### Matching PR in forked repository PR in forked repository: x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
