vinkal-chudgar opened a new issue, #24964: URL: https://github.com/apache/pulsar/issues/24964
### Search before reporting - [x] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation ### Problem Statement `BinaryProtoLookupService#getTopicsUnderNamespace` does not deduplicate concurrent requests with identical parameters (`namespace`, `mode`, `topicsPattern`, `topicsHash`). Each call creates a new `CompletableFuture` and immediately invokes the private helper to get a connection and send a request. When multiple callers ask for the same namespace at the same time, this causes request amplification and unnecessary broker load. In the same class, sibling methods do deduplicate: - `BinaryProtoLookupService#getBroker` uses `lookupInProgress.computeIfAbsent(...)` with removal in `whenComplete(...)`. - `BinaryProtoLookupService#getPartitionedTopicMetadata` uses `partitionedMetadataInProgress.computeIfAbsent(...)` with removal in `whenComplete(...)`. This inconsistency means `BinaryProtoLookupService#getTopicsUnderNamespace` can issue duplicate network requests for identical inputs, increasing broker load, while `BinaryProtoLookupService#getBroker` and `BinaryProtoLookupService#getPartitionedTopicMetadata` already deduplicate. This is a performance improvement and consistency change, not a functional bug. It brings getTopicsUnderNamespace to the same coalescing pattern as `BinaryProtoLookupService#getBroker` and `BinaryProtoLookupService#getPartitionedTopicMetadata` ### Current Behavior When multiple threads or components request topics under the same namespace with identical parameters (namespace, mode, pattern, hash), each call results in a separate network request to the broker: https://github.com/apache/pulsar/blob/0896c0a5fe23c57179faf8752b416bee5e43b6a2/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java#L395-L411 ### Expected Behavior Concurrent requests with identical parameters should be deduplicated, sharing a single network request and result, consistent with the patterns used in `BinaryProtoLookupService#getBroker` and `BinaryProtoLookupService#getPartitionedTopicMetadata`. ### Impact Analysis **Request amplification** Within the same `PulsarClient / BinaryProtoLookupService` instance, pattern subscriptions call `client.getLookup().getTopicsUnderNamespace(...)` (e.g., `PatternMultiTopicsConsumerImpl` and during initial subscribe in `PulsarClientImpl#patternTopicSubscribeAsync`). When multiple pattern consumers perform periodic rechecks or start at the same time, they can issue concurrent calls with the same (`namespace`, `mode`, `topicsPattern`, `topicsHash`). Without deduplication, each call sends a separate request to the broker. **Broker load** Each duplicate call causes extra work on the broker for the same data. **Client resource waste** Duplicate network I/O and duplicate response handling on the client side; extra `CompletableFuture` instances are created for identical requests. **Inconsistency within the same class** `BinaryProtoLookupService#getBroker` and `BinaryProtoLookupService#getPartitionedTopicMetadata` already coalesce identical in-flight calls; BinaryProtoLookupService#getTopicsUnderNamespace currently does not. ### Evidence from Codebase The deduplication pattern is already implemented and proven effective in sibling methods: `BinaryProtoLookupService#getBroker` and `BinaryProtoLookupService#getPartitionedTopicMetadata` 1. `BinaryProtoLookupService#getBroker` https://github.com/apache/pulsar/blob/0896c0a5fe23c57179faf8752b416bee5e43b6a2/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java#L69-L70 https://github.com/apache/pulsar/blob/0896c0a5fe23c57179faf8752b416bee5e43b6a2/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java#L155-L181 2. `BinaryProtoLookupService#getPartitionedTopicMetadata` https://github.com/apache/pulsar/blob/0896c0a5fe23c57179faf8752b416bee5e43b6a2/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java#L72-L73 https://github.com/apache/pulsar/blob/0896c0a5fe23c57179faf8752b416bee5e43b6a2/pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java#L188-L206 ### Solution _No response_ ### Alternatives _No response_ ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
