gordeevbr opened a new pull request #2025: Allowed non-pesistent topics to be retrieved along with persistent ones. URL: https://github.com/apache/incubator-pulsar/pull/2025 ### Motivation Please see issue #2009 for a detailed bug report. In our use case we require using java client with pattern subscription to read from a set of non-persistent topics. Unfortunately, right now this feature doesn't work. After researching the cause for this I have found out that under the hood the client is requesting a list of topics by namespace from the server and then filters them out by pattern and subscribes to them. The method in Pulsar broker NamespaceService class that is responsible for searching for required topics only uses ledgers, thus returning only persistent topics to the client. The goal of this pull request is to provide a solution for that problem. ### Modifications This pull request updates `getListOfTopics` method of NamespaceService class to also include active non-pesistent topics from local broker cache inside the `multiLayerTopicsMap` collection of BrokerService in the result. ### Result As a result, requesting a list of topics by namespace using the HTTP API or binary API (and thus via the clients) will add non-persistent topics to search result, allowing pattern subscription to be used with non-persistent topics. ### Considerations 1. Since this method pulls non-persistent topics from local broker cache, this probably means that this solution will only work for Pulsar installations with a single broker. And if there are multiple brokers, results might be inconsistent. Unfortunately I don't really know if non-persistent themselves work in multi-broker setups. I have recently asked on Slack if non-persistent topics are being replicated in any way and @merlimat's response was that they don't. Also it seems to be that some other methods that are working with non-persistent topics are using this very same collection. 2. It seems to me that unit tests have made sure that Java client can work with this setup, but this might still be a breaking change for other clients or if applications working with this API are not expecting non-persistent topics in result. 3. I have made sure that old unit tests inside the `pulsar-broker` subproject are still working and updated some old tests for this particular use case. Are there any more tests that I can add. Overall, we really need this and I would appreciate if maintainers could share their opinion. Thanks in advance.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
