rdhabalia opened a new pull request, #20283: URL: https://github.com/apache/pulsar/pull/20283
[PIP: 268](https://github.com/apache/pulsar/issues/20265) ### Motivation Right now, Pulsar provides the topic's stats and stats-internal over HTTP admin API, and this stats data is used by user applications and also by Pulsar internal components such as Pulsar-functions to derive the certain states of the applications. for example, there are use cases where the application wants to check the topic's backlog, subscription's state (readPosition, list of subscriptions), numberOfEntriesSinceFirstNotAckedMessage, etc to bootstrap the application or handle the application’s resiliency and state dynamically. Applications can retrieve this stats information by using the broker’s admin HTTP APIs. However, stats retrieval over HTTP API doesn’t work well in use cases when users would like to access this API at a higher scale when a large number of application nodes would like to use it over HTTP which could overload brokers and sometimes makes broker irresponsive and impact admin API performance. It’s also become difficult when Pulsar is deployed in the cloud behind the SNI proxy and applications also want to access large-scale stats information periodically over different HTTP port instead it would be better if applications can fetch stats over on same binary protocol for scalability and accessibility reasons. Therefore, there are multiple use cases where producer/consumer applications need stats information for topics using the client library over binary protocol. Hence, this PIP introduces client API for producers and consumers to access topic stats/internal-stats information which can be used by applications as needed. ### Modifications The client library will allow both producer and consumer to retrieve stats using TopicStatsProvider that fetches stats/internal-stats for both persistent and non-persistent topics. Producer/Consumer should be able to provide generic API to access stats of single or multi topics/partitions served by them. Therefore, producer/consumer will have additional API that returns TopicStatsProvider which allows the application to fetch stats/internal-stats of all topics/partitions served by that producer/consumer. ### Does this pull request potentially affect one of the following parts: <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation <!-- DO NOT REMOVE THIS SECTION. CHECK THE PROPER BOX ONLY. --> - [ ] `doc` <!-- Your PR contains doc changes. --> - [ ] `doc-required` <!-- Your PR changes impact docs and you will update later --> - [ ] `doc-not-needed` <!-- Your PR changes do not impact docs --> - [ ] `doc-complete` <!-- Docs have been already added --> **Note:** I will create subsequent PR to pulsar-website repo to add additional documentation. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
