lhotari commented on PR #24833: URL: https://github.com/apache/pulsar/pull/24833#issuecomment-3408521407
To properly handle a large amount of requests, it was necessary to add a solution for keeping a cache of the estimated size of the topic listing. Without the solution heap memory would run out in the ProxyPatternConsumerBackPressureMultipleConsumersTest test after increasing concurrency. While debugging, I noticed that this happened due to the low initial permits. Since a lot of inflight requests were accepted, it caused heap memory to grow very quickly. The solution is explained [in the javadoc of TopicListSizeResultCache](https://github.com/lhotari/pulsar/blob/lh-implement-PIP-442/pulsar-broker-common/src/main/java/org/apache/pulsar/broker/topiclistlimit/TopicListSizeResultCache.java). It lets the first request to proceed with the initial permits and concurrent requests will wait until the first request updates the topic list size. This will ensure that all subsequent requests are using a sane value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
