sijie commented on issue #5513: Publish rate limit on broker to avoid OOM URL: https://github.com/apache/pulsar/issues/5513#issuecomment-558383987 @rdhabalia > I gave a try with similar setup and somehow I couldn't reproduce it. I am not sure how do you setup. Are you running the flink connector? or you simulate it? Did you run enough parallelism to stress test the cluster? OOM is a common scene during our test. > Broker restricts max pending publish request per connection. But the broker is still facing OOM when the number of connections increase, no? > also, introducing counter across all topics for throttling can cause bottleneck while publishing for all topics so, this feature might not be recommended for most of the users. The feature we are adding is controlled by a flag. People need to pre-configure what is the rate that a broker can accept. That rate is typically aligned with your NIC configuration. E.g. 80% of your NIC bandwidth. We might make it smart by automatically adjust the rate based on NIC bandwidth and memory usage. but we will always make it easy to turn on/off with a flag. What is the side effect of adding this rate limiter at broker level? > depend on maxPendingRequestPerConnection rather adding more complexity We can't really depend on `maxPendingRequestPerConnection`. Because when you have a lot of concurrent clients (connections), that setting doesn't worker. Also using number of requests doesn't give an accurate estimation. because the size of per request varies between applications. A broker level mechanism that based on bytes not requests is much robust than the other approaches. You mentioned that "I know disabling auto-read was not working all the time with ByteToMessageDecoder because of this reason and I think it's taken care in #3985." But the implementation that Jia proposed is using your implementation at #3985. If the problem is taken care by #3985, then there shouldn't be a problem in #5710; if the problem is not taken care by #3985, then it is a problem for both namespace level rate limiter and a broker level rate limiter, which I am not sure it is a problem for adding a broker level rate limiter not a problem for adding the namespace level rate limiter.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
