sijie commented on issue #5513: Publish rate limit on broker to avoid OOM URL: https://github.com/apache/pulsar/issues/5513#issuecomment-558394361 > my only point is to figure out root cause of OOM and address it. > do we think auto-read is not working as expected Yes. we think auto-read is not working as expected, because `maxPendingRequestPerConnection` is not a useful mechanism. 1) it is not a broker level mechanism. it only limits per connection. when a broker has increase number of connections, the limit is changed; 2) it is not a `bytes` mechanism. it limits on number of requests, the request size can change when the batching behavior changes in the client side. the number of bytes pending on the channel is non-deterministic. 3) it is almost impossible to determine a value based on `maxPendingRequestPerConnection` in a real production traffic. (think about the peak traffic in China's shopping festival day). The requirement from us is deadly simple - I have a broker, no matter how many clients send the request and how clients batch the request, broker should work as normal and give backpressure to the client when exceed the capacity that a broker offers. The ideal perfect solution is to disable auto-read when the resource usage (aka memory usage, cpu usage, or network usage) exceed a threshold and the process should be done automatically. However to achieve a perfect solution like that takes time and is usually complicated. The closest solution that we can provide is to have a broker-level rate limiter : 1) we can limit the traffic based on an aggregate throughput (bytes/second); 2) the mechanism is already available by #3985, we just piggyback.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
