sijie commented on issue #5513: Publish rate limit on broker to avoid OOM URL: https://github.com/apache/pulsar/issues/5513#issuecomment-549211390 @rdhabalia We have users doing load tests over Pulsar broker. The behavior we have observed is as follows: disks are overwhelmed. the entries are queuing up in the broker side. the load test is continuing and causing broker OOM. The tests were done in 2.4.2. They are upgrading the cluster to 2.5.0 and run the test again. Regarding the feature you introduced in #3985, we have asked them to try it out. However they can't really predicate the traffic and set a good publish rate in advance. Hence we are looking for a better solution at the broker level. We are thinking of reusing the mechanism you introduced in #3985. The idea is to disable autoRead when the max pending requests exceeds a threshold or the direct memory usage exceeds a threshold.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
