codelipenghui opened a new issue #7720:
URL: https://github.com/apache/pulsar/issues/7720


   **Is your feature request related to a problem? Please describe.**
   Currently, the Pulsar support topic level and subscription level dispatch 
rate limiter. When we have more topics and consumers(with high receive queue 
size) attach many topics, this will lead to a sudden increase in entry read 
traffic.
   It may bring unstable factors to the cluster(may cause broker outgoing 
network bandwidth with the heavy workload, may cause too many read request to 
bookies, etc.)
   
   The topic level rate limiter and the subscription level rate limiter unable 
to limit the resources of the entire broker because of the topics and the 
subscriptions may keep increasing.
   
   
![image](https://user-images.githubusercontent.com/12592133/89123315-00017300-d501-11ea-8579-fc1c4111d147.png)
   
   The above screenshot is happening when I test read entries with 200MB/s, but 
it reached 600MB/s at the moment of starting the consumer, and the message 
writing also been affected.
   
   **Describe the solution you'd like**
   Introduce broker level dispatch rate limiter. This will make it easier for 
us to do broker resource planning.
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to