poorbarcode commented on code in PR #19012: URL: https://github.com/apache/pulsar/pull/19012#discussion_r1058829758
########## site2/docs/cookbooks-message-dispatch-throttling.md: ########## @@ -0,0 +1,105 @@ +--- +id: message-dispatch-throttling +title: Message dispatch throttling +sidebar_label: "Message dispatch throttling" +--- + +## Message Dispatch throttling +Message dispatch throttling is a mechanism that limits the speed at which messages are delivered to the client. + +### When should I use message dispatch throttling? +* Messages are persisted on disk, `Storage` is the component for this part of the work. If a large number of read + requests fail to match the cache, the Storage becomes too busy and cannot work properly. Use message dispatch + throttling makes Pulsar work steadily by keeping Storage's read request load stable. +* An instance of 'Broker' serves multiple topics at the same time, and if a topic is too busy, it will occupy + almost all of the IO resources, other topics will not work well. Use message dispatch throttling can balance + the allocation of resources to agents across topics. +* There have large backlog of messages to consume, clients may receive a large amount of data in a short period of time, + thus monopolizing the client computer resources. Message dispatch throttling can also be used to balance the resource + allocation of the client computer. + +### Concepts of message dispatch throttling +- `ratePeriodInSecond` Usually the rate limiter is defined as how many times per second, how many times per minute, and + so on. In each of these definitions, there is the concept of a time period, such as one second, one minute, and the + counter is reset at the end of the time period. In Pulsar, the user can customize this time period, it is + `ratePeriodInSecond`, default is 1s. For example: if we want limit dispatch to 10,000 numbers of messages per minute, + we should set `ratePeriodInSecond` to 60 and set `dispatchThrottlingRateInMsg` to 10,000. +- `dispatchThrottlingRateInMsg` Specifies the maximum number of messages to be delivered in each rate limiting period. + The default is' -1 ', which means no limit. +- `dispatchThrottlingRateInByte` The maximum number of bytes of messages delivered per rate-limiting period. The default + is' -1 ', which means no limit. + +> `dispatchThrottlingRateInMsg` and `dispatchThrottlingRateInByte` are AND relations. + +### How it works +Message dispatch throttling works divided into these steps: +1. Calculate the number of messages or bytes to be delivered. +2. Estimate the amount of data to be read from the Storage (This estimate logic is not accurate, as described below). +3. Update the counter of message dispatch throttling. +4. Actually deliver the message to the client. + +> If the quota in the current rate limiting period is not used up, the quota will not be used in the next rate limiting +> period. However, if the quota in the current rate limiting period is exceeded, the quota in the next rate limiting +> period is reduced. For example, if the rate-limiting rule is set to `10 /s`, `11` messages are delivered to the client +> in the first rate-limiting period,`9` messages will be delivered to the client in the next rate-limiting period. + +#### Why are messages over-delivered? +- Cause-1: After batch-send enabled, we don't known how many messages per batch, so we can only estimate the number of +batch to read(see [Estimate the amount of data to be read from the Storage](#Estimate-the-amount-of-data-to-be-read-from-the-Storage)), +but the sum of all the number of messages in batches may already exceed the flow threshold. +- Cause-2: The logic of the dispatch throttling is: `1.get remaining amount` -> `2.load data` -> `3.deduct amount`, If +there are two process `a & b` are executed in parallel, it is possible to execute in this order: Review Comment: > Do we mean two threads here? yes. > why we don't track in-flight dispatching messages when checking the threshold. Dispatch prohibits concurrent execution: - if one dispatch task is running, the next dispatch will be delayed executed; - if one dispatch replayed messages task runs, the next dispatch replayed messages task will be delayed executed. but "dispatch task" will not prevent the next "dispatch replayed messages task". They don't affect each other, so there will be parallel dispatching. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
