gaoran10 commented on code in PR #386: URL: https://github.com/apache/pulsar-site/pull/386#discussion_r1132033901
########## docs/concepts-throttling.md: ########## @@ -0,0 +1,167 @@ +--- +id: concepts-throttling +title: Message dispatch throttling +sidebar_label: "Message throttling" +--- + +## Overview + +### What is message dispatch throttling? + +Large message payloads can cause memory usage spikes that lead to performance decreases. Pulsar adopts a rate-limit throttling mechanism for message dispatch, avoiding a traffic surge and improving message deliverability. You can set a threshold to limit the number of messages and the byte size of entries that can be delivered to clients, blocking subsequent deliveries when the traffic per unit of time exceeds the threshold. + +For example, when you configure the dispatch rate limit to 10 messages per second, then the number of messages that can be delivered to the client per second is up to 10. + + + +### Why use it? + +Message dispatch throttling brings the following benefits in detail: + +- **Limit broker's read request loads to BookKeeper** + + Messages are persistently stored in the BookKeeper cluster. If a large number of read requests cannot be fulfilled using the cached data, the BookKeeper cluster may become too busy to respond, and the broker's I/O or CPU resources can be fully occupied. Using the message dispatch throttling feature can regulate the data flow to limit the broker’s read request loads to BookKeeper. + +- **Balance the allocation of broker's hardware resources at topic/subscription levels** + + A broker instance serves multiple topics at one time. If a topic is overloaded with requests, it will occupy almost all of the I/O, CPU, and memory resources of the broker, causing other topics cannot be read. Using the message dispatch throttling feature can limit the allocation of broker’s hardware resources across topics. + +- **Limit the allocation of client's hardware resources at topic/subscription levels** + + When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client's hardware resources. + +### How it works? + +The process of message dispatch throttling can be divided into the following steps: +1. The broker approximates the number of entries to read from the bookies by calculating the remaining quota. +2. The broker reads the messages from the bookies. +3. The broker dispatches the messages to the client and updates the counter to decrease the quota. A scheduled task refreshes the quota when a throttling period ends. + +:::note + +- The quota cannot be decreased before step 3, because the broker doesn't know the actual number of messages per entry or the actual entry size until it reads the data. +- Operations like `seek` or `redeliver` may deliver messages to a client multiple times. The broker counts them as different messages and updates the counter. + +::: + +## Concepts + +### Throttling levels + +The following table outlines the three levels that you can throttle message dispatch. + +Level | Description +:-----|:------------ +Per broker | All subscriptions in a single broker share the quota. +Per topic | All subscriptions in the same topic share the quota.<br /><li>If it's a non-partitioned topic, the quota equals the maximum number of messages the topic can deliver per unit of time.</li><li>If a topic has multiple partitions, the quota refers to the maximum number of messages each partition can deliver per unit of time. In other words, the actual dispatch rate limit of a [partitioned topic](concepts-messaging.md#partitioned-topics) is N times the configured one (N is the number of partitions inside the topic). For example, the topic `t0` has two partitions. If you set the quota to `10/s`, then the rate limit of both `t0-p0` and `t0-p1` is `10/s`, while the total rate limit of `t0` is `20/s`. Note that the quota cannot be shared among partitions, but can be shared among subscriptions inside a partition.</li> +Per subscription | <li>If it's a non-partitioned topic, the rate limit refers to the maximum number of messages a subscription can deliver to clients per unit of time.</li><li>If the subscribed topic has multiple partitions, the rate limit refers to the maximum number of messages the subscription can deliver per partition per unit of time. In other words, a subscription's actual dispatch rate limit for a [partitioned topic](concepts-messaging.md#partitioned-topics) is N times the configured one (N is the number of partitions inside the topic). For example, the topic `t1` has two partitions with the subscription `s1`. If you set the rate limit to `10/s`, then the rate limit for `s1` in both `t1-p0` and `t1-p1` is `10/s`, while the total rate limit of `s1` is `20/s`.</li> + +:::note + +The dispatch rate limits configured at multiple levels take effect simultaneously (logical AND). + +::: + +### Throttling approaches + +The following table outlines multiple approaches to configure the dispatch rate limits at different levels. + +Approach | Per cluster | Per topic | Per subscription +:--------|:------------|:----------|:---------------- +Set [broker configurations](#throttling-configurations) or [dynamic broker configurations](admin-api-brokers.md#dynamic-broker-configuration) | <li>`dispatchThrottlingRateInMsg`</li><li>`dispatchThrottlingRateInByte`</li> | <li>`dispatchThrottlingRatePerTopicInMsg`</li><li>`dispatchThrottlingRatePerTopicInByte`</li><br />It applies to all topics in the cluster. | <li>`dispatchThrottlingRatePerSubscriptionInMsg`</li><li>`dispatchThrottlingRatePerSubscriptionInByte`</li><br />It applies to all subscriptions in the cluster. +Set namespace policies | N/A | Refer to [Configure dispatch throttling for topics](admin-api-namespaces.md#configure-dispatch-throttling-for-topics). | Refer to [Configure dispatch throttling for subscriptions](admin-api-namespaces.md#configure-dispatch-throttling-for-subscription). +Set topic policies | N/A | Refer to [Set topic-level dispatch rate](https://pulsar.apache.org/admin-rest-api/#operation/persistent_setDispatchRate). | Refer to [Set subscription-level dispatch rate](https://pulsar.apache.org/admin-rest-api/#operation/getSubscriptionLevelDispatchRate).<br />It applies to all subscriptions in a topic. + +:::note + +The dispatch rate limits configured through the above three approaches take effect with priorities, which is "topic policies" > "namespace policies" > "broker configurations". For example, if you have configured the dispatch rate limit for a subscription using all these three approaches, only the one configured through “topic policies” takes effect. + +::: + +### Throttling configurations + +The following table outlines the parameters that you can configure for message dispatch throttling in the `conf/broker.conf` file. + +Parameter | Description | Default value +:---------|:------------|:------------- +dispatchThrottlingRateInMsg | The total number of messages that can be delivered per cluster per throttling period.<br /><br />To set the topic-level or the subscription-level one, configure `dispatchThrottlingRatePerTopicInMsg` or `dispatchThrottlingRatePerSubscriptionInMsg`. | '-1', which means no limit. +dispatchThrottlingRateInByte | The total byte size of messages that can be delivered per cluster per throttling period.<br /><br />To set the topic-level or the subscription-level one, configure `dispatchThrottlingRatePerTopicInByte` or `dispatchThrottlingRatePerSubscriptionInByte`. | '-1', which means no limit. +ratePeriodInSecond | The period of time for dispatch throttling (in seconds). The counter is reset at the end of the period.<br />For example, if you want to configure the rate limit to `10,000 messages per minute`, you need to set `ratePeriodInSecond` to `60` and set `dispatchThrottlingRateInMsg` to `10,000`. | 1 (second) +preciseDispatcherFlowControl | Whether to apply a precise control on the dispatch throttling. By default, it's disabled, which means the broker approximates `the number of messages to read from bookies` using the minimum value between the remaining `consumer.receiverQueueSize` (defaults to 1000) and `dispatcherMaxReadBatchSize` (defaults to 100).<br /><br />When it's set to `true`, the broker approximates $$the \ number \ of \ entries \ to \ read \ from \ bookies$$ through the following equation:<br /><br />$$The \ number \ of \ entries \ to \ read \ from \ the \ bookies = {{The \ number \ of \ messages \ to \ read} \over{The \ average \ number \ of \ messages \ per \ entry}}$$<br /><br />For example, assume you've set the rate limit to `10/s`. When the average number of messages per entry is `6`, it means the broker reads 2 entries per second, and the total number of messages to read exceeds the quota. However, when it's set to `false`, the broker reads 10 entries (approximately 60 messages), which exceeds the quota much more. | false +dispatchThrottlingOnBatchMessageEnabled | Whether to count messages by entry (batch). By default, it's disabled.<br /><br />Note that setting it to `true` may lead to an inaccurate approximation of total message count but maximize Pulsar's throughput while keeping stable read requests to the bookies. For example, assume you've set the rate limit to `10/s`, if you set `dispatchThrottlingOnBatchMessageEnabled` to `true`, the broker only reads 10 entries and delivers them to the client per second, despite the number of messages per entry. | false +dispatchThrottlingOnNonBacklogConsumerEnabled | Whether the dispatch throttling on non-backlog consumers is enabled. By default, it's enabled.<br />When it is set to `false`:<br /><li>If all the consumers in one subscription have no backlog, the message dispatch throttling is turned off automatically even if `dispatchThrottlingRateInMsg` and `dispatchThrottlingRateInByte` are configured.</li><li>If at least one consumer has a backlog, the throttling is turned on automatically.</li> | true + +:::note + +- You can use `dispatchThrottlingRateInMsg` and `dispatchThrottlingRateInByte` simultaneously (logical AND). +- Ensure that only one of `preciseDispatcherFlowControl` and `dispatchThrottlingOnBatchMessageEnabled` is enabled at one time since they are mutually exclusive. Both parameters can be used to improve the over-delivery issues (see [Limitations](#limitations)). The difference between them is: + - When `preciseDispatcherFlowControl` is enabled, Pulsar considers the number of messages per entry. This parameter takes effect when the broker reads entries from the bookies. + - When `dispatchThrottlingOnBatchMessageEnabled` is enabled, Pulsar ignores the number of messages per entry. This parameter takes effect when the broker updates the counter after sending messages to the client. + +::: + +## Limitations + +Message dispatch throttling may cause messages over-delivered per unit of time due to the following reasons: + +1. **The broker may read more entries or bytes from the bookies than the throttling limit.** + + a) **The byte size of messages delivered to the client may exceed the configured threshold.** + + When you set the dispatch rate limit in bytes/throttling-period (`dispatchThrottlingRateInByte`/`ratePeriodInSecond`), the broker calculates $$the \ number \ of \ entries \ to \ read \ from \ bookies$$ in one throttling period through the following equation: + + $$ Review Comment: I found there are many symbols `$$` and `\`, it seems that they are not valid markdown symbols. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
