momo-jun commented on code in PR #386: URL: https://github.com/apache/pulsar-site/pull/386#discussion_r1106595382
########## docs/concepts-throttling.md: ########## @@ -0,0 +1,167 @@ +--- +id: concepts-throttling +title: Message dispatch throttling +sidebar_label: "Message throttling" +--- + +## Overview + +### What is message dispatch throttling? + +Large message payloads can cause memory usage spikes that lead to performance decreases. Pulsar adopts a rate-limit throttling mechanism for message dispatch, avoiding a traffic surge and improving message deliverability. You can set a threshold to limit the number of messages and the byte size of entries that can be delivered to clients, blocking subsequent deliveries when the traffic per unit of time exceeds the threshold. + +For example, when you configure the dispatch rate limit to 10 messages per second, then the number of messages that can be delivered to the client per second is up to 10. + + + +### Why use it? + +Message dispatch throttling brings the following benefits in detail: + +- **Limit broker's read request loads to BookKeeper** + + Messages are persistently stored in the BookKeeper cluster. If a large number of read requests cannot be fulfilled using the cached data, the BookKeeper cluster may become too busy to respond, and the broker's I/O or CPU resources can be fully occupied. Using the message dispatch throttling feature can regulate the data flow to limit the broker’s read request loads to BookKeeper. + +- **Balance the allocation of broker's hardware resources at topic/subscription levels** + + A broker instance serves multiple topics at one time. If a topic is overloaded with requests, it will occupy almost all of the I/O, CPU, and memory resources of the broker, causing other topics cannot be read. Using the message dispatch throttling feature can limit the allocation of broker’s hardware resources across topics. + +- **Limit the allocation of client's hardware resources at topic/subscription levels** + + When there is a large backlog of messages to consume, clients may receive a large amount of data in a short period of time, which monopolizes their computing resources. Since the client has no mechanisms to proactively limit the consumption rate, using the message dispatch throttling feature can also regulate the allocation of the client's hardware resources. + +### How it works? + +The process of message dispatch throttling can be divided into the following steps: +1. The broker approximates the number of entries to read from the bookies by calculating the remaining quota. +2. The broker reads the messages from the bookies. +3. The broker dispatches the messages to the client and updates the counter to decrease the quota. A scheduled task refreshes the quota when a throttling period ends. + +:::note + +- The quota cannot be decreased before step 3, because the broker doesn't know the actual number of messages per entry or the actual entry size until it reads the data. +- Operations like `seek` or `redeliver` may deliver messages to a client multiple times. The broker counts them as different messages and updates the counter. + +::: + +## Concepts + +### Throttling levels + +The following table outlines the three levels that you can throttle message dispatch. + +Level | Description +:-----|:------------ +Per broker | All subscriptions in a single broker share the quota. +Per topic | All subscriptions in the same topic share the quota.<br /><li>If it's a non-partitioned topic, the quota equals the maximum number of messages the topic can deliver per unit of time.</li><li>If a topic has multiple partitions, the quota refers to the maximum number of messages each partition can deliver per unit of time. In other words, the actual dispatch rate limit of a [partitioned topic](concepts-messaging.md#partitioned-topics) is N times the configured one (N is the number of partitions inside the topic). For example, the topic `t0` has two partitions. If you set the quota to `10/s`, then the rate limit of both `t0-p0` and `t0-p1` is `10/s`, while the total rate limit of `t0` is `20/s`. Note that the quota cannot be shared among partitions, but can be shared among subscriptions inside a partition.</li> +Per subscription | <li>If it's a non-partitioned topic, the rate limit refers to the maximum number of messages a subscription can deliver to clients per unit of time.</li><li>If the subscribed topic has multiple partitions, the rate limit refers to the maximum number of messages the subscription can deliver per partition per unit of time. In other words, a subscription's actual dispatch rate limit for a [partitioned topic](concepts-messaging.md#partitioned-topics) is N times the configured one (N is the number of partitions inside the topic). For example, the topic `t1` has two partitions with the subscription `s1`. If you set the rate limit to `10/s`, then the rate limit for `s1` in both `t1-p0` and `t1-p1` is `10/s`, while the total rate limit of `s1` is `20/s`.</li> + +:::note + +The dispatch rate limits configured at multiple levels take effect simultaneously (logical AND). + +::: + +### Throttling approaches + +The following table outlines multiple approaches to configure the dispatch rate limits at different levels. + +Approach | Per cluster | Per topic | Per subscription +:--------|:------------|:----------|:---------------- +Set [broker configurations](#throttling-configurations) or [dynamic broker configurations](admin-api-brokers.md#dynamic-broker-configuration) | <li>`dispatchThrottlingRateInMsg`</li><li>`dispatchThrottlingRateInByte`</li> | <li>`dispatchThrottlingRatePerTopicInMsg`</li><li>`dispatchThrottlingRatePerTopicInByte`</li><br />It applies to all topics in the cluster. | <li>`dispatchThrottlingRatePerSubscriptionInMsg`</li><li>`dispatchThrottlingRatePerSubscriptionInByte`</li><br />It applies to all subscriptions in the cluster. +Set namespace policies | N/A | Refer to [Configure dispatch throttling for topics](admin-api-namespaces.md#configure-dispatch-throttling-for-topics). | Refer to [Configure dispatch throttling for subscriptions](admin-api-namespaces.md#configure-dispatch-throttling-for-subscription). +Set topic policies | N/A | Refer to [Set topic-level dispatch rate](https://pulsar.apache.org/admin-rest-api/#operation/persistent_setDispatchRate). | Refer to [Set subscription-level dispatch rate](https://pulsar.apache.org/admin-rest-api/#operation/getSubscriptionLevelDispatchRate).<br />It applies to all subscriptions in a topic. Review Comment: This link validity has a dependency on https://github.com/apache/pulsar/pull/19193. I'll double-check it after https://github.com/apache/pulsar/pull/19193 is merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
