heesung-sn commented on code in PR #19012:
URL: https://github.com/apache/pulsar/pull/19012#discussion_r1059526388
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
+ throttling makes Pulsar work steadily by keeping Storage's read request load
stable.
+* An instance of 'Broker' serves multiple topics at the same time, and if a
topic is too busy, it will occupy
Review Comment:
This makes sense for the throttling purpose: "Broker resource limiting at
broker/topics/subscription levels"
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
Review Comment:
> If a large number of read requests fail to match the cache, the Storage
becomes too busy and cannot work properly.
This sounds that Bookkeeper does not have a back-pressure feature. (can we
explain how BK implements back-pressure a little bit here?)
I think one would expect the increased I/O latency from BK if BK is
overloaded.
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
+ throttling makes Pulsar work steadily by keeping Storage's read request load
stable.
+* An instance of 'Broker' serves multiple topics at the same time, and if a
topic is too busy, it will occupy
+ almost all of the IO resources, other topics will not work well. Use message
dispatch throttling can balance
+ the allocation of resources to agents across topics.
+* There have large backlog of messages to consume, clients may receive a large
amount of data in a short period of time,
Review Comment:
This sounds like clients(consumers) cannot control the dispatch speed on the
client end.
I think one would wonder why the `Flow` command does not configure the
message dispatch speed in addition to `permits`, a.k.a the number of messages
to receive.
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
+ throttling makes Pulsar work steadily by keeping Storage's read request load
stable.
+* An instance of 'Broker' serves multiple topics at the same time, and if a
topic is too busy, it will occupy
+ almost all of the IO resources, other topics will not work well. Use message
dispatch throttling can balance
+ the allocation of resources to agents across topics.
+* There have large backlog of messages to consume, clients may receive a large
amount of data in a short period of time,
+ thus monopolizing the client computer resources. Message dispatch throttling
can also be used to balance the resource
+ allocation of the client computer.
+
+### Concepts of message dispatch throttling
+- `ratePeriodInSecond` Usually the rate limiter is defined as how many times
per second, how many times per minute, and
+ so on. In each of these definitions, there is the concept of a time period,
such as one second, one minute, and the
+ counter is reset at the end of the time period. In Pulsar, the user can
customize this time period, it is
+ `ratePeriodInSecond`, default is 1s. For example: if we want limit dispatch
to 10,000 numbers of messages per minute,
+ we should set `ratePeriodInSecond` to 60 and set
`dispatchThrottlingRateInMsg` to 10,000.
+- `dispatchThrottlingRateInMsg` Specifies the maximum number of messages to be
delivered in each rate limiting period.
+ The default is' -1 ', which means no limit.
+- `dispatchThrottlingRateInByte` The maximum number of bytes of messages
delivered per rate-limiting period. The default
+ is' -1 ', which means no limit.
+
+> `dispatchThrottlingRateInMsg` and `dispatchThrottlingRateInByte` are AND
relations.
+
+### How it works
+Message dispatch throttling works divided into these steps:
+1. Calculate the number of messages or bytes to be delivered.
Review Comment:
- One would wonder whether these steps repeat for each connected consumer.
Please clarify this part.
- Why don't we update the counter after dispatching the message?
- Can we further clarify these steps like the below?
```
For each connected client, the broker
1. Calculates the number of messages or bytes to be delivered to the
client(the consumer).
2. Estimates the number of data blocks to read from the Storage (This
estimation could read more data blocks from the Storage than the configured
limits, as described below).
3. Reads messages from the Storage or the Cache.
4. Dispatches the messages to the client.
5. Updates the counter of message dispatch throttling.
```
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
+ throttling makes Pulsar work steadily by keeping Storage's read request load
stable.
+* An instance of 'Broker' serves multiple topics at the same time, and if a
topic is too busy, it will occupy
+ almost all of the IO resources, other topics will not work well. Use message
dispatch throttling can balance
+ the allocation of resources to agents across topics.
+* There have large backlog of messages to consume, clients may receive a large
amount of data in a short period of time,
+ thus monopolizing the client computer resources. Message dispatch throttling
can also be used to balance the resource
+ allocation of the client computer.
+
+### Concepts of message dispatch throttling
+- `ratePeriodInSecond` Usually the rate limiter is defined as how many times
per second, how many times per minute, and
Review Comment:
Is the throttling logic based on the Token-Bucket algorithm? If yes, please
mention it, as Token-Bucket is popular and sets a good mental context for
readers.
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
Review Comment:
Can we clarify if the dispatch throttling works for redelivery and other
edge cases?
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
+ throttling makes Pulsar work steadily by keeping Storage's read request load
stable.
+* An instance of 'Broker' serves multiple topics at the same time, and if a
topic is too busy, it will occupy
+ almost all of the IO resources, other topics will not work well. Use message
dispatch throttling can balance
+ the allocation of resources to agents across topics.
+* There have large backlog of messages to consume, clients may receive a large
amount of data in a short period of time,
+ thus monopolizing the client computer resources. Message dispatch throttling
can also be used to balance the resource
+ allocation of the client computer.
+
+### Concepts of message dispatch throttling
+- `ratePeriodInSecond` Usually the rate limiter is defined as how many times
per second, how many times per minute, and
+ so on. In each of these definitions, there is the concept of a time period,
such as one second, one minute, and the
+ counter is reset at the end of the time period. In Pulsar, the user can
customize this time period, it is
+ `ratePeriodInSecond`, default is 1s. For example: if we want limit dispatch
to 10,000 numbers of messages per minute,
+ we should set `ratePeriodInSecond` to 60 and set
`dispatchThrottlingRateInMsg` to 10,000.
+- `dispatchThrottlingRateInMsg` Specifies the maximum number of messages to be
delivered in each rate limiting period.
+ The default is' -1 ', which means no limit.
+- `dispatchThrottlingRateInByte` The maximum number of bytes of messages
delivered per rate-limiting period. The default
+ is' -1 ', which means no limit.
+
+> `dispatchThrottlingRateInMsg` and `dispatchThrottlingRateInByte` are AND
relations.
+
+### How it works
+Message dispatch throttling works divided into these steps:
+1. Calculate the number of messages or bytes to be delivered.
+2. Estimate the amount of data to be read from the Storage (This estimate
logic is not accurate, as described below).
+3. Update the counter of message dispatch throttling.
+4. Actually deliver the message to the client.
+
+> If the quota in the current rate limiting period is not used up, the quota
will not be used in the next rate limiting
+> period. However, if the quota in the current rate limiting period is
exceeded, the quota in the next rate limiting
+> period is reduced. For example, if the rate-limiting rule is set to `10 /s`,
`11` messages are delivered to the client
+> in the first rate-limiting period,`9` messages will be delivered to the
client in the next rate-limiting period.
+
+#### Why are messages over-delivered?
+- Cause-1: Messages is stored in blocks. If batch-send is not enabled, each
message is packaged into an independent data
+block. And if batch-send is enabled, a batch of messages are packaged into one
data block, then we can not determine the
+true count of messages in one data block. So Pulsar counts batch as one
message to calculates how many batch need to
+read, after read data-block from storage, we can exactly known how many
messages in these data-blocks, so the counter
+of dispatch throttling can be corrected calculate. But it has also led to
over-delivered(enable the feature
+`preciseDispatcherFlowControl` and `dispatchThrottlingOnBatchMessageEnabled`
will improve this situation,see [Features](#features)).
+- Cause-2: The logic of the dispatch throttling is: `1.get remaining amount`
-> `2.load data` -> `3.deduct amount`, If
+there are two process "dispatch replay messages(we call it `a`)" and "dispatch
non-replay messages(we call it `b`)" in
+the same subscription are executed in parallel, it is possible to execute in
this order:
+`process-a: 1.get remaining amount` -> `process-a: 2.load data` -> `process-b:
1.get remaining amount` ->
+`process-b: 2.load data` -> `process-a: 3.deduct amount` -> `process-b:
3.deduct amount`, both `process-a` and
+`process-b` dispatch enough messages, and the total number exceeds the
threshold.
+
+#### Estimate the bytes size of data to be read from the Storage
+We cannot determine the true size of this data block until we have actually
read the it from Storage.So in order to
+satisfy the constraints `dispatchThrottlingRateInMsg` and
`dispatchThrottlingRateInByte`, requires a mechanism to
+estimate how much need to read a data block.
+
+When Pulsar receives a request to send messages, Pulsar maintains the total
count and total bytes size of data blocks,
+then Pulsar known the avg bytes size per data block: `total bytes size / total
count of blocks`. What if the broker has
+just started and has not received any new send message request? Pulsar also
records the total count and the total bytes
+size of data previously delivered to clients, so can calculate the average of
the messages previously delivered. What if
+the broker has just started and has not redelivered any messages? The current
workaround is that just delivers one data
+block to clients if it is the first delivery of a new topic.
+
+### Features
+| feature-name | description
|
example case
| default
value |
+|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------|---------------|
+| preciseDispatcherFlowControl | When delivering a message to
the client, the Broker looks at the size and number of messages per data block,
calculates an average and caches it in memory, using this average to estimate
how many data blocks to be read. What if there is no estimated value in memory
when the first reading of a new topic? The current workaround is that just
delivers one data block to clients if it is the first delivery of a new topic.
Since yesterday's batch size and today's batch size may be different, so we
will not take a strict average, instead the newer data will have more weight,
the algorithm is 'avg = (history-avg * 0.9 + new-avg * 0.1)' is used.
| We set rate limiter to 10/s, if the average number of messages per
batch currently recorded is 6, then we should read 2 data blocks, and if there
have 7 and 8 messages each blocks, we will eventually exceed the limit. But if
we disabled feature `preciseDispatcherFlowControl`, 10 data
blocks will be read, which will make the excess problem worse. | false
|
Review Comment:
It would be great to make the descriptions short and succinct and move the
verbose explanation to the example section.
For example,
preciseDispatcherFlowControl :
description :
If enabled, the broker estimates the number of messages to read from
storage( `messagesToRead`) based on the average number of messages of BK ledger
entries(message batches) of the topic.
If disabled, the broker computes the `messagesToRead` based on
`Math.min(permits, dispatcherMaxReadBatchSize);` The `messagesToRead` is
compared against `dispatchThrottlingRateInMsg`.
Also, regarding this example,
> We set rate limiter to 10/s, if the average number of messages per batch
currently recorded is 6
Shouldn't it be `per batch currently recorded is 5`
##########
site2/docs/cookbooks-message-dispatch-throttling.md:
##########
@@ -0,0 +1,104 @@
+---
+id: message-dispatch-throttling
+title: Message dispatch throttling
+sidebar_label: "Message dispatch throttling"
+---
+
+## Message Dispatch throttling
+Message dispatch throttling is a mechanism that limits the speed at which
messages are delivered to the client.
+
+### When should I use message dispatch throttling?
+* Messages are persisted on disk, `Storage` is the component for this part of
the work. If a large number of read
+ requests fail to match the cache, the Storage becomes too busy and cannot
work properly. Use message dispatch
+ throttling makes Pulsar work steadily by keeping Storage's read request load
stable.
+* An instance of 'Broker' serves multiple topics at the same time, and if a
topic is too busy, it will occupy
+ almost all of the IO resources, other topics will not work well. Use message
dispatch throttling can balance
+ the allocation of resources to agents across topics.
+* There have large backlog of messages to consume, clients may receive a large
amount of data in a short period of time,
+ thus monopolizing the client computer resources. Message dispatch throttling
can also be used to balance the resource
+ allocation of the client computer.
+
+### Concepts of message dispatch throttling
+- `ratePeriodInSecond` Usually the rate limiter is defined as how many times
per second, how many times per minute, and
+ so on. In each of these definitions, there is the concept of a time period,
such as one second, one minute, and the
+ counter is reset at the end of the time period. In Pulsar, the user can
customize this time period, it is
+ `ratePeriodInSecond`, default is 1s. For example: if we want limit dispatch
to 10,000 numbers of messages per minute,
+ we should set `ratePeriodInSecond` to 60 and set
`dispatchThrottlingRateInMsg` to 10,000.
+- `dispatchThrottlingRateInMsg` Specifies the maximum number of messages to be
delivered in each rate limiting period.
+ The default is' -1 ', which means no limit.
+- `dispatchThrottlingRateInByte` The maximum number of bytes of messages
delivered per rate-limiting period. The default
+ is' -1 ', which means no limit.
+
+> `dispatchThrottlingRateInMsg` and `dispatchThrottlingRateInByte` are AND
relations.
+
+### How it works
+Message dispatch throttling works divided into these steps:
+1. Calculate the number of messages or bytes to be delivered.
+2. Estimate the amount of data to be read from the Storage (This estimate
logic is not accurate, as described below).
+3. Update the counter of message dispatch throttling.
+4. Actually deliver the message to the client.
+
+> If the quota in the current rate limiting period is not used up, the quota
will not be used in the next rate limiting
+> period. However, if the quota in the current rate limiting period is
exceeded, the quota in the next rate limiting
+> period is reduced. For example, if the rate-limiting rule is set to `10 /s`,
`11` messages are delivered to the client
+> in the first rate-limiting period,`9` messages will be delivered to the
client in the next rate-limiting period.
+
+#### Why are messages over-delivered?
+- Cause-1: Messages is stored in blocks. If batch-send is not enabled, each
message is packaged into an independent data
+block. And if batch-send is enabled, a batch of messages are packaged into one
data block, then we can not determine the
+true count of messages in one data block. So Pulsar counts batch as one
message to calculates how many batch need to
+read, after read data-block from storage, we can exactly known how many
messages in these data-blocks, so the counter
+of dispatch throttling can be corrected calculate. But it has also led to
over-delivered(enable the feature
+`preciseDispatcherFlowControl` and `dispatchThrottlingOnBatchMessageEnabled`
will improve this situation,see [Features](#features)).
+- Cause-2: The logic of the dispatch throttling is: `1.get remaining amount`
-> `2.load data` -> `3.deduct amount`, If
+there are two process "dispatch replay messages(we call it `a`)" and "dispatch
non-replay messages(we call it `b`)" in
+the same subscription are executed in parallel, it is possible to execute in
this order:
+`process-a: 1.get remaining amount` -> `process-a: 2.load data` -> `process-b:
1.get remaining amount` ->
+`process-b: 2.load data` -> `process-a: 3.deduct amount` -> `process-b:
3.deduct amount`, both `process-a` and
+`process-b` dispatch enough messages, and the total number exceeds the
threshold.
+
+#### Estimate the bytes size of data to be read from the Storage
+We cannot determine the true size of this data block until we have actually
read the it from Storage.So in order to
+satisfy the constraints `dispatchThrottlingRateInMsg` and
`dispatchThrottlingRateInByte`, requires a mechanism to
+estimate how much need to read a data block.
+
+When Pulsar receives a request to send messages, Pulsar maintains the total
count and total bytes size of data blocks,
+then Pulsar known the avg bytes size per data block: `total bytes size / total
count of blocks`. What if the broker has
+just started and has not received any new send message request? Pulsar also
records the total count and the total bytes
+size of data previously delivered to clients, so can calculate the average of
the messages previously delivered. What if
+the broker has just started and has not redelivered any messages? The current
workaround is that just delivers one data
+block to clients if it is the first delivery of a new topic.
+
+### Features
+| feature-name | description
|
example case
| default
value |
+|-----------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
----------------------------------------------------------------|---------------|
+| preciseDispatcherFlowControl | When delivering a message to
the client, the Broker looks at the size and number of messages per data block,
calculates an average and caches it in memory, using this average to estimate
how many data blocks to be read. What if there is no estimated value in memory
when the first reading of a new topic? The current workaround is that just
delivers one data block to clients if it is the first delivery of a new topic.
Since yesterday's batch size and today's batch size may be different, so we
will not take a strict average, instead the newer data will have more weight,
the algorithm is 'avg = (history-avg * 0.9 + new-avg * 0.1)' is used.
| We set rate limiter to 10/s, if the average number of messages per
batch currently recorded is 6, then we should read 2 data blocks, and if there
have 7 and 8 messages each blocks, we will eventually exceed the limit. But if
we disabled feature `preciseDispatcherFlowControl`, 10 data
blocks will be read, which will make the excess problem worse. | false
|
+| dispatchThrottlingOnBatchMessageEnabled | Support rate-limiting
dispatching on the batch messages rather than individual messages within batch
messages. Since one batch is one data block, this is sufficient to constrain
the concurrency of read requests on storage. This makes the count of the number
of messages inaccurate,but also maximizes pulsar's throughput while keeping
storage read requests stable.
| We set rate limiter to 10/s, after we enabled
`dispatchThrottlingOnBatchMessageEnabled`, Pulsar do not care about how many
messages per batch, 10 data blocks will be read and delivered, and the counter
of rate limiter mark delivered 10 messages.
| false
|
+| dispatchThrottlingOnNonBacklogConsumerEnabled | After we turn off
`dispatchThrottlingOnNonBacklogConsumerEnabled`(default is enabled), if all
consumers in one subscription have no backlog(it is clear that almost all read
requests can hit the cache, so Pulsar will not send read request to the Storage
any more), then message dispatch throttling is turned off automatically(it
means that even if we set `dispatchThrottlingRateInMsg` and
`dispatchThrottlingRateInByte`, throttling won't work because there is no
backlog)，and if any consumer has backlog,it will be turned on automatically. If
we need only to prevent the excessive read requests results in Storage can not
work well, this feature is very useful. |
| true
|
+
+> `preciseDispatcherFlowControl` and `dispatchThrottlingOnBatchMessageEnabled`
are mutually exclusive. We should ensure
+> that a maximum of one is enabled.
+
+### Only three granularity supports
Review Comment:
Can we clarify the priority order of these throttling limit levels?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
