This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 9d7385b Docs sync done from apache/pulsar(#9a88508)
9d7385b is described below
commit 9d7385b2971c7438b51994ea7fb473c81307e88b
Author: Pulsar Site Updater <[email protected]>
AuthorDate: Tue Mar 15 18:07:03 2022 +0000
Docs sync done from apache/pulsar(#9a88508)
---
site2/docs/assets/backlog-quota.svg | 1 +
site2/docs/assets/retention-storage-size.svg | 1 +
site2/docs/assets/retention.svg | 1 +
site2/docs/assets/ttl.svg | 1 +
site2/docs/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
site2/docs/reference-configuration.md | 1 -
site2/docs/reference-metrics.md | 5 ----
.../docs/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
site2/website-next/docs/reference-configuration.md | 1 -
site2/website-next/docs/reference-metrics.md | 5 ----
site2/website-next/static/assets/backlog-quota.svg | 1 +
.../static/assets/retention-storage-size.svg | 1 +
site2/website-next/static/assets/retention.svg | 1 +
site2/website-next/static/assets/ttl.svg | 1 +
.../version-2.2.0/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.2.1/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.3.1/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.3.2/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.4.0/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.4.1/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.4.2/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.5.1/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.5.2/cookbooks-retention-expiry.md | 29 ++++++++++++++++++++--
.../version-2.8.0/reference-configuration.md | 1 -
.../version-2.8.0/reference-metrics.md | 5 ----
.../version-2.8.1/reference-configuration.md | 1 -
.../version-2.8.1/reference-metrics.md | 5 ----
.../version-2.8.2/reference-configuration.md | 1 -
.../version-2.8.2/reference-metrics.md | 5 ----
.../version-2.9.0/reference-configuration.md | 1 -
.../version-2.9.0/reference-metrics.md | 5 ----
.../version-2.9.1/reference-configuration.md | 1 -
.../version-2.9.1/reference-metrics.md | 5 ----
.../version-2.8.0/reference-configuration.md | 1 -
.../version-2.8.0/reference-metrics.md | 5 ----
.../version-2.8.1/reference-configuration.md | 1 -
.../version-2.8.1/reference-metrics.md | 5 ----
.../version-2.8.2/reference-configuration.md | 1 -
.../version-2.8.2/reference-metrics.md | 5 ----
.../version-2.9.0/reference-configuration.md | 1 -
.../version-2.9.0/reference-metrics.md | 5 ----
.../version-2.9.1/reference-configuration.md | 1 -
.../version-2.9.1/reference-metrics.md | 5 ----
43 files changed, 305 insertions(+), 94 deletions(-)
diff --git a/site2/docs/assets/backlog-quota.svg
b/site2/docs/assets/backlog-quota.svg
new file mode 100644
index 0000000..c508d03
--- /dev/null
+++ b/site2/docs/assets/backlog-quota.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="1465.32"
height="906.61"><g transform="translate(-52.89186139290442 997.2734706364487)"
lucid:page-tab-id="0_0"><path d="M0-1258.83h1870.87V64H0z" fill="#fff"/><path
d="M312-550.86c17.67 0 32 13.43 32 30s-14.33 30-32 30h-96c-17.67
0-32-13.43-32-30s14.33-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,189,-545.8608166743202) tr [...]
\ No newline at end of file
diff --git a/site2/docs/assets/retention-storage-size.svg
b/site2/docs/assets/retention-storage-size.svg
new file mode 100644
index 0000000..217f2ce
--- /dev/null
+++ b/site2/docs/assets/retention-storage-size.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="1158.73"
height="613.67"><g transform="translate(-602.6666666666667
-106.99999999998668)" lucid:page-tab-id="0_0"><path d="M0 0h1870.87v1322.83H0z"
fill="#fff"/><path d="M750.67 260.47c17.67 0 32 13.43 32 30s-14.33 30-32
30h-96c-17.68 0-32-13.43-32-30s14.32-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,627.6666666666667,265. [...]
\ No newline at end of file
diff --git a/site2/docs/assets/retention.svg b/site2/docs/assets/retention.svg
new file mode 100644
index 0000000..9f8b936
--- /dev/null
+++ b/site2/docs/assets/retention.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="920"
height="581.33"><g transform="translate(-602.6666666666667
-139.33333333333314)" lucid:page-tab-id="0_0"><path d="M0 0h1870.87v1322.83H0z"
fill="#fff"/><path d="M750.67 260.47c17.67 0 32 13.43 32 30s-14.33 30-32
30h-96c-17.68 0-32-13.43-32-30s14.32-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,627.6666666666667,265.4725 [...]
\ No newline at end of file
diff --git a/site2/docs/assets/ttl.svg b/site2/docs/assets/ttl.svg
new file mode 100644
index 0000000..fec44a2
--- /dev/null
+++ b/site2/docs/assets/ttl.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="1100"
height="580.67"><g transform="translate(-313.33333333333337 -180)"
lucid:page-tab-id="0_0"><path d="M0 0h1870.87v1322.83H0z" fill="#fff"/><path
d="M461.33 297.14c17.68 0 32 13.43 32 30s-14.32 30-32 30h-96c-17.67
0-32-13.43-32-30s14.33-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,338.33333333333337,302.13918332567994) [...]
\ No newline at end of file
diff --git a/site2/docs/cookbooks-retention-expiry.md
b/site2/docs/cookbooks-retention-expiry.md
index 69b514b..35d60db 100644
--- a/site2/docs/cookbooks-retention-expiry.md
+++ b/site2/docs/cookbooks-retention-expiry.md
@@ -29,6 +29,9 @@ Pulsar's [admin interface](admin-api-overview.md) enables you
to manage both ret
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -153,7 +156,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -282,6 +291,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
<!--DOCUSAURUS_CODE_TABS-->
@@ -358,7 +372,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -366,3 +386,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git a/site2/docs/reference-configuration.md
b/site2/docs/reference-configuration.md
index f85d228..f8a6aaa 100644
--- a/site2/docs/reference-configuration.md
+++ b/site2/docs/reference-configuration.md
@@ -638,7 +638,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | Runtime.getRuntime().availableProcessors() |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | Runtime.getRuntime().availableProcessors() |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/docs/reference-metrics.md b/site2/docs/reference-metrics.md
index bee590d..b11f4e7 100644
--- a/site2/docs/reference-metrics.md
+++ b/site2/docs/reference-metrics.md
@@ -420,13 +420,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br>The
number of metrics determined by the scheduler executor thread number configured
by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br>The number of
metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br>The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git a/site2/website-next/docs/cookbooks-retention-expiry.md
b/site2/website-next/docs/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
--- a/site2/website-next/docs/cookbooks-retention-expiry.md
+++ b/site2/website-next/docs/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git a/site2/website-next/docs/reference-configuration.md
b/site2/website-next/docs/reference-configuration.md
index 3dfaa35..2b34949 100644
--- a/site2/website-next/docs/reference-configuration.md
+++ b/site2/website-next/docs/reference-configuration.md
@@ -634,7 +634,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | Runtime.getRuntime().availableProcessors() |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | Runtime.getRuntime().availableProcessors() |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/website-next/docs/reference-metrics.md
b/site2/website-next/docs/reference-metrics.md
index f04c390..683b40d 100644
--- a/site2/website-next/docs/reference-metrics.md
+++ b/site2/website-next/docs/reference-metrics.md
@@ -416,13 +416,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br
/>The number of metrics determined by the scheduler executor thread number
configured by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br />The number
of metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br />The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git a/site2/website-next/static/assets/backlog-quota.svg
b/site2/website-next/static/assets/backlog-quota.svg
new file mode 100644
index 0000000..c508d03
--- /dev/null
+++ b/site2/website-next/static/assets/backlog-quota.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="1465.32"
height="906.61"><g transform="translate(-52.89186139290442 997.2734706364487)"
lucid:page-tab-id="0_0"><path d="M0-1258.83h1870.87V64H0z" fill="#fff"/><path
d="M312-550.86c17.67 0 32 13.43 32 30s-14.33 30-32 30h-96c-17.67
0-32-13.43-32-30s14.33-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,189,-545.8608166743202) tr [...]
\ No newline at end of file
diff --git a/site2/website-next/static/assets/retention-storage-size.svg
b/site2/website-next/static/assets/retention-storage-size.svg
new file mode 100644
index 0000000..217f2ce
--- /dev/null
+++ b/site2/website-next/static/assets/retention-storage-size.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="1158.73"
height="613.67"><g transform="translate(-602.6666666666667
-106.99999999998668)" lucid:page-tab-id="0_0"><path d="M0 0h1870.87v1322.83H0z"
fill="#fff"/><path d="M750.67 260.47c17.67 0 32 13.43 32 30s-14.33 30-32
30h-96c-17.68 0-32-13.43-32-30s14.32-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,627.6666666666667,265. [...]
\ No newline at end of file
diff --git a/site2/website-next/static/assets/retention.svg
b/site2/website-next/static/assets/retention.svg
new file mode 100644
index 0000000..9f8b936
--- /dev/null
+++ b/site2/website-next/static/assets/retention.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="920"
height="581.33"><g transform="translate(-602.6666666666667
-139.33333333333314)" lucid:page-tab-id="0_0"><path d="M0 0h1870.87v1322.83H0z"
fill="#fff"/><path d="M750.67 260.47c17.67 0 32 13.43 32 30s-14.33 30-32
30h-96c-17.68 0-32-13.43-32-30s14.32-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,627.6666666666667,265.4725 [...]
\ No newline at end of file
diff --git a/site2/website-next/static/assets/ttl.svg
b/site2/website-next/static/assets/ttl.svg
new file mode 100644
index 0000000..fec44a2
--- /dev/null
+++ b/site2/website-next/static/assets/ttl.svg
@@ -0,0 +1 @@
+<svg xmlns="http://www.w3.org/2000/svg";
xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:lucid="lucid" width="1100"
height="580.67"><g transform="translate(-313.33333333333337 -180)"
lucid:page-tab-id="0_0"><path d="M0 0h1870.87v1322.83H0z" fill="#fff"/><path
d="M461.33 297.14c17.68 0 32 13.43 32 30s-14.32 30-32 30h-96c-17.67
0-32-13.43-32-30s14.33-30 32-30z" stroke="#666" stroke-width="4"
fill="#fff"/><use xlink:href="#a"
transform="matrix(1,0,0,1,338.33333333333337,302.13918332567994) [...]
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.2.0/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.2.0/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.2.0/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.2.0/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.2.1/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.2.1/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.2.1/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.2.1/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.3.1/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.3.1/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.3.1/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.3.1/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.3.2/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.3.2/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.3.2/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.3.2/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.4.0/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.4.0/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.4.0/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.4.0/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.4.1/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.4.1/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.4.1/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.4.1/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.4.2/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.4.2/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.4.2/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.4.2/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.5.1/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.5.1/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.5.1/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.5.1/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.5.2/cookbooks-retention-expiry.md
b/site2/website-next/versioned_docs/version-2.5.2/cookbooks-retention-expiry.md
index 3a2f91f..1229144 100644
---
a/site2/website-next/versioned_docs/version-2.5.2/cookbooks-retention-expiry.md
+++
b/site2/website-next/versioned_docs/version-2.5.2/cookbooks-retention-expiry.md
@@ -35,6 +35,9 @@ Pulsar's [admin interface](admin-api-overview) enables you to
manage both retent
By default, when a Pulsar message arrives at a broker, the message is stored
until it has been acknowledged on all subscriptions, at which point it is
marked for deletion. You can override this behavior and retain messages that
have already been acknowledged on all subscriptions by setting a *retention
policy* for all topics in a given namespace. Retention is based on both a *size
limit* and a *time limit*.
+The diagram below illustrates the concept of message retention.
+
+
Retention policies are useful when you use the Reader interface. The Reader
interface does not use acknowledgements, and messages do not exist within
backlogs. It is required to configure retention for Reader-only use cases.
When you set a retention policy on topics in a namespace, you must set
**both** a *size limit* (via `defaultRetentionSizeInMB`) and a *time limit*
(via `defaultRetentionTimeInMinutes`) . You can refer to the following table to
set retention policies in `pulsar-admin` and Java.
@@ -198,7 +201,13 @@ admin.namespaces().getRetention(namespace);
*Backlogs* are sets of unacknowledged messages for a topic that have been
stored by bookies. Pulsar stores all unacknowledged messages in backlogs until
they are processed and acknowledged.
-You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Setting a backlog quota involves setting:
+You can control the allowable size and/or time of backlogs, at the namespace
level, using *backlog quotas*. Pulsar uses a quota to enforce a hard limit on
the logical size of the backlogs in a topic. Backlog quota triggers an alert
policy (for example, producer exception) once the quota limit is reached.
+
+The diagram below illustrates the concept of backlog quota.
+
+
+
+Setting a backlog quota involves setting:
* an allowable *size and/or time threshold* for each topic in the namespace
* a *retention policy* that determines which action the
[broker](reference-terminology.md#broker) takes if the threshold is exceeded.
@@ -370,6 +379,11 @@ By default, you will be prompted to ensure that you really
want to clear the bac
By default, Pulsar stores all unacknowledged messages forever. This can lead
to heavy disk space usage in cases where a lot of messages are going
unacknowledged. If disk space is a concern, you can set a time to live (TTL)
that determines how long unacknowledged messages will be retained.
+The TTL parameter is like a stopwatch attached to each message that defines
the amount of time a message is allowed to stay in the unacknowledged state.
When the TTL expires, Pulsar automatically moves the message to the
acknowledged state (and thus makes it ready for deletion).
+
+The diagram below illustrates the concept of TTL.
+
+
### Set the TTL for a namespace
````mdx-code-block
@@ -485,7 +499,13 @@ admin.namespaces().removeNamespaceMessageTTL(namespace)
## Delete messages from namespaces
-If you do not have any retention period and that you never have much of a
backlog, the upper limit for retaining messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
+When it comes to the physical storage size, message expiry and retention are
just like two sides of the same coin.
+* The backlog quota and TTL parameters prevent disk size from growing
indefinitely, as Pulsar’s default behaviour is to persist unacknowledged
messages.
+* The retention policy allocates storage space to accommodate the messages
that are supposed to be deleted by Pulsar by default.
+
+As a conclusion, the size of your physical storage should accommodate the sum
of the backlog quota and the retention size.
+
+The message deletion rate (releasing rate of disk space) can be determined by
multiple factors.
- **Segment rollover period**: basically, the segment rollover period is how
often a new segment is created. Once a new segment is created, the old segment
will be deleted. By default, this happens either when you have written 50,000
entries (messages) or have waited 240 minutes. You can tune this in your broker.
@@ -493,3 +513,8 @@ If you do not have any retention period and that you never
have much of a backlo
The entry log rollover period is configurable, but is purely based on the
entry log size. For details, see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-settings).
Once the entry log is rolled over, the entry log can be garbage collected.
- **Garbage collection interval**: because entry logs have interleaved
ledgers, to free up space, the entry logs need to be rewritten. The garbage
collection interval is how often BookKeeper performs garbage collection. which
is related to minor compaction and major compaction of entry logs. For details,
see
[here](https://bookkeeper.apache.org/docs/4.11.1/reference/config/#entry-log-compaction-settings).
+
+The diagram below illustrates one of the cases that the consumed storage size
is larger than the given limits for backlog and retention. Messages over the
retention limit are kept because other messages in the same segment are still
within retention period.
+
+
+If you do not have any retention period and that you never have much of a
backlog, the upper limit for retained messages, which are acknowledged, equals
to the Pulsar segment rollover period + entry log rollover period + (garbage
collection interval * garbage collection ratios).
\ No newline at end of file
diff --git
a/site2/website-next/versioned_docs/version-2.8.0/reference-configuration.md
b/site2/website-next/versioned_docs/version-2.8.0/reference-configuration.md
index 4f54320..417ec48 100644
--- a/site2/website-next/versioned_docs/version-2.8.0/reference-configuration.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/reference-configuration.md
@@ -606,7 +606,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git
a/site2/website-next/versioned_docs/version-2.8.0/reference-metrics.md
b/site2/website-next/versioned_docs/version-2.8.0/reference-metrics.md
index 5526385..c0a8d8e 100644
--- a/site2/website-next/versioned_docs/version-2.8.0/reference-metrics.md
+++ b/site2/website-next/versioned_docs/version-2.8.0/reference-metrics.md
@@ -352,13 +352,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br
/>The number of metrics determined by the scheduler executor thread number
configured by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br />The number
of metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br />The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website-next/versioned_docs/version-2.8.1/reference-configuration.md
b/site2/website-next/versioned_docs/version-2.8.1/reference-configuration.md
index 75cd412..e746f97 100644
--- a/site2/website-next/versioned_docs/version-2.8.1/reference-configuration.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/reference-configuration.md
@@ -607,7 +607,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git
a/site2/website-next/versioned_docs/version-2.8.1/reference-metrics.md
b/site2/website-next/versioned_docs/version-2.8.1/reference-metrics.md
index 3d20296..1a4a9aa 100644
--- a/site2/website-next/versioned_docs/version-2.8.1/reference-metrics.md
+++ b/site2/website-next/versioned_docs/version-2.8.1/reference-metrics.md
@@ -355,13 +355,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br
/>The number of metrics determined by the scheduler executor thread number
configured by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br />The number
of metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br />The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website-next/versioned_docs/version-2.8.2/reference-configuration.md
b/site2/website-next/versioned_docs/version-2.8.2/reference-configuration.md
index 69cc2f7..06d24c4 100644
--- a/site2/website-next/versioned_docs/version-2.8.2/reference-configuration.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/reference-configuration.md
@@ -609,7 +609,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git
a/site2/website-next/versioned_docs/version-2.8.2/reference-metrics.md
b/site2/website-next/versioned_docs/version-2.8.2/reference-metrics.md
index bf6b463..97bf22a 100644
--- a/site2/website-next/versioned_docs/version-2.8.2/reference-metrics.md
+++ b/site2/website-next/versioned_docs/version-2.8.2/reference-metrics.md
@@ -364,13 +364,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br
/>The number of metrics determined by the scheduler executor thread number
configured by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br />The number
of metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br />The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website-next/versioned_docs/version-2.9.0/reference-configuration.md
b/site2/website-next/versioned_docs/version-2.9.0/reference-configuration.md
index fea0b95..4a6d6d3 100644
--- a/site2/website-next/versioned_docs/version-2.9.0/reference-configuration.md
+++ b/site2/website-next/versioned_docs/version-2.9.0/reference-configuration.md
@@ -592,7 +592,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git
a/site2/website-next/versioned_docs/version-2.9.0/reference-metrics.md
b/site2/website-next/versioned_docs/version-2.9.0/reference-metrics.md
index c1bcfcf..4658cce 100644
--- a/site2/website-next/versioned_docs/version-2.9.0/reference-metrics.md
+++ b/site2/website-next/versioned_docs/version-2.9.0/reference-metrics.md
@@ -385,13 +385,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br
/>The number of metrics determined by the scheduler executor thread number
configured by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br />The number
of metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br />The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website-next/versioned_docs/version-2.9.1/reference-configuration.md
b/site2/website-next/versioned_docs/version-2.9.1/reference-configuration.md
index 539f348..840fcd0 100644
--- a/site2/website-next/versioned_docs/version-2.9.1/reference-configuration.md
+++ b/site2/website-next/versioned_docs/version-2.9.1/reference-configuration.md
@@ -592,7 +592,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git
a/site2/website-next/versioned_docs/version-2.9.1/reference-metrics.md
b/site2/website-next/versioned_docs/version-2.9.1/reference-metrics.md
index c1bcfcf..4658cce 100644
--- a/site2/website-next/versioned_docs/version-2.9.1/reference-metrics.md
+++ b/site2/website-next/versioned_docs/version-2.9.1/reference-metrics.md
@@ -385,13 +385,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br
/>The number of metrics determined by the scheduler executor thread number
configured by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br />The number
of metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br />The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br />The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br /> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website/versioned_docs/version-2.8.0/reference-configuration.md
b/site2/website/versioned_docs/version-2.8.0/reference-configuration.md
index f03a397..70accc2 100644
--- a/site2/website/versioned_docs/version-2.8.0/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.8.0/reference-configuration.md
@@ -610,7 +610,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/website/versioned_docs/version-2.8.0/reference-metrics.md
b/site2/website/versioned_docs/version-2.8.0/reference-metrics.md
index c1f2bfd..8467ab0 100644
--- a/site2/website/versioned_docs/version-2.8.0/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.8.0/reference-metrics.md
@@ -356,13 +356,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br>The
number of metrics determined by the scheduler executor thread number configured
by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br>The number of
metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br>The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website/versioned_docs/version-2.8.1/reference-configuration.md
b/site2/website/versioned_docs/version-2.8.1/reference-configuration.md
index 49cd19d..583b3a4 100644
--- a/site2/website/versioned_docs/version-2.8.1/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.8.1/reference-configuration.md
@@ -611,7 +611,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/website/versioned_docs/version-2.8.1/reference-metrics.md
b/site2/website/versioned_docs/version-2.8.1/reference-metrics.md
index 14e91ef..f1b2a24 100644
--- a/site2/website/versioned_docs/version-2.8.1/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.8.1/reference-metrics.md
@@ -359,13 +359,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br>The
number of metrics determined by the scheduler executor thread number configured
by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br>The number of
metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br>The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website/versioned_docs/version-2.8.2/reference-configuration.md
b/site2/website/versioned_docs/version-2.8.2/reference-configuration.md
index bf2556b..e2e936f 100644
--- a/site2/website/versioned_docs/version-2.8.2/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.8.2/reference-configuration.md
@@ -613,7 +613,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/website/versioned_docs/version-2.8.2/reference-metrics.md
b/site2/website/versioned_docs/version-2.8.2/reference-metrics.md
index 20f43d8..e984f60 100644
--- a/site2/website/versioned_docs/version-2.8.2/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.8.2/reference-metrics.md
@@ -368,13 +368,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br>The
number of metrics determined by the scheduler executor thread number configured
by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br>The number of
metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br>The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website/versioned_docs/version-2.9.0/reference-configuration.md
b/site2/website/versioned_docs/version-2.9.0/reference-configuration.md
index e85fb61..286743b 100644
--- a/site2/website/versioned_docs/version-2.9.0/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.9.0/reference-configuration.md
@@ -596,7 +596,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/website/versioned_docs/version-2.9.0/reference-metrics.md
b/site2/website/versioned_docs/version-2.9.0/reference-metrics.md
index 4af6740..71c5c87 100644
--- a/site2/website/versioned_docs/version-2.9.0/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.9.0/reference-metrics.md
@@ -389,13 +389,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br>The
number of metrics determined by the scheduler executor thread number configured
by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br>The number of
metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br>The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
diff --git
a/site2/website/versioned_docs/version-2.9.1/reference-configuration.md
b/site2/website/versioned_docs/version-2.9.1/reference-configuration.md
index f94126c..71a9379 100644
--- a/site2/website/versioned_docs/version-2.9.1/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.9.1/reference-configuration.md
@@ -596,7 +596,6 @@ You can set the log level and configuration in the
[log4j2.yaml](https://github
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
| managedLedgerDigestType | Default type of checksum to use when writing to
BookKeeper. | CRC32C |
-| managedLedgerNumWorkerThreads | Number of threads to be used for managed
ledger tasks dispatching. | 8 |
| managedLedgerNumSchedulerThreads | Number of threads to be used for managed
ledger scheduled tasks. | 8 |
|managedLedgerCacheSizeMB| |N/A|
|managedLedgerCacheCopyEntries| Whether to copy the entry payloads when
inserting in cache.| false|
diff --git a/site2/website/versioned_docs/version-2.9.1/reference-metrics.md
b/site2/website/versioned_docs/version-2.9.1/reference-metrics.md
index e7dd70d..b566584 100644
--- a/site2/website/versioned_docs/version-2.9.1/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.9.1/reference-metrics.md
@@ -389,13 +389,8 @@ All the managed ledger bookie client metrics are labelled
with the following lab
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_completed_tasks_* |
Gauge | The number of tasks the scheduler executor execute completed. <br>The
number of metrics determined by the scheduler executor thread number configured
by `managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_queue_* | Gauge | The
number of tasks queued in the scheduler executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_total_tasks_* | Gauge |
The total number of tasks the scheduler executor received. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumSchedulerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_completed_tasks_* | Gauge
| The number of tasks the worker executor execute completed. <br>The number of
metrics determined by the number of worker task thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf` <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_queue_* | Gauge | The
number of tasks queued in the worker executor's queue. <br>The number of
metrics determined by scheduler executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_total_tasks_* | Gauge |
The total number of tasks the worker executor received. <br>The number of
metrics determined by worker executor's thread number configured by
`managedLedgerNumWorkerThreads` in `broker.conf`. <br> |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_execution | Summary
| The scheduler task execution latency calculated in milliseconds. |
| pulsar_managedLedger_client_bookkeeper_ml_scheduler_task_queued | Summary |
The scheduler task queued latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_execution | Summary |
The worker task execution latency calculated in milliseconds. |
-| pulsar_managedLedger_client_bookkeeper_ml_workers_task_queued | Summary |
The worker task queued latency calculated in milliseconds. |
### Token metrics
