This is an automated email from the ASF dual-hosted git repository.
sijie pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6f713a1 [docs] Update configuration information for 2.4.x releases
(#6047)
6f713a1 is described below
commit 6f713a1368865568e6cedca319a5c7aba67eea40
Author: Jennifer Huang <[email protected]>
AuthorDate: Sun Jan 19 05:18:03 2020 +0800
[docs] Update configuration information for 2.4.x releases (#6047)
### Motivation
Some parameters are added in the `broker.conf` and `standalone.conf` files.
However, those parameters are not updated in the docs.
See the following PRs for details: #4150, #4066, #4197, #3819, #4261,
#4273, #4320.
### Modifications
Add those parameter info, and sync docs with the code.
Does not update the description quite much, there are two reasons for this:
1. Keep doc content consistent with code. We need to update the description
for those parameters in the code first, and then sync them in docs.
2. Will adopt a generator to generate those content automatically in the
near future.
---
site2/docs/reference-configuration.md | 12 ++++++-
site2/docs/reference-metrics.md | 1 +
.../version-2.4.0/reference-configuration.md | 38 ++++++++++++++++------
.../version-2.4.0}/reference-metrics.md | 4 ++-
.../version-2.4.1/reference-configuration.md | 38 +++++++++++++++-------
.../version-2.4.1/reference-metrics.md | 3 ++
.../version-2.4.2/reference-configuration.md | 38 +++++++++++++++-------
.../version-2.4.2/reference-metrics.md | 3 ++
8 files changed, 103 insertions(+), 34 deletions(-)
diff --git a/site2/docs/reference-configuration.md
b/site2/docs/reference-configuration.md
index a733eac..3f31893 100644
--- a/site2/docs/reference-configuration.md
+++ b/site2/docs/reference-configuration.md
@@ -122,11 +122,13 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for
which information will be stored for deduplication purposes. |10000|
|brokerDeduplicationEntriesInterval| The number of entries after which a
deduplication informational snapshot is taken. A larger interval will lead to
fewer snapshots being taken, though this would also lengthen the topic recovery
time (the time required for entries published after the snapshot to be
replayed). |1000|
|brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity
(in minutes) after which the broker will discard deduplication information
related to a disconnected producer. |360|
+|dispatchThrottlingRatePerReplicatorInMsg| The default messages per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message dispatch-throttling| 0 |
+|dispatchThrottlingRatePerReplicatorInByte| The default bytes per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message-byte dispatch-throttling| 0 |
|zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds
|30000|
|brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After
this time elapses, the process will be killed |60000|
|backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on
topic when the quota is reached |true|
|backlogQuotaCheckIntervalInSeconds| How often to check for topics that have
reached the quota |60|
-|backlogQuotaDefaultLimitGB| Default per-topic backlog quota limit |10|
+|backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 |
|allowAutoTopicCreation| Enable topic auto creation if a new producer or
consumer connected |true|
|allowAutoTopicCreationType| The topic type (partitioned or non-partitioned)
that is allowed to be automatically created. |Partitioned|
|defaultNumPartitions| The number of partitioned topics that is allowed to be
automatically created if `allowAutoTopicCreationType` is partitioned |1|
@@ -225,6 +227,7 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|loadManagerClassName| Name of load manager to use
|org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl|
|managedLedgerOffloadDriver| Driver to use to offload old data to long term
storage (Possible values: S3) ||
|managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for
ledger offloading |2|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
|managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully
offloaded to long term storage and the ledger being deleted from bookkeeper |
14400000|
|managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before
triggering automatic offload to long term storage |-1 (disabled)|
|s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region ||
@@ -361,11 +364,18 @@ The
[`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
|bookkeeperClientRegionawarePolicyEnabled| |false|
|bookkeeperClientReorderReadSequenceEnabled| |false|
|bookkeeperClientIsolationGroups|||
+|bookkeeperClientSecondaryIsolationGroups| Enable bookie secondary-isolation
group if bookkeeperClientIsolationGroups doesn't have enough bookie available.
||
+|bookkeeperClientMinAvailableBookiesInIsolationGroups| Minimum bookies that
should be available as part of bookkeeperClientIsolationGroups else broker will
include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. ||
|managedLedgerDefaultEnsembleSize| |1|
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
|managedLedgerCacheSizeMB| |1024|
+|managedLedgerCacheCopyEntries| Whether we should make a copy of the entry
payloads when inserting in cache| false|
|managedLedgerCacheEvictionWatermark| |0.9|
+|managedLedgerCacheEvictionFrequency| Configure the cache eviction frequency
for the managed ledger cache (evictions/sec) | 100.0 |
+|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed
in cache for more than the configured time, will be evicted | 1000 |
+|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of
entries) from where a cursor should be considered 'backlogged' and thus should
be set as inactive. | 1000|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
|managedLedgerDefaultMarkDeleteRateLimit| |0.1|
|managedLedgerMaxEntriesPerLedger| |50000|
|managedLedgerMinLedgerRolloverTimeMinutes| |10|
diff --git a/site2/docs/reference-metrics.md b/site2/docs/reference-metrics.md
index 3983dcd..d3e21bd 100644
--- a/site2/docs/reference-metrics.md
+++ b/site2/docs/reference-metrics.md
@@ -167,6 +167,7 @@ All the topic metrics are labelled with the following
labels:
| pulsar_storage_size | Gauge | The total storage size of the topics in this
topic owned by this broker (bytes). |
| pulsar_storage_backlog_size | Gauge | The total backlog size of the topics
of this topic owned by this broker (messages). |
| pulsar_storage_offloaded_size | Gauge | The total amount of the data in this
topic offloaded to the tiered storage (bytes). |
+| pulsar_storage_backlog_quota_limit | Gauge | The total amount of the data in
this topic that limit the backlog quota (bytes). |
| pulsar_storage_write_rate | Gauge | The total message batches (entries)
written to the storage for this topic (message batches / second). |
| pulsar_storage_read_rate | Gauge | The total message batches (entries) read
from the storage for this topic (message batches / second). |
| pulsar_subscription_delayed | Gauge | The total message batches (entries)
are delayed for dispatching. |
diff --git
a/site2/website/versioned_docs/version-2.4.0/reference-configuration.md
b/site2/website/versioned_docs/version-2.4.0/reference-configuration.md
index 6f8d4be..8540700 100644
--- a/site2/website/versioned_docs/version-2.4.0/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.4.0/reference-configuration.md
@@ -5,6 +5,7 @@ sidebar_label: Pulsar configuration
original_id: reference-configuration
---
+
<style type="text/css">
table{
font-size: 80%;
@@ -14,15 +15,16 @@ original_id: reference-configuration
Pulsar configuration can be managed either via a series of configuration files
contained in the [`conf`](https://github.com/apache/pulsar/tree/master/conf)
directory of a Pulsar [installation](getting-started-standalone.md)
-* [BookKeeper](#bookkeeper)
-* [Broker](#broker)
-* [Client](#client)
-* [Service discovery](#service-discovery)
-* [Log4j](#log4j)
-* [Log4j shell](#log4j-shell)
-* [Standalone](#standalone)
-* [WebSocket](#websocket)
-* [ZooKeeper](#zookeeper)
+- [BookKeeper](#bookkeeper)
+- [Broker](#broker)
+- [Client](#client)
+- [Service discovery](#service-discovery)
+- [Log4j](#log4j)
+- [Log4j shell](#log4j-shell)
+- [Standalone](#standalone)
+- [WebSocket](#websocket)
+- [Pulsar proxy](#pulsar-proxy)
+- [ZooKeeper](#zookeeper)
## BookKeeper
@@ -122,11 +124,16 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for
which information will be stored for deduplication purposes. |10000|
|brokerDeduplicationEntriesInterval| The number of entries after which a
deduplication informational snapshot is taken. A larger interval will lead to
fewer snapshots being taken, though this would also lengthen the topic recovery
time (the time required for entries published after the snapshot to be
replayed). |1000|
|brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity
(in minutes) after which the broker will discard deduplication information
related to a disconnected producer. |360|
+|dispatchThrottlingRatePerReplicatorInMsg| The default messages per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message dispatch-throttling| 0 |
+|dispatchThrottlingRatePerReplicatorInByte| The default bytes per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message-byte dispatch-throttling| 0 |
|zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds
|30000|
|brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After
this time elapses, the process will be killed |60000|
|backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on
topic when the quota is reached |true|
|backlogQuotaCheckIntervalInSeconds| How often to check for topics that have
reached the quota |60|
-|backlogQuotaDefaultLimitGB| Default per-topic backlog quota limit |10|
+|backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 |
+|allowAutoTopicCreation| Enable topic auto creation if a new producer or
consumer connected |true|
+|allowAutoTopicCreationType| The topic type (partitioned or non-partitioned)
that is allowed to be automatically created. |Partitioned|
+|defaultNumPartitions| The number of partitioned topics that is allowed to be
automatically created if `allowAutoTopicCreationType` is partitioned |1|
|brokerDeleteInactiveTopicsEnabled| Enable the deletion of inactive topics
|true|
|brokerDeleteInactiveTopicsFrequencySeconds| How often to check for inactive
topics |60|
|messageExpiryCheckIntervalInMinutes| How frequently to proactively check and
purge expired messages |5|
@@ -221,6 +228,9 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|loadManagerClassName| Name of load manager to use
|org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl|
|managedLedgerOffloadDriver| Driver to use to offload old data to long term
storage (Possible values: S3) ||
|managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for
ledger offloading |2|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
+|managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully
offloaded to long term storage and the ledger being deleted from bookkeeper |
14400000|
+|managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before
triggering automatic offload to long term storage |-1 (disabled)|
|s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region ||
|s3ManagedLedgerOffloadBucket| For Amazon S3 ledger offload, Bucket to place
offloaded ledger into ||
|s3ManagedLedgerOffloadServiceEndpoint| For Amazon S3 ledger offload,
Alternative endpoint to connect to (useful for testing) ||
@@ -355,11 +365,18 @@ The
[`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
|bookkeeperClientRegionawarePolicyEnabled| |false|
|bookkeeperClientReorderReadSequenceEnabled| |false|
|bookkeeperClientIsolationGroups|||
+|bookkeeperClientSecondaryIsolationGroups| Enable bookie secondary-isolation
group if bookkeeperClientIsolationGroups doesn't have enough bookie available.
||
+|bookkeeperClientMinAvailableBookiesInIsolationGroups| Minimum bookies that
should be available as part of bookkeeperClientIsolationGroups else broker will
include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. ||
|managedLedgerDefaultEnsembleSize| |1|
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
|managedLedgerCacheSizeMB| |1024|
+|managedLedgerCacheCopyEntries| Whether we should make a copy of the entry
payloads when inserting in cache| false|
|managedLedgerCacheEvictionWatermark| |0.9|
+|managedLedgerCacheEvictionFrequency| Configure the cache eviction frequency
for the managed ledger cache (evictions/sec) | 100.0 |
+|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed
in cache for more than the configured time, will be evicted | 1000 |
+|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of
entries) from where a cursor should be considered 'backlogged' and thus should
be set as inactive. | 1000|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
|managedLedgerDefaultMarkDeleteRateLimit| |0.1|
|managedLedgerMaxEntriesPerLedger| |50000|
|managedLedgerMinLedgerRolloverTimeMinutes| |10|
@@ -437,6 +454,7 @@ The [Pulsar
proxy](concepts-architecture-overview.md#pulsar-proxy) can be config
|servicePortTls| The port to use to server binary Protobuf TLS requests
|6651|
|statusFilePath| Path for the file used to determine the rotation status for
the proxy instance when responding to service discovery health checks ||
|authenticationEnabled| Whether authentication is enabled for the Pulsar proxy
|false|
+|authenticateMetricsEndpoint| Whether the '/metrics' endpoint requires
authentication. Defaults to true. 'authenticationEnabled' must also be set for
this to take effect. |true|
|authenticationProviders| Authentication provider name list (a comma-separated
list of class names) ||
|authorizationEnabled| Whether authorization is enforced by the Pulsar proxy
|false|
|authorizationProvider| Authorization provider as a fully qualified class name
|org.apache.pulsar.broker.authorization.PulsarAuthorizationProvider|
diff --git a/site2/docs/reference-metrics.md
b/site2/website/versioned_docs/version-2.4.0/reference-metrics.md
similarity index 98%
copy from site2/docs/reference-metrics.md
copy to site2/website/versioned_docs/version-2.4.0/reference-metrics.md
index 3983dcd..5cdb406 100644
--- a/site2/docs/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.4.0/reference-metrics.md
@@ -1,7 +1,8 @@
---
-id: reference-metrics
+id: version-2.4.0-reference-metrics
title: Pulsar Metrics
sidebar_label: Pulsar Metrics
+original_id: reference-metrics
---
<style type="text/css">
@@ -167,6 +168,7 @@ All the topic metrics are labelled with the following
labels:
| pulsar_storage_size | Gauge | The total storage size of the topics in this
topic owned by this broker (bytes). |
| pulsar_storage_backlog_size | Gauge | The total backlog size of the topics
of this topic owned by this broker (messages). |
| pulsar_storage_offloaded_size | Gauge | The total amount of the data in this
topic offloaded to the tiered storage (bytes). |
+| pulsar_storage_backlog_quota_limit | Gauge | The total amount of the data in
this topic that limit the backlog quota (bytes). |
| pulsar_storage_write_rate | Gauge | The total message batches (entries)
written to the storage for this topic (message batches / second). |
| pulsar_storage_read_rate | Gauge | The total message batches (entries) read
from the storage for this topic (message batches / second). |
| pulsar_subscription_delayed | Gauge | The total message batches (entries)
are delayed for dispatching. |
diff --git
a/site2/website/versioned_docs/version-2.4.1/reference-configuration.md
b/site2/website/versioned_docs/version-2.4.1/reference-configuration.md
index 7fde456..ec0291d 100644
--- a/site2/website/versioned_docs/version-2.4.1/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.4.1/reference-configuration.md
@@ -5,6 +5,7 @@ sidebar_label: Pulsar configuration
original_id: reference-configuration
---
+
<style type="text/css">
table{
font-size: 80%;
@@ -14,15 +15,16 @@ original_id: reference-configuration
Pulsar configuration can be managed either via a series of configuration files
contained in the [`conf`](https://github.com/apache/pulsar/tree/master/conf)
directory of a Pulsar [installation](getting-started-standalone.md)
-* [BookKeeper](#bookkeeper)
-* [Broker](#broker)
-* [Client](#client)
-* [Service discovery](#service-discovery)
-* [Log4j](#log4j)
-* [Log4j shell](#log4j-shell)
-* [Standalone](#standalone)
-* [WebSocket](#websocket)
-* [ZooKeeper](#zookeeper)
+- [BookKeeper](#bookkeeper)
+- [Broker](#broker)
+- [Client](#client)
+- [Service discovery](#service-discovery)
+- [Log4j](#log4j)
+- [Log4j shell](#log4j-shell)
+- [Standalone](#standalone)
+- [WebSocket](#websocket)
+- [Pulsar proxy](#pulsar-proxy)
+- [ZooKeeper](#zookeeper)
## BookKeeper
@@ -122,12 +124,16 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for
which information will be stored for deduplication purposes. |10000|
|brokerDeduplicationEntriesInterval| The number of entries after which a
deduplication informational snapshot is taken. A larger interval will lead to
fewer snapshots being taken, though this would also lengthen the topic recovery
time (the time required for entries published after the snapshot to be
replayed). |1000|
|brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity
(in minutes) after which the broker will discard deduplication information
related to a disconnected producer. |360|
+|dispatchThrottlingRatePerReplicatorInMsg| The default messages per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message dispatch-throttling| 0 |
+|dispatchThrottlingRatePerReplicatorInByte| The default bytes per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message-byte dispatch-throttling| 0 |
|zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds
|30000|
|brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After
this time elapses, the process will be killed |60000|
|backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on
topic when the quota is reached |true|
|backlogQuotaCheckIntervalInSeconds| How often to check for topics that have
reached the quota |60|
-|backlogQuotaDefaultLimitGB| Default per-topic backlog quota limit |10|
-|allowAutoTopicCreation| Enable topic auto creation if new producer or
consumer connected |true|
+|backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 |
+|allowAutoTopicCreation| Enable topic auto creation if a new producer or
consumer connected |true|
+|allowAutoTopicCreationType| The topic type (partitioned or non-partitioned)
that is allowed to be automatically created. |Partitioned|
+|defaultNumPartitions| The number of partitioned topics that is allowed to be
automatically created if `allowAutoTopicCreationType` is partitioned |1|
|brokerDeleteInactiveTopicsEnabled| Enable the deletion of inactive topics
|true|
|brokerDeleteInactiveTopicsFrequencySeconds| How often to check for inactive
topics |60|
|messageExpiryCheckIntervalInMinutes| How frequently to proactively check and
purge expired messages |5|
@@ -222,6 +228,9 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|loadManagerClassName| Name of load manager to use
|org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl|
|managedLedgerOffloadDriver| Driver to use to offload old data to long term
storage (Possible values: S3) ||
|managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for
ledger offloading |2|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
+|managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully
offloaded to long term storage and the ledger being deleted from bookkeeper |
14400000|
+|managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before
triggering automatic offload to long term storage |-1 (disabled)|
|s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region ||
|s3ManagedLedgerOffloadBucket| For Amazon S3 ledger offload, Bucket to place
offloaded ledger into ||
|s3ManagedLedgerOffloadServiceEndpoint| For Amazon S3 ledger offload,
Alternative endpoint to connect to (useful for testing) ||
@@ -356,11 +365,18 @@ The
[`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
|bookkeeperClientRegionawarePolicyEnabled| |false|
|bookkeeperClientReorderReadSequenceEnabled| |false|
|bookkeeperClientIsolationGroups|||
+|bookkeeperClientSecondaryIsolationGroups| Enable bookie secondary-isolation
group if bookkeeperClientIsolationGroups doesn't have enough bookie available.
||
+|bookkeeperClientMinAvailableBookiesInIsolationGroups| Minimum bookies that
should be available as part of bookkeeperClientIsolationGroups else broker will
include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. ||
|managedLedgerDefaultEnsembleSize| |1|
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
|managedLedgerCacheSizeMB| |1024|
+|managedLedgerCacheCopyEntries| Whether we should make a copy of the entry
payloads when inserting in cache| false|
|managedLedgerCacheEvictionWatermark| |0.9|
+|managedLedgerCacheEvictionFrequency| Configure the cache eviction frequency
for the managed ledger cache (evictions/sec) | 100.0 |
+|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed
in cache for more than the configured time, will be evicted | 1000 |
+|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of
entries) from where a cursor should be considered 'backlogged' and thus should
be set as inactive. | 1000|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
|managedLedgerDefaultMarkDeleteRateLimit| |0.1|
|managedLedgerMaxEntriesPerLedger| |50000|
|managedLedgerMinLedgerRolloverTimeMinutes| |10|
diff --git a/site2/website/versioned_docs/version-2.4.1/reference-metrics.md
b/site2/website/versioned_docs/version-2.4.1/reference-metrics.md
index 192fe7c..c852bbd 100644
--- a/site2/website/versioned_docs/version-2.4.1/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.4.1/reference-metrics.md
@@ -168,11 +168,14 @@ All the topic metrics are labelled with the following
labels:
| pulsar_storage_size | Gauge | The total storage size of the topics in this
topic owned by this broker (bytes). |
| pulsar_storage_backlog_size | Gauge | The total backlog size of the topics
of this topic owned by this broker (messages). |
| pulsar_storage_offloaded_size | Gauge | The total amount of the data in this
topic offloaded to the tiered storage (bytes). |
+| pulsar_storage_backlog_quota_limit | Gauge | The total amount of the data in
this topic that limit the backlog quota (bytes). |
| pulsar_storage_write_rate | Gauge | The total message batches (entries)
written to the storage for this topic (message batches / second). |
| pulsar_storage_read_rate | Gauge | The total message batches (entries) read
from the storage for this topic (message batches / second). |
| pulsar_subscription_delayed | Gauge | The total message batches (entries)
are delayed for dispatching. |
| pulsar_storage_write_latency_le_* | Histogram | The entry rate of a topic
that the storage write latency is smaller with a given threshold.<br> Available
thresholds: <br><ul><li>pulsar_storage_write_latency_le_0_5: <= 0.5ms
</li><li>pulsar_storage_write_latency_le_1: <=
1ms</li><li>pulsar_storage_write_latency_le_5: <=
5ms</li><li>pulsar_storage_write_latency_le_10: <=
10ms</li><li>pulsar_storage_write_latency_le_20: <=
20ms</li><li>pulsar_storage_write_latency_le_50: <= 50ms</li><li>p [...]
| pulsar_entry_size_le_* | Histogram | The entry rate of a topic that the
entry size is smaller with a given threshold.<br> Available thresholds:
<br><ul><li>pulsar_entry_size_le_128: <= 128 bytes
</li><li>pulsar_entry_size_le_512: <= 512
bytes</li><li>pulsar_entry_size_le_1_kb: <= 1
KB</li><li>pulsar_entry_size_le_2_kb: <= 2
KB</li><li>pulsar_entry_size_le_4_kb: <= 4
KB</li><li>pulsar_entry_size_le_16_kb: <= 16
KB</li><li>pulsar_entry_size_le_100_kb: <= 100 KB</li><li>pulsar_entry_size_
[...]
+| pulsar_in_bytes_total | Counter | The total number of bytes received for
this topic |
+| pulsar_producers_count | Counter | The total number of messages received for
this topic |
#### Replication metrics
diff --git
a/site2/website/versioned_docs/version-2.4.2/reference-configuration.md
b/site2/website/versioned_docs/version-2.4.2/reference-configuration.md
index 9d47e7a..adc576e 100644
--- a/site2/website/versioned_docs/version-2.4.2/reference-configuration.md
+++ b/site2/website/versioned_docs/version-2.4.2/reference-configuration.md
@@ -5,6 +5,7 @@ sidebar_label: Pulsar configuration
original_id: reference-configuration
---
+
<style type="text/css">
table{
font-size: 80%;
@@ -14,15 +15,16 @@ original_id: reference-configuration
Pulsar configuration can be managed either via a series of configuration files
contained in the [`conf`](https://github.com/apache/pulsar/tree/master/conf)
directory of a Pulsar [installation](getting-started-standalone.md)
-* [BookKeeper](#bookkeeper)
-* [Broker](#broker)
-* [Client](#client)
-* [Service discovery](#service-discovery)
-* [Log4j](#log4j)
-* [Log4j shell](#log4j-shell)
-* [Standalone](#standalone)
-* [WebSocket](#websocket)
-* [ZooKeeper](#zookeeper)
+- [BookKeeper](#bookkeeper)
+- [Broker](#broker)
+- [Client](#client)
+- [Service discovery](#service-discovery)
+- [Log4j](#log4j)
+- [Log4j shell](#log4j-shell)
+- [Standalone](#standalone)
+- [WebSocket](#websocket)
+- [Pulsar proxy](#pulsar-proxy)
+- [ZooKeeper](#zookeeper)
## BookKeeper
@@ -122,12 +124,16 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|brokerDeduplicationMaxNumberOfProducers| The maximum number of producers for
which information will be stored for deduplication purposes. |10000|
|brokerDeduplicationEntriesInterval| The number of entries after which a
deduplication informational snapshot is taken. A larger interval will lead to
fewer snapshots being taken, though this would also lengthen the topic recovery
time (the time required for entries published after the snapshot to be
replayed). |1000|
|brokerDeduplicationProducerInactivityTimeoutMinutes| The time of inactivity
(in minutes) after which the broker will discard deduplication information
related to a disconnected producer. |360|
+|dispatchThrottlingRatePerReplicatorInMsg| The default messages per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message dispatch-throttling| 0 |
+|dispatchThrottlingRatePerReplicatorInByte| The default bytes per second
dispatch throttling-limit for every replicator in replication. The value of `0`
means disabling replication message-byte dispatch-throttling| 0 |
|zooKeeperSessionTimeoutMillis| Zookeeper session timeout in milliseconds
|30000|
|brokerShutdownTimeoutMs| Time to wait for broker graceful shutdown. After
this time elapses, the process will be killed |60000|
|backlogQuotaCheckEnabled| Enable backlog quota check. Enforces action on
topic when the quota is reached |true|
|backlogQuotaCheckIntervalInSeconds| How often to check for topics that have
reached the quota |60|
-|backlogQuotaDefaultLimitGB| Default per-topic backlog quota limit |10|
-|allowAutoTopicCreation| Enable topic auto creation if new producer or
consumer connected |true|
+|backlogQuotaDefaultLimitGB| The default per-topic backlog quota limit | -1 |
+|allowAutoTopicCreation| Enable topic auto creation if a new producer or
consumer connected |true|
+|allowAutoTopicCreationType| The topic type (partitioned or non-partitioned)
that is allowed to be automatically created. |Partitioned|
+|defaultNumPartitions| The number of partitioned topics that is allowed to be
automatically created if `allowAutoTopicCreationType` is partitioned |1|
|brokerDeleteInactiveTopicsEnabled| Enable the deletion of inactive topics
|true|
|brokerDeleteInactiveTopicsFrequencySeconds| How often to check for inactive
topics |60|
|messageExpiryCheckIntervalInMinutes| How frequently to proactively check and
purge expired messages |5|
@@ -222,6 +228,9 @@ Pulsar brokers are responsible for handling incoming
messages from producers, di
|loadManagerClassName| Name of load manager to use
|org.apache.pulsar.broker.loadbalance.impl.SimpleLoadManagerImpl|
|managedLedgerOffloadDriver| Driver to use to offload old data to long term
storage (Possible values: S3) ||
|managedLedgerOffloadMaxThreads| Maximum number of thread pool threads for
ledger offloading |2|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
+|managedLedgerOffloadDeletionLagMs|Delay between a ledger being successfully
offloaded to long term storage and the ledger being deleted from bookkeeper |
14400000|
+|managedLedgerOffloadAutoTriggerSizeThresholdBytes|The number of bytes before
triggering automatic offload to long term storage |-1 (disabled)|
|s3ManagedLedgerOffloadRegion| For Amazon S3 ledger offload, AWS region ||
|s3ManagedLedgerOffloadBucket| For Amazon S3 ledger offload, Bucket to place
offloaded ledger into ||
|s3ManagedLedgerOffloadServiceEndpoint| For Amazon S3 ledger offload,
Alternative endpoint to connect to (useful for testing) ||
@@ -356,11 +365,18 @@ The
[`pulsar-client`](reference-cli-tools.md#pulsar-client) CLI tool can be used
|bookkeeperClientRegionawarePolicyEnabled| |false|
|bookkeeperClientReorderReadSequenceEnabled| |false|
|bookkeeperClientIsolationGroups|||
+|bookkeeperClientSecondaryIsolationGroups| Enable bookie secondary-isolation
group if bookkeeperClientIsolationGroups doesn't have enough bookie available.
||
+|bookkeeperClientMinAvailableBookiesInIsolationGroups| Minimum bookies that
should be available as part of bookkeeperClientIsolationGroups else broker will
include bookkeeperClientSecondaryIsolationGroups bookies in isolated list. ||
|managedLedgerDefaultEnsembleSize| |1|
|managedLedgerDefaultWriteQuorum| |1|
|managedLedgerDefaultAckQuorum| |1|
|managedLedgerCacheSizeMB| |1024|
+|managedLedgerCacheCopyEntries| Whether we should make a copy of the entry
payloads when inserting in cache| false|
|managedLedgerCacheEvictionWatermark| |0.9|
+|managedLedgerCacheEvictionFrequency| Configure the cache eviction frequency
for the managed ledger cache (evictions/sec) | 100.0 |
+|managedLedgerCacheEvictionTimeThresholdMillis| All entries that have stayed
in cache for more than the configured time, will be evicted | 1000 |
+|managedLedgerCursorBackloggedThreshold| Configure the threshold (in number of
entries) from where a cursor should be considered 'backlogged' and thus should
be set as inactive. | 1000|
+|managedLedgerUnackedRangesOpenCacheSetEnabled| Use Open Range-Set to cache
unacknowledged messages |true|
|managedLedgerDefaultMarkDeleteRateLimit| |0.1|
|managedLedgerMaxEntriesPerLedger| |50000|
|managedLedgerMinLedgerRolloverTimeMinutes| |10|
diff --git a/site2/website/versioned_docs/version-2.4.2/reference-metrics.md
b/site2/website/versioned_docs/version-2.4.2/reference-metrics.md
index 28c404f..ecb107f 100644
--- a/site2/website/versioned_docs/version-2.4.2/reference-metrics.md
+++ b/site2/website/versioned_docs/version-2.4.2/reference-metrics.md
@@ -168,11 +168,14 @@ All the topic metrics are labelled with the following
labels:
| pulsar_storage_size | Gauge | The total storage size of the topics in this
topic owned by this broker (bytes). |
| pulsar_storage_backlog_size | Gauge | The total backlog size of the topics
of this topic owned by this broker (messages). |
| pulsar_storage_offloaded_size | Gauge | The total amount of the data in this
topic offloaded to the tiered storage (bytes). |
+| pulsar_storage_backlog_quota_limit | Gauge | The total amount of the data in
this topic that limit the backlog quota (bytes). |
| pulsar_storage_write_rate | Gauge | The total message batches (entries)
written to the storage for this topic (message batches / second). |
| pulsar_storage_read_rate | Gauge | The total message batches (entries) read
from the storage for this topic (message batches / second). |
| pulsar_subscription_delayed | Gauge | The total message batches (entries)
are delayed for dispatching. |
| pulsar_storage_write_latency_le_* | Histogram | The entry rate of a topic
that the storage write latency is smaller with a given threshold.<br> Available
thresholds: <br><ul><li>pulsar_storage_write_latency_le_0_5: <= 0.5ms
</li><li>pulsar_storage_write_latency_le_1: <=
1ms</li><li>pulsar_storage_write_latency_le_5: <=
5ms</li><li>pulsar_storage_write_latency_le_10: <=
10ms</li><li>pulsar_storage_write_latency_le_20: <=
20ms</li><li>pulsar_storage_write_latency_le_50: <= 50ms</li><li>p [...]
| pulsar_entry_size_le_* | Histogram | The entry rate of a topic that the
entry size is smaller with a given threshold.<br> Available thresholds:
<br><ul><li>pulsar_entry_size_le_128: <= 128 bytes
</li><li>pulsar_entry_size_le_512: <= 512
bytes</li><li>pulsar_entry_size_le_1_kb: <= 1
KB</li><li>pulsar_entry_size_le_2_kb: <= 2
KB</li><li>pulsar_entry_size_le_4_kb: <= 4
KB</li><li>pulsar_entry_size_le_16_kb: <= 16
KB</li><li>pulsar_entry_size_le_100_kb: <= 100 KB</li><li>pulsar_entry_size_
[...]
+| pulsar_in_bytes_total | Counter | The total number of bytes received for
this topic |
+| pulsar_producers_count | Counter | The total number of messages received for
this topic |
#### Replication metrics
