This is an automated email from the ASF dual-hosted git repository.
urfree pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/pulsar-site.git
The following commit(s) were added to refs/heads/main by this push:
new 39258f1027b Docs sync done from apache/pulsar(#74bafe2)
39258f1027b is described below
commit 39258f1027bf056696569bf7e20edbb92089ede7
Author: Pulsar Site Updater <[email protected]>
AuthorDate: Wed Jul 20 18:01:34 2022 +0000
Docs sync done from apache/pulsar(#74bafe2)
---
.../docs/administration-isolation-bookie.md | 69 ++++++++++++
.../docs/administration-isolation-broker.md | 57 ++++++++++
.../website-next/docs/administration-isolation.md | 125 ++++++---------------
site2/website-next/docs/reference-metrics.md | 5 +-
site2/website-next/sidebars.json | 14 ++-
site2/website-next/static/assets/isolation-1.png | Bin 0 -> 74607 bytes
site2/website-next/static/assets/isolation-2.png | Bin 0 -> 83257 bytes
site2/website-next/static/assets/isolation-3.png | Bin 0 -> 79529 bytes
site2/website-next/static/assets/isolation-4.png | Bin 0 -> 129134 bytes
9 files changed, 175 insertions(+), 95 deletions(-)
diff --git a/site2/website-next/docs/administration-isolation-bookie.md
b/site2/website-next/docs/administration-isolation-bookie.md
new file mode 100644
index 00000000000..20ca5a155c0
--- /dev/null
+++ b/site2/website-next/docs/administration-isolation-bookie.md
@@ -0,0 +1,69 @@
+---
+id: administration-isolation-bookie
+title: Isolate bookies
+sidebar_label: "Isolate bookies"
+---
+
+````mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+````
+
+
+A namespace can be isolated into user-defined groups of bookies, which
guarantees all the data that belongs to the namespace is stored in desired
bookies. The bookie affinity group uses the BookKeeper [rack-aware placement
policy](https://bookkeeper.apache.org/docs/latest/api/javadoc/org/apache/bookkeeper/client/EnsemblePlacementPolicy.html)
and it is a way to feed rack information which is stored as JSON format in
znode.
+
+You can set a bookie affinity group using one of the following methods.
+
+````mdx-code-block
+<Tabs
+ defaultValue="Admin CLI"
+ values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST
API","value":"REST API"},{"label":"Java admin API","value":"Java admin API"}]}>
+
+<TabItem value="Admin CLI">
+
+```
+
+pulsar-admin namespaces set-bookie-affinity-group options
+
+```
+
+For more information about the command `pulsar-admin namespaces
set-bookie-affinity-group options`, see [Pulsar admin
docs](https://pulsar.apache.org/tools/pulsar-admin/).
+
+**Example**
+
+```shell
+
+bin/pulsar-admin bookies set-bookie-rack \
+--bookie 127.0.0.1:3181 \
+--hostname 127.0.0.1:3181 \
+--group group-bookie1 \
+--rack rack1
+
+bin/pulsar-admin namespaces set-bookie-affinity-group public/default \
+--primary-group group-bookie1
+
+```
+
+:::note
+
+- Do not set a bookie rack name to slash (`/`) or an empty string (`""`) if
you use Pulsar earlier than 2.7.5, 2.8.3, and 2.9.2. If you use Pulsar 2.7.5,
2.8.3, 2.9.2 or later versions, it falls back to `/default-rack` or
`/default-region/default-rack`.
+- When `RackawareEnsemblePlacementPolicy` is enabled, the rack name is not
allowed to contain slash (`/`) except for the beginning and end of the rack
name string. For example, rack name like `/rack0` is okay, but `/rack/0` is not
allowed.
+- When `RegionawareEnsemblePlacementPolicy` is enabled, the rack name can only
contain one slash (`/`) except for the beginning and end of the rack name
string. For example, rack name like `/region0/rack0` is okay, but
`/region0rack0` and `/region0/rack/0` are not allowed.
+For the bookie rack name restrictions, see [pulsar-admin bookies
set-bookie-rack](https://pulsar.apache.org/tools/pulsar-admin/).
+
+:::
+
+</TabItem>
+<TabItem value="REST API">
+
+[POST
/admin/v2/namespaces/{tenant}/{namespace}/persistence/bookieAffinity](https://pulsar.apache.org/admin-rest-api/?version=master&apiversion=v2#operation/setBookieAffinityGroup)
+
+</TabItem>
+<TabItem value="Java admin API">
+
+For how to set bookie affinity group for a namespace using Java admin API, see
[code](https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java#L1164).
+
+</TabItem>
+
+</Tabs>
+````
diff --git a/site2/website-next/docs/administration-isolation-broker.md
b/site2/website-next/docs/administration-isolation-broker.md
new file mode 100644
index 00000000000..11a857eacf6
--- /dev/null
+++ b/site2/website-next/docs/administration-isolation-broker.md
@@ -0,0 +1,57 @@
+---
+id: administration-isolation-broker
+title: Isolate brokers
+sidebar_label: "Isolate brokers"
+---
+
+````mdx-code-block
+import Tabs from '@theme/Tabs';
+import TabItem from '@theme/TabItem';
+````
+
+
+In Pulsar, when namespaces (more specifically, namespace bundles) are assigned
dynamically to brokers, the namespace isolation policy limits the set of
brokers that can be used for assignment. Before topics are assigned to brokers,
you can set the namespace isolation policy with a primary or a secondary regex
to select desired brokers.
+
+You can set a namespace isolation policy for a cluster using one of the
following methods.
+
+````mdx-code-block
+<Tabs
+ defaultValue="Admin CLI"
+ values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST
API","value":"REST API"},{"label":"Java admin API","value":"Java admin API"}]}>
+
+<TabItem value="Admin CLI">
+
+```
+
+pulsar-admin ns-isolation-policy set options
+
+```
+
+For more information about the command `pulsar-admin ns-isolation-policy set
options`, see [Pulsar admin
docs](https://pulsar.apache.org/tools/pulsar-admin/).
+
+**Example**
+
+```shell
+
+bin/pulsar-admin ns-isolation-policy set \
+--auto-failover-policy-type min_available \
+--auto-failover-policy-params min_limit=1,usage_threshold=80 \
+--namespaces my-tenant/my-namespace \
+--primary 10.193.216.* my-cluster policy-name
+
+```
+
+</TabItem>
+<TabItem value="REST API">
+
+[PUT
/admin/v2/namespaces/{tenant}/{namespace}](https://pulsar.apache.org/admin-rest-api/?version=master&apiversion=v2#operation/createNamespace)
+
+</TabItem>
+<TabItem value="Java admin API">
+
+For how to set namespace isolation policy using Java admin API, see
[code](https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java#L251).
+
+</TabItem>
+
+</Tabs>
+````
diff --git a/site2/website-next/docs/administration-isolation.md
b/site2/website-next/docs/administration-isolation.md
index 22f9b94337d..02a11a95ca7 100644
--- a/site2/website-next/docs/administration-isolation.md
+++ b/site2/website-next/docs/administration-isolation.md
@@ -4,120 +4,61 @@ title: Pulsar isolation
sidebar_label: "Pulsar isolation"
---
-````mdx-code-block
-import Tabs from '@theme/Tabs';
-import TabItem from '@theme/TabItem';
-````
-
In an organization, a Pulsar instance provides services to multiple teams.
When organizing the resources across multiple teams, you want to make a
suitable isolation plan to avoid the resource competition between different
teams and applications and provide high-quality messaging service. In this
case, you need to take resource isolation into consideration and weigh your
intended actions against expected and unexpected consequences.
-To enforce resource isolation, you can use the Pulsar isolation policy, which
allows you to allocate resources (**broker** and **bookie**) for the namespace.
-
-## Broker isolation
-
-In Pulsar, when namespaces (more specifically, namespace bundles) are assigned
dynamically to brokers, the namespace isolation policy limits the set of
brokers that can be used for assignment. Before topics are assigned to brokers,
you can set the namespace isolation policy with a primary or a secondary regex
to select desired brokers.
-
-You can set a namespace isolation policy for a cluster using one of the
following methods.
-
-````mdx-code-block
-<Tabs groupId="api-choice"
- defaultValue="Admin CLI"
- values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST
API","value":"REST API"},{"label":"Java Admin API","value":"Java Admin API"}]}>
-
-<TabItem value="Admin CLI">
-
-```
-
-pulsar-admin ns-isolation-policy set options
-
-```
-
-For more information about the command `pulsar-admin ns-isolation-policy set
options`, see [here](/tools/pulsar-admin/).
-
-**Example**
-
-```shell
-
-bin/pulsar-admin ns-isolation-policy set \
---auto-failover-policy-type min_available \
---auto-failover-policy-params min_limit=1,usage_threshold=80 \
---namespaces my-tenant/my-namespace \
---primary 10.193.216.* my-cluster policy-name
+The multi-layer and segment-centric architecture and hierarchical resource
management of Pulsar provide a solid foundation for isolation, which allows you
to isolate resources in your desired manner, prevent resource competition, and
attain stability.
-```
-</TabItem>
-<TabItem value="REST API">
+## Isolation methods
+To enforce resource isolation within Pulsar, you can use the following methods
of isolating broker and bookie resources:
+* [Isolate brokers](administration-isolation-broker.md)
+* [Isolate bookies](administration-isolation-bookie.md)
-[PUT
/admin/v2/namespaces/{tenant}/{namespace}](/admin-rest-api/?version=master&apiversion=v2#operation/createNamespace)
-</TabItem>
-<TabItem value="Java Admin API">
+## Deployments to achieve isolation within Pulsar
-For how to set namespace isolation policy using Java admin API, see
[here](https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java#L251).
+### Separate Pulsar clusters
-</TabItem>
+The following illustration demonstrates the deployment of separate Pulsar
clusters to achieve the highest-level isolation.
-</Tabs>
-````
+
-## Bookie isolation
+Here are some key points for understanding how it works:
+- Separate Pulsar clusters use a shared [configuration
store](concepts-architecture-overview.md/#configuration-store).
+- Each cluster exposes its service through a DNS entry point and makes sure a
client can access the cluster through the DNS entry point. Clients can use one
or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
+- Each Pulsar cluster has one or multiple brokers and bookies.
+- Each Pulsar cluster has one metadata store, which can be separated into
[Pulsar metadata store](concepts-architecture-overview.md/#metadata-store) and
[BookKeeper metadata
store](https://bookkeeper.apache.org/docs/latest/getting-started/concepts/#metadata-storage).
-A namespace can be isolated into user-defined groups of bookies, which
guarantees all the data that belongs to the namespace is stored in desired
bookies. The bookie affinity group uses the BookKeeper [rack-aware placement
policy](https://bookkeeper.apache.org/docs/latest/api/javadoc/org/apache/bookkeeper/client/EnsemblePlacementPolicy.html)
and it is a way to feed rack information which is stored as JSON format in
znode.
-
-You can set a bookie affinity group using one of the following methods.
-
-````mdx-code-block
-<Tabs groupId="api-choice"
- defaultValue="Admin CLI"
- values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST
API","value":"REST API"},{"label":"Java Admin API","value":"Java Admin API"}]}>
-
-<TabItem value="Admin CLI">
-
-```
-
-pulsar-admin namespaces set-bookie-affinity-group options
-
-```
-
-For more information about the command `pulsar-admin namespaces
set-bookie-affinity-group options`, see [here](/tools/pulsar-admin/).
-
-**Example**
+:::note
-```shell
+When using this approach, if you want to achieve namespace isolation, you need
to specify a cluster for a namespace. The cluster must be in the allowed
cluster list of the tenant. Topics under the namespace are assigned to this
cluster.
-bin/pulsar-admin bookies set-bookie-rack \
---bookie 127.0.0.1:3181 \
---hostname 127.0.0.1:3181 \
---group group-bookie1 \
---rack rack1
+:::
-bin/pulsar-admin namespaces set-bookie-affinity-group public/default \
---primary-group group-bookie1
+### Shared BookKeeper cluster
-```
+The following illustration demonstrates the deployment of shared BookKeeper
clusters to achieve isolation.
-:::note
+
-- Do not set a bookie rack name to slash (`/`) or an empty string (`""`) if
you use Pulsar earlier than 2.7.5, 2.8.3, and 2.9.2. If you use Pulsar 2.7.5,
2.8.3, 2.9.2 or later versions, it falls back to `/default-rack` or
`/default-region/default-rack`.
-- When `RackawareEnsemblePlacementPolicy` is enabled, the rack name is not
allowed to contain slash (`/`) except for the beginning and end of the rack
name string. For example, rack name like `/rack0` is okay, but `/rack/0` is not
allowed.
-- When `RegionawareEnsemblePlacementPolicy` is enabled, the rack name can only
contain one slash (`/`) except for the beginning and end of the rack name
string. For example, rack name like `/region0/rack0` is okay, but
`/region0rack0` and `/region0/rack/0` are not allowed.
-For the bookie rack name restrictions, see [pulsar-admin bookies
set-bookie-rack](/tools/pulsar-admin/).
+Here are some key points for understanding how it works:
+- Separate Pulsar clusters share a BookKeeper cluster and a [configuration
store](concepts-architecture-overview.md/#configuration-store).
+- Each cluster exposes its service through a DNS entry point and makes sure a
client can access the cluster through the DNS entry point. Clients can use one
or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
+- Each Pulsar cluster has one or multiple brokers.
+- Each Pulsar cluster has one metadata store.
-:::
+As illustrated below, storage isolation is achieved by setting [bookie
affinity groups](administration-isolation-bookie.md). All bookie groups use a
shared BookKeeper cluster and a metadata store, and each bookie isolation group
has one or several bookies. You can specify one or multiple primary/secondary
groups for a namespace. Topics under the namespace are created on the bookies
in the primary group firstly and then created on the bookies in the secondary
group.
-</TabItem>
-<TabItem value="REST API">
+
-[POST
/admin/v2/namespaces/{tenant}/{namespace}/persistence/bookieAffinity](/admin-rest-api/?version=master&apiversion=v2#operation/setBookieAffinityGroup)
+### Single Pulsar cluster
-</TabItem>
-<TabItem value="Java Admin API">
+The following illustration demonstrates how to achieve isolation inside a
single Pulsar cluster.
-For how to set bookie affinity group for a namespace using Java admin API, see
[here](https://github.com/apache/pulsar/blob/master/pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/NamespacesImpl.java#L1164).
+
-</TabItem>
+Here are some key points for understanding how it works:
+- Each cluster exposes its service through a DNS entry point and makes sure a
client can access the cluster through the DNS entry point. Clients can use one
or multiple Pulsar URLs that the Pulsar cluster exposes as the service URL.
+- Broker isolation is achieved by setting [namespace isolation
policy](administration-isolation-broker.md).
-</Tabs>
-````
diff --git a/site2/website-next/docs/reference-metrics.md
b/site2/website-next/docs/reference-metrics.md
index ec978f877c4..0b7c98f2dc3 100644
--- a/site2/website-next/docs/reference-metrics.md
+++ b/site2/website-next/docs/reference-metrics.md
@@ -97,14 +97,15 @@ in the `bookkeeper.conf` configuration file.
| auditor_NUM_LEDGERS_HAVING_NO_REPLICA_OF_AN_ENTRY | Gauge | Gauge for number
of ledgers having an entry with all the replicas missing. |
| auditor_NUM_LEDGERS_HAVING_LESS_THAN_AQ_REPLICAS_OF_AN_ENTRY | Gauge | Gauge
for number of ledgers having an entry with less than AQ number of replicas,
this doesn't include ledgers counted towards
numLedgersHavingNoReplicaOfAnEntry. |
| auditor_NUM_LEDGERS_HAVING_LESS_THAN_WQ_REPLICAS_OF_AN_ENTRY | Gauge | Gauge
for number of ledgers having an entry with less than WQ number of replicas,
this doesn't include ledgers counted towards
numLedgersHavingLessThanAQReplicasOfAnEntry. |
+| election_attempts | Counter | The number of auditor election attempts. |
| replication_worker_NUM_BYTES_READ | Summary | The distribution of size of
entries read by the replicator. |
| replication_worker_NUM_ENTRIES_READ | Counter | Number of entries read by
the replicator. |
| replication_worker_NUM_ENTRIES_WRITTEN | Counter | Number of entries written
by the replicator. |
| replication_worker_NUM_BYTES_WRITTEN | Summary | The distribution of size of
entries written by the replicator. |
| replication_worker_READ_DATA_LATENCY | Summary | The distribution of latency
of read entries by the replicator. |
| replication_worker_WRITE_DATA_LATENCY | Summary | The distribution of
latency of write entries by the replicator. |
-| replication_worker_REPLICATE_EXCEPTION | Summary | Replication related
exceptions. |
-| replication_worker_REREPLICATE_OP | Summary | Operation stats of
re-replicating ledgers. |
+| replication_worker_exceptions | Summary | Replication related exceptions. |
+| replication_worker_rereplicate | Summary | Operation stats of re-replicating
ledgers. |
| replication_worker_NUM_FULL_OR_PARTIAL_LEDGERS_REPLICATED | Counter | The
number of ledgers re-replicated. |
| replication_worker_NUM_DEFER_LEDGER_LOCK_RELEASE_OF_FAILED_LEDGER | Counter
| The number of defer-ledger-lock-releases of failed ledgers. |
| replication_worker_NUM_ENTRIES_UNABLE_TO_READ_FOR_REPLICATION | Counter |
The number of entries ReplicationWorker unable to read. |
diff --git a/site2/website-next/sidebars.json b/site2/website-next/sidebars.json
index 960baf32f87..32bd994ecee 100644
--- a/site2/website-next/sidebars.json
+++ b/site2/website-next/sidebars.json
@@ -225,6 +225,7 @@
"label": "Administration",
"items": [
"administration-zk-bk",
+ "administration-metadata-store"
"administration-geo",
"administration-pulsar-manager",
"administration-stats",
@@ -232,7 +233,18 @@
"administration-proxy",
"administration-isolation",
"administration-upgrade",
- "administration-metadata-store"
+ {
+ "type": "category",
+ "label": "Pulsar isolation",
+ "link": {
+ "type": "doc",
+ "id": "administration-isolation"
+ },
+ "items": [
+ "administration-isolation-broker",
+ "administration-isolation-bookie"
+ ]
+ }
]
},
{
diff --git a/site2/website-next/static/assets/isolation-1.png
b/site2/website-next/static/assets/isolation-1.png
new file mode 100644
index 00000000000..53c41192144
Binary files /dev/null and b/site2/website-next/static/assets/isolation-1.png
differ
diff --git a/site2/website-next/static/assets/isolation-2.png
b/site2/website-next/static/assets/isolation-2.png
new file mode 100644
index 00000000000..9ba8f609c04
Binary files /dev/null and b/site2/website-next/static/assets/isolation-2.png
differ
diff --git a/site2/website-next/static/assets/isolation-3.png
b/site2/website-next/static/assets/isolation-3.png
new file mode 100644
index 00000000000..ffe4545f87c
Binary files /dev/null and b/site2/website-next/static/assets/isolation-3.png
differ
diff --git a/site2/website-next/static/assets/isolation-4.png
b/site2/website-next/static/assets/isolation-4.png
new file mode 100644
index 00000000000..9b4670b8a21
Binary files /dev/null and b/site2/website-next/static/assets/isolation-4.png
differ
