codelipenghui commented on code in PR #21648:
URL: https://github.com/apache/pulsar/pull/21648#discussion_r1462601069
##########
pip/pip-321.md:
##########
@@ -0,0 +1,241 @@
+# PIP-321: Split the responsibilities of namespace replication-clusters
+
+# Background knowledge
+
+Pulsar's geo-replication mechanism is typically used for disaster recovery,
enabling the replication of persistently stored message data across multiple
data centers. For instance, your application publishes data in one region, and
you would like to process it for consumption in other regions. With Pulsar's
geo-replication mechanism, messages can be produced and consumed in different
geo-replicated regions. See the introduction of geo-replication to get more
information.[1]
+
+A client can set allowed clusters for a tenant. The allowed-cluster for the
tenant is a cluster that the tenant can access.
+
+A client can set replication clusters for a namespace, and the pulsar broker
internally manages replication to all the replication clusters. And the
replication clusters for a namespace must be a subgroup of the tenant's allowed
clusters.
+
+A client can not set allowed clusters at the namespace level, although PIP-8
[2] essentially achieves a similar functionality. It does not formally propose
this configuration, but the implementation uses replication-clusters as
allowed-clusters for a namespace. It introduces peer cluster for global
namespace redirection and fails `PartitionedMetadata-Lookup` request if global
namespace's replication-clusters doesn't contain current/peer-clusters. See
more information about this in PIP-8. [2]
+
+A namespace has multiple topics. Once a namespace is configured with
replication clusters, all the topics under this namespace will enable
replication in these clusters.
+
+Namespace Policy is a configuration in the namespace level, that is stored in
the metadata store, e.g. zookeeper, and this configuration can not be accessed
across multiple clusters with different metadata stores.
+
+Topic Policy refers to the configurations at the topic level, which are stored
in the BookKeeper ledger. Users can specify whether it is global. If it is
global, the topic policy will be replicated to the multiple clusters. The
clusters are configured by replication clusters now. In fact, these are the
clusters specified in the replication clusters of the `topic policy` topic. See
more information about global topic policy in PIP-92 [3]
+
+Replication clusters can be configured at the message level. Pulsar support
setting replication clusters when send messages.
+```java
+producer.newMessage().replicationClusters(List.of("cluster1",
"cluster2")).send();
+```
+
+[1] https://pulsar.apache.org/docs/3.1.x/concepts-replication
+[2] https://github.com/apache/pulsar/pull/903
+[3]
https://github.com/apache/pulsar/wiki/PIP-92%3A-Topic-policy-across-multiple-clusters
+
+# Motivation
+
+Geo-replication at the topic level and message level can't work as expected
when geo-replication is disabled at the namespace level and the clusters share
the same metadata store.
+Let's see an example:
+
+**Example For Topic Level:**
+
+- Environment:
+ - cluster1 and cluster2 in different regions sharing the same Zookeeper
cluster.
+
+- Replication clusters configuration:
+ - Set namespace `ns` replication clusters : cluster1 (local cluster)
+ - Set topic `ns/topic1` replication clusters : cluster1, cluster2.
+
+- Expected:
+ - Topic `ns/topic1` can replicate between cluster1 and cluster2.
+
+- Actual:
+ - Topic cannot be created at cluster2.
+
+**Example For Message Level**
+
+- Environment:
+ - cluster1 and cluster2 in different regions sharing the same Zookeeper
cluster.
+
+- Replication clusters configuration:
+ - Set namespace `ns` replication clusters : cluster1 (local cluster)
+ - Set replication clusters when send message1: cluster1, cluster2.
+
+- Expected:
+ - Message1 can replicate between cluster1 and cluster2.
+
+- Actual:
+ - Topic cannot be created at cluster2, and so the message1 can not be
replicated to cluster2.
+
+The root cause of these issues is that topics cannot access clusters that are
not included in the replication-clusters of the namespace policy.
+The replication clusters and allowed clusters are different in the definition,
and users don’t always want them to keep the same configuration.
+But in the current implementation, the replication-clusters and
allowed-Clusters are all configured by specifying the replication clusters.
+This will make the topic unable to have its replication policies.
+
+To support geo-replication policies at the topic level and the message level,
we must make the cluster configuration at the namespace level more clearly.
+Introduce `allowed-clusters` at the namespace level and make
`replication-clusters` only the default replication clusters for the topics
under the namespace.
+
+# Goals
+
+## In Scope
+
+The namespace will have a more detailed configuration for clusters. Users can
use `replication-clusters` and `allowed-clusters` to configure the different
behavioral logic of topics in the multiple clusters, rather than just using
`replication-clusters` to control all behaviors.
+
+## Out of Scope
+
+This proposal can be used to solve the problem of topic-level and message
level geo-replication can not work as expected. It is the initial motivation
for this proposal, but this proposal does not involve modifications to
geo-replication.
+
+# High Level Design
+A new namespace policy, `allowed_clusters`, should be added. The
`allowed_clusters` policy will specify the clusters where topics under this
namespace can be created or loaded. The `replication_cluster` indicates the
clusters that are used to create a full mesh replication for all topics under
this namespace.
+
+When a namespace has the policy with `allowed_clusters` and
`replication_clusters`, the topics under this namespace will replicate data to
all `replication_clusters` by default. Additionally, they can have their own
flexible replication configuration, which should be a subset of the
`allowed_clusters`.
+
+If `allowed_clusters` is not set, `replication_cluster` will be used as the
default value for `allowed_cluster`.
+
+If neither `allowed_clusters` nor `replication_cluster` are set, topics under
this namespace will only be able to publish/subscribe at the local cluster.
+
+Message-level replication is similar to topic-level replication. The
replication clusters of a message should be the subset of the
`allowed_cluster`, and are the `replication_clusters` configured at the topic
level or namespace level by default.
+
+# Detailed Design
+
+## Public-facing Changes
+
+### Public API
+
+#### `setNamespaceAllowedClusters` Endpoint
+
+This new endpoint allows setting the list of allowed clusters for a specific
namespace.
+
+**Method:**
+```
+POST
+```
+
+**Path:**
+```
+/namespaces/{tenant}/{namespace}/allowedclusters
+```
+
+**Query Parameters:**
+
+- `tenant`: The tenant within which the namespace resides.
+- `namespace`: The namespace for which you are setting the allowed clusters.
+
+**HTTP Body Parameters:**
+
+- `clusterIds`: A list of cluster IDs.
+
+**Response Codes:**
+
+- `400 Bad Request`: The list of allowed clusters should include all
replication clusters.
+- `403 Forbidden`: The requester does not have admin permissions.
+- `404 Not Found`: The specified tenant, cluster, or namespace does not exist.
+- `409 Conflict`: A peer-cluster cannot be part of an allowed-cluster.
+- `412 Precondition Failed`: The namespace is not global or the provided
cluster IDs are invalid.
+
+**Explanation for 409 Conflict:** This follows the behavior of namespace
replication clusters. As per PIP-8, a peer-cluster cannot be part of a
replication-cluster. Similarly, for allowed-clusters, users could enable
replication at the topic level, hence a peer-cluster cannot be part of
allowed-clusters as well.
+
+#### `getNamespaceAllowedClusters` Endpoint
+
+This new endpoint allows retrieving the list of allowed clusters for a
specific namespace.
+
+**Method:**
+```
+GET
+```
+
+**Path:**
+```
+/namespaces/{tenant}/{namespace}/allowedclusters
+```
+
+**Query Parameters:**
+
+- `tenant`: The tenant within which the namespace resides.
+- `namespace`: The namespace for which you are retrieving the allowed clusters.
+
+**Response Codes:**
+
+- `403 Forbidden`: The requester does not have admin permissions.
+- `404 Not Found`: The specified tenant, cluster, or namespace does not exist.
+- `412 Precondition Failed`: The namespace is not global.
+
+### Binary protocol
+
+### Configuration
+
+### CLI
Review Comment:
Oh I see.
We don't need to have methods like `createNamespace(String namespace,
Set<String> allowedClusters)` and `createNamespace(String namespace,
Set<String> clusters, Set<String> allowedClusters)` and deprecate the existing
method is out of the scope of this proposal.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
