szetszwo commented on code in PR #9580:
URL: https://github.com/apache/ozone/pull/9580#discussion_r2669180459
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
+ across the entire cluster. This acts as a final cap on the total
number of pipelines.
+ * **Default Value**: `0` (which means no global limit is enforced by
default).
Review Comment:
> ... no global limit is enforced by default).
no global limit by default).
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
+ across the entire cluster. This acts as a final cap on the total
number of pipelines.
+ * **Default Value**: `0` (which means no global limit is enforced by
default).
+
+2. **Datanode-level Fixed Limit (`ozone.scm.datanode.pipeline.limit`)**
+ * **Description**: When set to a positive number, this property defines
a fixed maximum number of pipelines for
+ every datanode. This is one of two ways to calculate a cluster-wide
target.
+ * **Default Value**: `2`
Review Comment:
This is tricky since we have a bug; see HDDS-14369.
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
Review Comment:
> ... FACTOR_THREE ...
Let's use
[ReplicationFactor.THREE](https://github.com/apache/ozone/blob/fd181c69464afd3ed3240b376b81f7b81f0806e9/hadoop-hdds/common/src/main/java/org/apache/hadoop/hdds/client/ReplicationFactor.java#L27),
which is a client API.
> ... three properties ...
three configuration properties
> ... that define the maximum
that limit the
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
Review Comment:
> ... a cluster-wide level, datanode level, and metadata disk level,
respectively.
a cluster-wide level and a datanode level.
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
Review Comment:
Replace this sentence with
- "The number of pipelines created by SCM is restricted by these limits."
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
+ across the entire cluster. This acts as a final cap on the total
number of pipelines.
+ * **Default Value**: `0` (which means no global limit is enforced by
default).
+
+2. **Datanode-level Fixed Limit (`ozone.scm.datanode.pipeline.limit`)**
+ * **Description**: When set to a positive number, this property defines
a fixed maximum number of pipelines for
+ every datanode. This is one of two ways to calculate a cluster-wide
target.
+ * **Default Value**: `2`
+ * **Calculation**: If this is set, the target is `(<this value> *
<number of healthy datanodes>) / 3`.
Review Comment:
```md
* **Cluster-wide Limit Calculation**: If this is set, the cluster-wide
limit is
`(<this value> * <number of healthy datanodes>) / 3`.
```
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
+ across the entire cluster. This acts as a final cap on the total
number of pipelines.
+ * **Default Value**: `0` (which means no global limit is enforced by
default).
+
+2. **Datanode-level Fixed Limit (`ozone.scm.datanode.pipeline.limit`)**
+ * **Description**: When set to a positive number, this property defines
a fixed maximum number of pipelines for
+ every datanode. This is one of two ways to calculate a cluster-wide
target.
+ * **Default Value**: `2`
+ * **Calculation**: If this is set, the target is `(<this value> *
<number of healthy datanodes>) / 3`.
+
+3. **Datanode-level Dynamic Limit (`ozone.scm.pipeline.per.metadata.disk`)**
+ * **Description**: This property is used only when
`ozone.scm.datanode.pipeline.limit` is explicitly set to `0`.
Review Comment:
> ... This property is used ... is explicitly set to `0`.
This property takes effect ... is not set to a positive number.
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
+ across the entire cluster. This acts as a final cap on the total
number of pipelines.
+ * **Default Value**: `0` (which means no global limit is enforced by
default).
+
+2. **Datanode-level Fixed Limit (`ozone.scm.datanode.pipeline.limit`)**
+ * **Description**: When set to a positive number, this property defines
a fixed maximum number of pipelines for
+ every datanode. This is one of two ways to calculate a cluster-wide
target.
+ * **Default Value**: `2`
+ * **Calculation**: If this is set, the target is `(<this value> *
<number of healthy datanodes>) / 3`.
+
+3. **Datanode-level Dynamic Limit (`ozone.scm.pipeline.per.metadata.disk`)**
+ * **Description**: This property is used only when
`ozone.scm.datanode.pipeline.limit` is explicitly set to `0`.
+ It calculates a dynamic limit for each datanode based on its available
metadata disks.
+ * **Default Value**: `2`
+ * **Calculation**: The limit for each datanode is
+ `(<this value> * <number of metadata disks on that datanode>)`.
+ The total cluster-wide target is the sum of all individual datanode
limits, divided by 3.
Review Comment:
This seems not true (or I cannot find the code enforcing it). I do see that
the value is used for filtering datanodes.
https://github.com/apache/ozone/blob/4fd83602c3080a8174876bc9c19c356af857fd56/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java#L104-L131
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
+ across the entire cluster. This acts as a final cap on the total
number of pipelines.
+ * **Default Value**: `0` (which means no global limit is enforced by
default).
+
+2. **Datanode-level Fixed Limit (`ozone.scm.datanode.pipeline.limit`)**
+ * **Description**: When set to a positive number, this property defines
a fixed maximum number of pipelines for
+ every datanode. This is one of two ways to calculate a cluster-wide
target.
Review Comment:
Remove the sentence "This is one of two ways to calculate a cluster-wide
target." since the Dynamic Limit
is not used in RatisPipelineProvider.exceedPipelineNumberLimit(..).
##########
hadoop-hdds/docs/content/feature/multi-raft-support.md:
##########
@@ -69,6 +69,64 @@ Ratis handles concurrent logs per node.
This property is effective only when the previous property is set to 0.
The value of this property must be greater than 0.
+### Calculating Ratis Pipeline Limits
+
+The target number of open, FACTOR_THREE Ratis pipelines is controlled by three
properties that define the maximum
+number of pipelines in the cluster at a cluster-wide level, datanode level,
and metadata disk level, respectively.
+SCM will create pipelines until the most restrictive limit is met.
+
+1. **Cluster-wide Limit (`ozone.scm.ratis.pipeline.limit`)**
+ * **Description**: An absolute, global limit for the total number of
open, FACTOR_THREE Ratis pipelines
Review Comment:
Remove FACTOR_THREE since the code actually includes ReplicationFactor.ONE
pipeline; see
https://github.com/apache/ozone/blob/4fd83602c3080a8174876bc9c19c356af857fd56/hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/pipeline/RatisPipelineProvider.java#L123-L128
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
