anuragp010 commented on code in PR #12062:
URL: https://github.com/apache/ozone/pull/12062#discussion_r4624707957


##########
hadoop-hdds/docs/content/feature/ContainerBalancer.md:
##########
@@ -84,9 +84,29 @@ does not count as spare SSD space. Byte and datanode limits 
remain shared across
 Assessment warns when storage-type reports are incomplete, and estimation 
rejects those
 reports. When no storage-type breakdown is available, both commands use 
overall usage.
 
-These commands report and estimate balancing work. The running balancer still 
chooses
-moves using overall datanode usage; selecting moves separately for each 
storage type is
-separate work.
+### Balancing by storage type
+
+When complete storage-type usage reports are available, the running balancer 
selects
+sources and targets separately for each type. For example, a datanode at 80% 
SSD usage
+can send an SSD replica to another datanode at 20% SSD usage, even when both 
datanodes
+have the same overall utilization. Free DISK space is not a destination for 
that SSD copy.
+The selected type is included in the copy command so the destination uses a 
matching volume.
+
+Storage types take turns within each iteration, and the first type rotates 
between
+iterations. The iteration byte limit, each datanode's incoming and outgoing 
byte limits,
+and the distinct-datanode limit are shared across all types. A datanode that 
sends SSD
+replicas and receives DISK replicas counts once toward the datanode limit.
+
+For an existing replica without a recorded storage type, the balancer uses its 
reported
+volume type. It skips replicas whose type is unknown or whose recorded type 
conflicts

Review Comment:
   Nit: `whose type is unknown` is vague. Could this be rewritten as `It skips 
replicas whose recorded type and reported volume type are both unknown or whose 
recorded type conflicts with their reported volume type` ?



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerSelectionCriteria.java:
##########
@@ -426,6 +458,32 @@ Set<ContainerID> getExcludeNotFoundContainers() {
     return excludeContainersNotFound;
   }
 
+  /**
+   * Whether the container's replica on {@code node} sits on {@code 
storageType}.
+   * Unknown types and conflicting recorded/volume types cannot participate in 
a typed move.
+   */
+  private static boolean isReplicaOnStorageType(Set<ContainerReplica> replicas,
+      DatanodeDetails node, StorageType storageType) {
+    for (ContainerReplica replica : replicas) {
+      if (replica.getDatanodeDetails().equals(node)) {
+        if (replica.getStorageType() != null && replica.getVolumeStorageType() 
!= null
+            && replica.getStorageType() != replica.getVolumeStorageType()) {

Review Comment:
   The check for unknown and conflicting types is in multiple places. Can we 
extract this into a single method in `ContainerReplica` and use that ? Perhaps 
something like:
   ```java
    /** True when the recorded type and the reported volume type are both known 
and differ. */
    public boolean hasConflictingStorageTypes() {
      return storageType != null && volumeStorageType != null && storageType != 
volumeStorageType;
    }
   ```



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/ContainerReplica.java:
##########
@@ -150,6 +150,14 @@ public StorageType getVolumeStorageType() {
     return volumeStorageType;
   }
 
+  /**
+   * Returns the recorded replica type, or its reported volume type when the 
recorded type is absent.
+   * A null result means neither type is known. Callers must handle 
conflicting known types separately.
+   */
+  public StorageType getTargetStorageTypeForCopy() {

Review Comment:
   Nit: `getTargetStorageTypeForCopy()` returns the type of this replica: the 
recorded type, or the volume type if no type is recorded. This actually 
represents the effective storage type of the existing replica. Would 
`getEffectiveStorageType()` be a better method name here?



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java:
##########
@@ -557,56 +560,45 @@ private boolean initializeIteration() {
 
     this.totalNodesInCluster = datanodeUsageInfos.size();
 
-    double clusterAvgUtilisation = calculateAvgUtilization(datanodeUsageInfos);
-    if (LOG.isDebugEnabled()) {
-      LOG.debug("Average utilization of the cluster is {}", 
clusterAvgUtilisation);
+    // Node include/exclude filters have already been applied above.
+    ContainerBalancerClusterSnapshot analysis = 
ContainerBalancerClusterAnalyzer.analyze(datanodeUsageInfos.stream()
+        .map(node -> 
node.toProto(ClientVersion.DEFAULT_VERSION)).collect(Collectors.toList()),
+        config.getThresholdAsRatio(), Collections.emptySet(), 
Collections.emptySet());
+    if (analysis.hasIncompleteStorageTypeUsage()) {
+      lastInitializationFailureDetail = "Incomplete storage-type usage 
reports; cannot safely select typed moves";
+      LOG.warn(lastInitializationFailureDetail);
+      return false;
     }
-
-    double threshold = config.getThresholdAsRatio();
-    // over utilized nodes have utilization(that is, used / capacity) greater 
than upper limit
-    this.upperLimit = ContainerBalancerClusterAnalyzer.computeUpperLimit(
-        clusterAvgUtilisation, threshold);
-    // under utilized nodes have utilization(that is, used / capacity) less 
than lower limit
-    this.lowerLimit = ContainerBalancerClusterAnalyzer.computeLowerLimit(
-        clusterAvgUtilisation, threshold);
-
-    if (LOG.isDebugEnabled()) {
-      LOG.debug("Lower limit for utilization is {} and Upper limit for 
utilization is {}", lowerLimit, upperLimit);
+    if (analysis.getStorageTypeSnapshots().isEmpty()) {
+      storageTypeStates.add(new StorageTypeBalancingState(null, 
analysis.getLowerLimit(), analysis.getUpperLimit(),
+          datanodeUsageInfos, analysis.getSourceDatanodes(), 
analysis.getTargetDatanodes()));
+    } else {
+      analysis.getStorageTypeSnapshots().forEach((type, snapshot) ->
+          storageTypeStates.add(new StorageTypeBalancingState(type, 
snapshot.getLowerLimit(), snapshot.getUpperLimit(),
+              datanodeUsageInfos, snapshot.getSourceDatanodes(), 
snapshot.getTargetDatanodes())));
     }
-
-    long totalOverUtilizedBytes = 0L, totalUnderUtilizedBytes = 0L;
-    // find over and under utilized nodes
-    for (DatanodeUsageInfo datanodeUsageInfo : datanodeUsageInfos) {
+    Set<DatanodeDetails> sources = new HashSet<>();
+    Set<DatanodeDetails> targets = new HashSet<>();

Review Comment:
   I want to check my understanding here. `sources` and `targets` are built 
from every `StorageTypeBalancingState`. That includes a type whose snapshot has 
sources but no targets, or targets but no sources. For the combined 
`analysis.getSourceDatanodes()` and `getTargetDatanodes()`, 
`combineStorageTypes` only adds a type when `snapshot.getBytesToMove() > 0`.
   
   So `numDatanodesUnbalanced` and `overUtilizedNodes` can include a node that 
no typed move can use. Is that what is intended? 



##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerMetrics.java:
##########
@@ -34,6 +34,9 @@ public final class ContainerBalancerMetrics {
 
   private final MetricsSystem ms;
 
+  @Metric("Number of times scheduling reached the shared iteration byte limit")
+  private MutableCounterLong numIterationLimitsSkipped;

Review Comment:
   Since this tracks the number of times an iteration of the container balancer 
stops because the `maxSizeToMovePerIteration` limit is reached, should we 
rename this to `numIterationsReachedMaxSizeToMoveLimit` or 
`numIterationsReachedMaxSizeToMove` ?



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to