sodonnel commented on code in PR #5726:
URL: https://github.com/apache/ozone/pull/5726#discussion_r1424125045
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/SCMCommonPlacementPolicy.java:
##########
@@ -445,6 +445,7 @@ public ContainerPlacementStatus validateContainerPlacement(
}
}
List<Integer> currentRackCount = new ArrayList<>(dns.stream()
+ .filter(d -> !(d.isDecommissioned()))
Review Comment:
Again here, I wonder about maintenance replicas. It would be possible for
there to be extra copies of maintenance replicas in the cluster, like with
decommission, and until the node goes offline, there would be too many replicas.
The problem here, is not the number of racks, but that we are violating the
"max nodes per rack". The reason is that we pass in the expected number of
nodes (repFactor) and the list of replicas, and the list of replicas is greater
than the expected number.
In this case, we pass in 4 replicas (3xin_service 1xdecommissioning) and say
we expect 3 (repFactor).
Then the calculation of max nodes per rack says:
```
protected int getMaxReplicasPerRack(int numReplicas, int numberOfRacks) {
return numReplicas / numberOfRacks
+ Math.min(numReplicas % numberOfRacks, 1);
}
```
So if we have 3 replicas, we get:
```
3 / 2 + min( 3 % 2)=1, 1), so 1 + 1 = 2.
```
But as the container is effectively over-replicated, we have to adjust the
max nodes per rack upwards by the delta between the actual replica count and
expected replicas.
So if we add to that max per rack Math.max(0, dns.size() - replicas) = 1,
then we get 3 allowed per rack, which means the policy would not be violated
any more.
Do you think that change would fix it in the general case for any type of
excess replica count?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
