Montura commented on PR #5758:
URL: https://github.com/apache/ozone/pull/5758#issuecomment-1882658248
> > Sometimes the number of under-utilized nodes may not be sufficient to
satisfy the limit about the max percent of datanodes participating in the
balance iteration (datanodes.involved.max.percentage.per.iteration). Thus,
collections of source and target datanodes are reset and balancing is skipped.
>
> I didn't really get this. Can you please elaborate? It'd be helpful to
have a small example where you describe this problem.
Let's imaging that you have a cluster with the total DNs number equals to
any value of [4, 9] (4 or 5 or 6 or 7 or 8 or 9).
Then the **maximum** value of DN's that could be involved in balancing by
default is 1, because default value for
**maxDatanodesRatioToInvolvePerIteration** is 0.2 (20 %). So in the next two
methods will skip balancing when DNs count is less then 10.
```
// ContainerBalancerTask#adaptWhenNearingIterationLimits
int maxDatanodesToInvolve =
config.getMaxDatanodesRatioToInvolvePerIteration() * totalNodesInCluster;
if (countDatanodesInvolvedPerIteration + 1 == maxDatanodesToInvolve) {
// Restricts potential target datanodes to nodes that have already been
selected
}
// ContainerBalancerTask#adaptOnReachingIterationLimits
int maxDatanodesToInvolve =
config.getMaxDatanodesRatioToInvolvePerIteration() * totalNodesInCluster;
if (countDatanodesInvolvedPerIteration == maxDatanodesToInvolve) {
// Restricts potential source and target datanodes to nodes that have
already been selected
}
// 4 * 0.2 = (0.8 -> cast_to_int) 0;
// 5 * 0.2 = (1.0 -> cast_to_int) 1;
// 6 * 0.2 = (1.2 -> cast_to_int) 1;
// 7 * 0.2 = (1.4 -> cast_to_int) 1;
// 8 * 0.2 = (1.6 -> cast_to_int) 1;
// 9 * 0.2 = (1.8 -> cast_to_int) 1;
```
By spec [java primitive narrowing
conversion](https://docs.oracle.com/javase/specs/jls/se10/html/jls-5.html#jls-5.1.3),
_the floating-point value is rounded to an integer value V, rounding toward
zero using IEEE 754 round-toward-zero mode
([ยง4.2.3](https://docs.oracle.com/javase/specs/jls/se10/html/jls-4.html#jls-4.2.3))_
So we got under-utilized nodes that will never take part in balancing at
all.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
