JacksonYao287 commented on code in PR #3895:
URL: https://github.com/apache/ozone/pull/3895#discussion_r1024179113
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java:
##########
@@ -663,36 +678,68 @@ private ContainerMoveSelection
matchSourceWithTarget(DatanodeDetails source) {
return moveSelection;
}
+ private boolean reachedMaxSizeToMovePerIteration() {
+ // since candidate containers in ContainerBalancerSelectionCriteria are
+ // filtered out according to this limit, balancer should not have crossed
it
+ if (sizeScheduledForMoveInLatestIteration >= maxSizeToMovePerIteration) {
+ LOG.warn("Reached max size to move limit. {} bytes have already been" +
+ " scheduled for balancing and the limit is {} bytes.",
+ sizeScheduledForMoveInLatestIteration, maxSizeToMovePerIteration);
+ return true;
+ }
+ return false;
+ }
+
/**
- * Checks if limits maxDatanodesPercentageToInvolvePerIteration and
- * maxSizeToMovePerIteration have been hit.
- *
- * @return true if a limit was hit, else false
+ * Restricts potential target datanodes to nodes that have
+ * already been selected if balancer is one datanode away from
+ * "datanodes.involved.max.percentage.per.iteration" limit.
+ * @return true if potential targets were restricted, else false
*/
- private boolean checkIterationLimits() {
+ private boolean adaptWhenNearingIterationLimits() {
+ // check if we're nearing max datanodes to involve
int maxDatanodesToInvolve =
(int) (maxDatanodesRatioToInvolvePerIteration * totalNodesInCluster);
- if (countDatanodesInvolvedPerIteration + 2 > maxDatanodesToInvolve) {
- if (LOG.isDebugEnabled()) {
- LOG.debug("Hit max datanodes to involve limit. {} datanodes have" +
- " already been scheduled for balancing and the limit is {}.",
- countDatanodesInvolvedPerIteration, maxDatanodesToInvolve);
- }
+ if (countDatanodesInvolvedPerIteration + 1 == maxDatanodesToInvolve) {
+ /* We're one datanode away from reaching the limit. Restrict potential
+ targets to targets that have already been selected.
+ */
+ findTargetStrategy.resetPotentialTargets(selectedTargets);
Review Comment:
if we reach here, i think we can restrict any of potential targets or
sources, as long as we make sure we will involve at most one more datanode.
for example , when all the selected targets can not be a candidate target
again(they may be overutilized after receiving a container replica) , we can
restrict potential sources to sources that have already been selected, so that
we may select a new target which will not be overutilized after receiving a
container replica, meanwhile we involve only one more datanode in this balance
iteration.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
