[
https://issues.apache.org/jira/browse/HDDS-5526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sammi Chen resolved HDDS-5526.
------------------------------
Fix Version/s: 1.4.0
Resolution: Fixed
> ContainerBalancer#checkConditionsForBalancing pre-emptively checks iteration
> limits.
> ------------------------------------------------------------------------------------
>
> Key: HDDS-5526
> URL: https://issues.apache.org/jira/browse/HDDS-5526
> Project: Apache Ozone
> Issue Type: Sub-task
> Components: SCM
> Reporter: Siddhant Sangwan
> Assignee: Siddhant Sangwan
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.4.0
>
>
> ContainerBalancer#checkConditionsForBalancing is used to check whether the
> max size to move or max datanodes to involve limits have been reached.
> Currently, ContainerBalancer#doIteration uses this method before finding a
> ContainerMoveSelection for a source datanode. Since the check is performed
> before selecting the next container to move, the configuration
> OZONE_SCM_CONTAINER_SIZE (default is 5GB) is used to predetermine if:
> sizeMovedPerIteration + configured container size >=
> maxSizeToMovePerIteration.
> This check will cause the current iteration to stop in cases like:
> maxSizeToMovePerIteration = 100GB, sizeMovedPerIteration = 96GB, and
> OZONE_SCM_CONTAINER_SIZE = 5GB.
> On one hand the current implementation can avoid extra work by not letting
> balancer continue when we are very close to the limit. On the other hand,
> because of this implementation, we will never reach the limit. This Jira
> tracks if it's better to check conditions using container size in
> ContainerBalancer#checkConditionsForBalancing after having found a
> ContainerMoveSelection in ContainerBalancer#doIteration.
> Another solution is to allow one container move before checking the limit.
> The downside is that if the limits are really low (size lesser than 5GB,
> number of datanodes lesser than 2), we will fail to respect them.
> Another way is to check for container size _while_ picking a
> ContainerMoveSelection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
