smengcl commented on code in PR #11399:
URL: https://github.com/apache/ozone/pull/11399#discussion_r4280558935
##########
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerTask.java:
##########
@@ -896,8 +935,11 @@ private ContainerMoveSelection
matchSourceWithTarget(DatanodeDetails source) {
Set<ContainerID> toRemoveContainerIds = new HashSet<>();
for (ContainerID containerId: sourceContainerIDSet) {
if (selectionCriteria.shouldBeExcluded(containerId, source,
- sizeScheduledForMoveInLatestIteration)) {
- toRemoveContainerIds.add(containerId);
+ sizeScheduledForMoveInLatestIteration, storageType)) {
+ // A different tier may still consider this container with its own
utilization bounds.
+ if (storageType == null ||
containerToSourceMap.containsKey(containerId)) {
+ toRemoveContainerIds.add(containerId);
+ }
Review Comment:
Keep rejected candidates out of later searches on the same storage type.
With `storageType != null`, rejected containers remain in the shared
candidate set unless they have already been selected for a move. This includes
open containers and replicas on other tiers, so every later move from the same
source scans them again. A matcher probe with 200 open containers and 100 moves
performed 20,000 rejected-replica lookups, compared with 200 through the legacy
path. The mixed-tier case showed the same repeated work.
Please cache candidates per source and storage type, removing rejected
candidates from that type's set while retaining them for other types. Otherwise
scheduling work grows as rejected containers multiplied by moves.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
