lokeshj1703 commented on a change in pull request #2808:
URL: https://github.com/apache/ozone/pull/2808#discussion_r751965087
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/FindTargetGreedy.java
##########
@@ -44,37 +46,64 @@
private ContainerManager containerManager;
private PlacementPolicy placementPolicy;
+ private Map<DatanodeDetails, Long> sizeEnteringNode;
+ private NodeManager nodeManager;
+ private ContainerBalancerConfiguration config;
+ private Double upperLimit;
+ private List<DatanodeUsageInfo> potentialTargets;
public FindTargetGreedy(
ContainerManager containerManager,
- PlacementPolicy placementPolicy) {
+ PlacementPolicy placementPolicy,
+ NodeManager nodeManager) {
+ sizeEnteringNode = new HashMap<>();
this.containerManager = containerManager;
this.placementPolicy = placementPolicy;
+ this.nodeManager = nodeManager;
+ }
+
+ private void setUpperLimit(Double upperLimit){
+ this.upperLimit = upperLimit;
+ }
+
+ private void setPotentialTargets(List<DatanodeUsageInfo> potentialTargets) {
+ this.potentialTargets = potentialTargets;
+ sizeEnteringNode.clear();
+ potentialTargets.forEach(
+ p -> sizeEnteringNode.put(p.getDatanodeDetails(), 0L));
+ }
+
+ private void setConfiguration(ContainerBalancerConfiguration conf) {
+ this.config = conf;
}
/**
* Find a {@link ContainerMoveSelection} consisting of a target and
* container to move for a source datanode. Favours more under-utilized
nodes.
* @param source Datanode to find a target for
- * @param potentialTargets Collection of potential target datanodes
* @param candidateContainers Set of candidate containers satisfying
* selection criteria
* {@link ContainerBalancerSelectionCriteria}
- * @param canSizeEnterTarget A functional interface whose apply
* (DatanodeDetails, Long) method returns true if the size specified in the
* second argument can enter the specified DatanodeDetails node
* @return Found target and container
*/
@Override
public ContainerMoveSelection findTargetForContainerMove(
- DatanodeDetails source, Collection<DatanodeDetails> potentialTargets,
- Set<ContainerID> candidateContainers,
- BiFunction<DatanodeDetails, Long, Boolean> canSizeEnterTarget) {
- for (DatanodeDetails target : potentialTargets) {
+ DatanodeDetails source, Set<ContainerID> candidateContainers) {
+ potentialTargets.sort((a, b) -> {
Review comment:
Let's do a priority queue implementation here as well.
##########
File path:
hadoop-hdds/server-scm/src/main/java/org/apache/hadoop/hdds/scm/container/balancer/ContainerBalancerConfiguration.java
##########
@@ -75,17 +73,15 @@
defaultValue = "", tags = {ConfigTag.BALANCER}, description = "The " +
"maximum size that can enter a target datanode in each " +
"iteration while balancing. This is the sum of data from multiple " +
- "sources. The default value is greater than the configured" +
- " (or default) ozone.scm.container.size by 1GB.")
- private long maxSizeEnteringTarget;
+ "sources. by default, we do not limit this value")
+ private long maxSizeEnteringTarget = Long.MAX_VALUE;
Review comment:
Let's revert the configuration changes here and do it in a separate PR.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
