jiajunwang commented on a change in pull request #639: Refine the WAGED
rebalancer to minimize the partial rebalance workload.
URL: https://github.com/apache/helix/pull/639#discussion_r352902854
##########
File path:
helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/model/ClusterModelProvider.java
##########
@@ -159,33 +337,29 @@ public static ClusterModel
generateClusterModelFromCurrentState(
.getOrDefault(HelixConstants.ChangeType.RESOURCE_CONFIG,
Collections.emptySet())
.contains(resourceName) || clusterChanges
.getOrDefault(HelixConstants.ChangeType.IDEAL_STATE,
Collections.emptySet())
- .contains(resourceName) ||
!bestPossibleAssignment.containsKey(resourceName)) {
+ .contains(resourceName) ||
!currentAssignment.containsKey(resourceName)) {
toBeAssignedReplicas.addAll(replicas);
continue; // go to check next resource
} else {
// check for every best possible assignments to identify if the
related replicas need to reassign.
- ResourceAssignment assignment =
bestPossibleAssignment.get(resourceName);
- // <partition, <instance, state>>
- Map<String, Map<String, String>> stateMap =
assignment.getMappedPartitions().stream()
- .collect(Collectors.toMap(partition ->
partition.getPartitionName(),
- partition -> new
HashMap<>(assignment.getReplicaMap(partition))));
+ // <partition, <state, instances list>>
+ Map<String, Map<String, Set<String>>> stateMap =
+ getValidStateInstanceMap(currentAssignment.get(resourceName),
activeInstances);
for (AssignableReplica replica : replicas) {
// Find any ACTIVE instance allocation that has the same state
with the replica
- Optional<Map.Entry<String, String>> instanceNameOptional =
- stateMap.getOrDefault(replica.getPartitionName(),
Collections.emptyMap()).entrySet()
- .stream().filter(instanceStateMap ->
-
instanceStateMap.getValue().equals(replica.getReplicaState()) && activeInstances
- .contains(instanceStateMap.getKey())).findAny();
- // 3. if no such an instance in the bestPossible assignment, need
to reassign the replica
- if (!instanceNameOptional.isPresent()) {
+ Set<String> validInstances =
+ stateMap.getOrDefault(replica.getPartitionName(),
Collections.emptyMap())
+ .getOrDefault(replica.getReplicaState(),
Collections.emptySet());
+ if (validInstances.isEmpty()) {
+ // 3. if no such an instance in the bestPossible assignment,
need to reassign the replica
toBeAssignedReplicas.add(replica);
continue; // go to check the next replica
} else {
- String instanceName = instanceNameOptional.get().getKey();
- // * cleanup the best possible state map record,
+ Iterator<String> iter = validInstances.iterator();
+ // * cleanup the best possible assignment allocation record
after one is picked up.
Review comment:
I truly think that won't be necessary. The usage code is 2 lines away. It is
very clear.
A long description is also a burden to read and maintain.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
