[ 
https://issues.apache.org/jira/browse/HELIX-674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16397846#comment-16397846
 ] 

ASF GitHub Bot commented on HELIX-674:
--------------------------------------

Github user mcvsubbu commented on a diff in the pull request:

    https://github.com/apache/helix/pull/145#discussion_r174313827
  
    --- Diff: 
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
    @@ -0,0 +1,141 @@
    +package org.apache.helix.util;
    +
    +import org.apache.helix.HelixException;
    +import org.apache.helix.ZNRecord;
    +import org.apache.helix.api.config.RebalanceConfig;
    +import 
org.apache.helix.api.rebalancer.constraint.AbstractRebalanceHardConstraint;
    +import 
org.apache.helix.api.rebalancer.constraint.AbstractRebalanceSoftConstraint;
    +import org.apache.helix.controller.common.PartitionStateMap;
    +import org.apache.helix.controller.common.ResourcesStateMap;
    +import 
org.apache.helix.controller.rebalancer.strategy.ConstraintRebalanceStrategy;
    +import org.apache.helix.controller.stages.ClusterDataCache;
    +import org.apache.helix.model.*;
    +
    +import java.util.ArrayList;
    +import java.util.HashMap;
    +import java.util.List;
    +import java.util.Map;
    +
    +/**
    + * A rebalance tool that generate an resource partition assignment based 
on the input.
    + * Note the assignment won't be automatically applied to the cluster. 
Applications are supposed to
    + * apply the change.
    + */
    +public class WeightAwareRebalanceUtil {
    +  private final ClusterConfig _clusterConfig;
    +  private final Map<String, InstanceConfig> _instanceConfigMap = new 
HashMap<>();
    +  // For the possible customized state models.
    +  private final Map<String, StateModelDefinition> _stateModelDefs = new 
HashMap<>();
    +  private final ClusterDataCache _dataCache;
    +
    +  public enum RebalanceOption {
    +    INIT,
    +    REASSIGN
    +  }
    +
    +  public WeightAwareRebalanceUtil(ClusterConfig clusterConfig,
    +      List<InstanceConfig> instanceConfigs) {
    +    for (InstanceConfig instanceConfig : instanceConfigs) {
    +      _instanceConfigMap.put(instanceConfig.getInstanceName(), 
instanceConfig);
    +    }
    +    _clusterConfig = clusterConfig;
    +
    +    _dataCache = new ClusterDataCache();
    +    _dataCache.setInstanceConfigMap(_instanceConfigMap);
    +    _dataCache.setClusterConfig(_clusterConfig);
    +    List<LiveInstance> liveInstanceList = new ArrayList<>();
    +    for (String instance : _instanceConfigMap.keySet()) {
    +      LiveInstance liveInstance = new LiveInstance(instance);
    +      liveInstanceList.add(liveInstance);
    +    }
    +    _dataCache.setLiveInstances(liveInstanceList);
    +  }
    +
    +  /**
    +   * The method to generate partition assignment mappings.
    +   *
    +   * @param resourceConfigs    Config of all the resources that need to be 
rebalanced.
    +   *                           The tool throws Exception if any resource 
has no IS or broken/uninitialized IS.
    +   *                           The tool throws Exception if any resource 
is in full-auto mode.
    +   * @param existingAssignment The existing partition assignment of the 
input resources.
    --- End diff --
    
    s/input resources/resources specified in param resourceConfigs/


> Constraint Based Resource Rebalancer
> ------------------------------------
>
>                 Key: HELIX-674
>                 URL: https://issues.apache.org/jira/browse/HELIX-674
>             Project: Apache Helix
>          Issue Type: New Feature
>            Reporter: Jiajun Wang
>            Assignee: Jiajun Wang
>            Priority: Major
>             Fix For: 0.8.x
>
>         Attachments: Constraint-BasedResourceRebalancing-080318-2226-240.pdf
>
>
> Helix rebalancer assigns resources according to different strategies. 
> Recently, we optimize the strategy for evenness and minimize movement. 
> However, the evenness here only applies to partition numbers. Moreover, we've 
> got more requests for customizable rebalancer from our users.
> Take partition weight as an example:
> In reality, partition replicas have different size. We use "partition weight" 
> as an abstraction of the partition size. It can be network traffic usage, 
> disk usage, or any other combined factors.
> Given each partition may have different weights, Helix should be able to 
> assign partition accordingly. So that the distribution would be even 
> regarding the weight.
> In this project, we are planning new rebalancer mechanism that generates 
> resource partition assignment according to a list of "constraints". Current 
> rebalance strategy can be regarded as one kind of constraint. Moving forward, 
> Helix users would be able to extend the constraint interface using their own 
> logic.
> Some init discussions are in progress and we will have a proposal posted here 
> soon.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Reply via email to