[
https://issues.apache.org/jira/browse/HBASE-23949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
niuyulin updated HBASE-23949:
-----------------------------
Description:
now can not achieve overallbalanced when use rsgroup balancer and by table is
on,
because balance every table actually use the clusterload only contain one
table's load.
we should use clusterload contain all this rsgroup table's load to balance
overall
hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
{code:java}
public boolean balance(boolean force) throws IOException {
......
boolean isByTable =
getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
this.assignmentManager.getRegionStates()
.getAssignmentsForBalancer(tableStateManager,
this.serverManager.getOnlineServersList(),
isByTable);
for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
}
//Give the balancer the current cluster state.
this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
this.balancer.setClusterLoad(assignments);
List<RegionPlan> plans = new ArrayList<>();
for (Entry<TableName, Map<ServerName, List<RegionInfo>>> e :
assignments.entrySet()) {
List<RegionPlan> partialPlans = this.balancer.balanceCluster(e.getKey(),
e.getValue());
if (partialPlans != null) {
plans.addAll(partialPlans);
}
}
{code}
now do refactor:
# add method 'balanceTable' in interface LoadBalancer
# SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' ,
and 'balanceTable' is not support in BaseLoadBalancer and
RSGroupBasedLoadBalancer
# RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad
to internal balacer by group
# internal balancer balance cluster invoke 'balanceTable'
was:
use rsgroup balancer and by table config is on ,
now balance every table actually use the clusterload only contain one table's
load.
we should use clusterload contain all this rsgroup table's load to balance
overall
hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
{code:java}
public boolean balance(boolean force) throws IOException {
......
boolean isByTable =
getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
this.assignmentManager.getRegionStates()
.getAssignmentsForBalancer(tableStateManager,
this.serverManager.getOnlineServersList(),
isByTable);
for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
}
//Give the balancer the current cluster state.
this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
this.balancer.setClusterLoad(assignments);
List<RegionPlan> plans = new ArrayList<>();
for (Entry<TableName, Map<ServerName, List<RegionInfo>>> e :
assignments.entrySet()) {
List<RegionPlan> partialPlans = this.balancer.balanceCluster(e.getKey(),
e.getValue());
if (partialPlans != null) {
plans.addAll(partialPlans);
}
}
{code}
now do refactor:
# add method 'balanceTable' in interface LoadBalancer
# SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' ,
and 'balanceTable' is not support in BaseLoadBalancer and
RSGroupBasedLoadBalancer
# RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad
to internal balacer by group
# internal balancer balance cluster invoke 'balanceTable'
> refactor loadBalancer implements for rsgroup balance by table
> --------------------------------------------------------------
>
> Key: HBASE-23949
> URL: https://issues.apache.org/jira/browse/HBASE-23949
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
> Affects Versions: 2.2.0
> Reporter: niuyulin
> Assignee: niuyulin
> Priority: Major
>
> now can not achieve overallbalanced when use rsgroup balancer and by table
> is on,
> because balance every table actually use the clusterload only contain one
> table's load.
> we should use clusterload contain all this rsgroup table's load to balance
> overall
>
> hbase/hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
> {code:java}
> public boolean balance(boolean force) throws IOException {
> ......
> boolean isByTable =
> getConfiguration().getBoolean("hbase.master.loadbalance.bytable", false);
> Map<TableName, Map<ServerName, List<RegionInfo>>> assignments =
> this.assignmentManager.getRegionStates()
> .getAssignmentsForBalancer(tableStateManager,
> this.serverManager.getOnlineServersList(),
> isByTable);
> for (Map<ServerName, List<RegionInfo>> serverMap : assignments.values()) {
>
> serverMap.keySet().removeAll(this.serverManager.getDrainingServersList());
> }
> //Give the balancer the current cluster state.
> this.balancer.setClusterMetrics(getClusterMetricsWithoutCoprocessor());
> this.balancer.setClusterLoad(assignments);
> List<RegionPlan> plans = new ArrayList<>();
> for (Entry<TableName, Map<ServerName, List<RegionInfo>>> e :
> assignments.entrySet()) {
> List<RegionPlan> partialPlans =
> this.balancer.balanceCluster(e.getKey(), e.getValue());
> if (partialPlans != null) {
> plans.addAll(partialPlans);
> }
> }
> {code}
> now do refactor:
> # add method 'balanceTable' in interface LoadBalancer
> # SimpleLoadBalancer and StochasticLoadBalancer do the real 'balanceTable' ,
> and 'balanceTable' is not support in BaseLoadBalancer and
> RSGroupBasedLoadBalancer
> # RSGroupBasedLoadBalancer invoke balanceCluster , and pass GroupClusterLoad
> to internal balacer by group
> # internal balancer balance cluster invoke 'balanceTable'
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
