[
https://issues.apache.org/jira/browse/HBASE-13965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14606864#comment-14606864
]
Lei Chen commented on HBASE-13965:
----------------------------------
Sorry for the delay. I propose the following changes which can be seen
illustrated in the attached class diagram.
The {{StochasticLoadBalancer}} extends {{BaseLoadBalancer}} which has a class
variable of type {{MetricsBalancer}}. The {{MetricsBalancer}} contains a
private class variable of type {{MetricsBalancerSource}}. The
{{MetricsBalancerSource}} is an interface which defines which metrics can be
reported to JMX. This proves to make extension difficult for load balancer
implementation specific metrics (e.g. the {{StochasticLoadBalancer}}).
Adding metrics to the generic interface is not appropriate being it is used by
all load balancers and should not contain any load balancer specific metrics. I
propose to create a class extending {{MetricsBalancer}} to provide specific
load balancer metrics. To use this class, I propose to add a constructor to
{{BaseLoadBalancer}} which allows for the balancer instance metrics class to be
passed in. (Thanks [~enis] for code review and giving the constructor
suggestion!)
In the constructor of {{StochasticLoadBalancer}}, an instance of
{{MetricsStochasticBalancer}} is created and passed to a new constructor added
to {{BaseLoadBalancer}}, which will use it to replace the default
{{MetricsBalancer}}. The function used to add metrics is declared as following:
{code}
public void updateStochasticCost(String tableName, String costFunctionName,
String costFunctionDesc, Double value);
{code}
In {{MetricsBalancer}}, the {{private final}} class variable {{source}} was
previously hardcoded and instantiated in its constructor; I propose a new
function {{initSource}} which can be overridden to set this variable. As such,
in the subclass {{MetricsStochasticBalancer}}, {{initSource}} will create a
{{MetricsStochasticBalancerSource}} instance instead of the default
{{MetricsBalancerSource}}.
Finally, to give good insight to the internal status of
{{StochasticLoadBalancer}}, we are considering adding metrics for each cost
function, as well as the overall cost. For example, if the balancing is carried
out for table "MyTable1" and the {{StochasticLoadBalancer}} has 3 cost
functions "MoveCost", "LocalityCost", and "RegionReplicaHostCost", then 4
metrics will be added to “HBase -> Master -> Balancer” as following:
MyTable1_Overall
MyTable1_MoveCost
MyTable1_LocalityCost
MyTable1_RegionReplicaHostCost
I'm building the patch, any suggestion is appreciated.
> Stochastic Load Balancer JMX Metrics
> ------------------------------------
>
> Key: HBASE-13965
> URL: https://issues.apache.org/jira/browse/HBASE-13965
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, metrics
> Reporter: Lei Chen
> Assignee: Lei Chen
>
> Today’s default HBase load balancer (the Stochastic load balancer) is cost
> function based. The cost function weights are tunable but no visibility into
> those cost function results is directly provided.
> A driving example is a cluster we have been tuning which has skewed rack size
> (one rack has half the nodes of the other few racks). We are tuning the
> cluster for uniform response time from all region servers with the ability to
> tolerate a rack failure. Balancing LocalityCost, RegionReplicaRack Cost and
> RegionCountSkew Cost is difficult without a way to attribute each cost
> function’s contribution to overall cost.
> What this jira proposes is to provide visibility via JMX into each cost
> function of the stochastic load balancer, as well as the overall cost of the
> balancing plan.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
