clarax commented on a change in pull request #3356:
URL: https://github.com/apache/hbase/pull/3356#discussion_r657546722
##########
File path:
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##########
@@ -319,38 +318,33 @@ boolean needsBalance(TableName tableName,
BalancerClusterState cluster) {
return true;
}
+ sumMultiplier = 0.0f;
double total = 0.0;
- float sumMultiplier = 0.0f;
for (CostFunction c : costFunctions) {
float multiplier = c.getMultiplier();
- if (multiplier <= 0) {
- LOG.trace("{} not needed because multiplier is <= 0",
c.getClass().getSimpleName());
- continue;
- }
+ double cost = c.cost();
if (!c.isNeeded()) {
LOG.trace("{} not needed", c.getClass().getSimpleName());
continue;
}
+ total += cost * multiplier;
sumMultiplier += multiplier;
- total += c.cost() * multiplier;
}
- boolean balanced = total <= 0 || sumMultiplier <= 0 ||
- (sumMultiplier > 0 && (total / sumMultiplier) < minCostNeedBalance);
+ boolean balanced = (total / sumMultiplier < minCostNeedBalance);
+
if (balanced) {
final double calculatedTotal = total;
- final double calculatedMultiplier = sumMultiplier;
- sendRejectionReasonToRingBuffer(() -> getBalanceReason(calculatedTotal,
calculatedMultiplier),
- costFunctions);
- }
- if (LOG.isDebugEnabled()) {
- LOG.debug("{} {}; total cost={}, sum multiplier={}; cost/multiplier to
need a balance is {}",
- balanced ? "Skipping load balancing because balanced" : "We need to
load balance",
- isByTable ? String.format("table (%s)", tableName) : "cluster",
- total, sumMultiplier, minCostNeedBalance);
- if (LOG.isTraceEnabled()) {
- LOG.trace("Balance decision detailed function costs={}",
functionCost());
- }
+ sendRejectionReasonToRingBuffer(() -> getBalanceReason(calculatedTotal),
costFunctions);
+ LOG.info("{} - skipping load balancing because weighted average
imbalance={} > threshold({})."
+ + "functionCost={}."
+ + "If you want more aggressive balancing, either lower
minCostNeedbalance {}"
+ + "or increase the relative weight(s) of the specific cost
function(s).",
+ isByTable ? "Table specific ("+tableName+")" : "Cluster wide",
functionCost(),
+ total / sumMultiplier, minCostNeedBalance);
Review comment:
Updated log from test run: Cluster wide - skipping load balancing
because weighted average imbalance=0.02843086355657742 <= threshold(0.05). If
you want more aggressive balancing, either lower
hbase.master.balancer.stochastic.minCostNeedBalance from 0.05 or increase the
relative multiplier(s) of the specific cost function(s).
functionCost=RegionCountSkewCostFunction : (multiplier=500.0, imbalance=0.0,
balanced); PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction
: (multiplier=7.0, imbalance=0.0, balanced); RackLocalityCostFunction :
(multiplier=15.0, imbalance=0.0, balanced); TableSkewCostFunction :
(multiplier=35.0, imbalance=0.4727646454265159); RegionReplicaHostCostFunction
: (not needed); RegionReplicaRackCostFunction : (not needed);
ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0, balanced);
CPRequestCostFunction : (multiplier=5.0, imbalance=0.0, balanced);
WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0, balanced);
MemStoreSizeCostFun
ction : (multiplier=5.0, imbalance=0.0, balanced); StoreFileCostFunction :
(multiplier=5.0, imbalance=0.0, balanced);
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
