clarax commented on a change in pull request #3356:
URL: https://github.com/apache/hbase/pull/3356#discussion_r657546722
##########
File path:
hbase-balancer/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java
##########
@@ -319,38 +318,33 @@ boolean needsBalance(TableName tableName,
BalancerClusterState cluster) {
return true;
}
+ sumMultiplier = 0.0f;
double total = 0.0;
- float sumMultiplier = 0.0f;
for (CostFunction c : costFunctions) {
float multiplier = c.getMultiplier();
- if (multiplier <= 0) {
- LOG.trace("{} not needed because multiplier is <= 0",
c.getClass().getSimpleName());
- continue;
- }
+ double cost = c.cost();
if (!c.isNeeded()) {
LOG.trace("{} not needed", c.getClass().getSimpleName());
continue;
}
+ total += cost * multiplier;
sumMultiplier += multiplier;
- total += c.cost() * multiplier;
}
- boolean balanced = total <= 0 || sumMultiplier <= 0 ||
- (sumMultiplier > 0 && (total / sumMultiplier) < minCostNeedBalance);
+ boolean balanced = (total / sumMultiplier < minCostNeedBalance);
+
if (balanced) {
final double calculatedTotal = total;
- final double calculatedMultiplier = sumMultiplier;
- sendRejectionReasonToRingBuffer(() -> getBalanceReason(calculatedTotal,
calculatedMultiplier),
- costFunctions);
- }
- if (LOG.isDebugEnabled()) {
- LOG.debug("{} {}; total cost={}, sum multiplier={}; cost/multiplier to
need a balance is {}",
- balanced ? "Skipping load balancing because balanced" : "We need to
load balance",
- isByTable ? String.format("table (%s)", tableName) : "cluster",
- total, sumMultiplier, minCostNeedBalance);
- if (LOG.isTraceEnabled()) {
- LOG.trace("Balance decision detailed function costs={}",
functionCost());
- }
+ sendRejectionReasonToRingBuffer(() -> getBalanceReason(calculatedTotal),
costFunctions);
+ LOG.info("{} - skipping load balancing because weighted average
imbalance={} > threshold({})."
+ + "functionCost={}."
+ + "If you want more aggressive balancing, either lower
minCostNeedbalance {}"
+ + "or increase the relative weight(s) of the specific cost
function(s).",
+ isByTable ? "Table specific ("+tableName+")" : "Cluster wide",
functionCost(),
+ total / sumMultiplier, minCostNeedBalance);
Review comment:
Updated log from test run: skipping load balancing because weighted
average imbalance=RegionCountSkewCostFunction : (multiplier=500.0,
imbalance=0.0, balanced); PrimaryRegionCountSkewCostFunction : (not needed);
MoveCostFunction : (multiplier=7.0, imbalance=0.0, balanced);
RackLocalityCostFunction : (multiplier=15.0, imbalance=0.0, balanced);
TableSkewCostFunction : (multiplier=35.0, imbalance=1.0);
RegionReplicaHostCostFunction : (not needed); RegionReplicaRackCostFunction :
(not needed); ReadRequestCostFunction : (multiplier=5.0, imbalance=0.0,
balanced); CPRequestCostFunction : (multiplier=5.0, imbalance=0.0, balanced);
WriteRequestCostFunction : (multiplier=5.0, imbalance=0.0, balanced);
MemStoreSizeCostFunction : (multiplier=5.0, imbalance=0.0, balanced);
StoreFileCostFunction : (multiplier=5.0, imbalance=0.0, balanced); >
threshold(0.06013745704467354).functionCost=1.0. If you want more aggressive
balancing, either lower hbase.master.balancer.stochastic.minCostNeedBala
nce from {}or increase the relative multiplier(s) of the specific cost
function(s).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
