[
https://issues.apache.org/jira/browse/HBASE-26023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17367770#comment-17367770
]
Clara Xiong commented on HBASE-26023:
-------------------------------------
{code:java}
2021-06-22T15:51:12,959 INFO [Time-limited test]
balancer.TestStochasticLoadBalancerBalanceCluster(67): Mock Balance : {
srv1410554708:1 , srv2138765763:1 }
2021-06-22T15:51:12,959 INFO [Time-limited test]
balancer.BaseLoadBalancer(611): Start Generate Balance plan for cluster.
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.BalancerClusterState(293): number of table = 2
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.BalancerClusterState(313): max updated to 1 for table 0
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.BalancerClusterState(313): max updated to 1 for table 1
2021-06-22T15:51:12,960 DEBUG [Time-limited test]
balancer.RegionCountSkewCostFunction(53): RegionCountSkewCostFunction sees a
total of 2 servers and 2 regions.
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.TableSkewCostFunction(51): min = 1.0, max = 2.0, cost= 2.0
2021-06-22T15:51:12,960 DEBUG [Time-limited test]
balancer.StochasticLoadBalancer(347): We need to load balance cluster; total
cost=35.0, sum multiplier=582.0; cost/multiplier to need a balance is 0.05
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.TableSkewCostFunction(51): min = 1.0, max = 2.0, cost= 2.0
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.TableSkewCostFunction(51): min = 1.0, max = 2.0, cost= 2.0
2021-06-22T15:51:12,960 INFO [Time-limited test]
balancer.StochasticLoadBalancer(426): start StochasticLoadBalancer.balancer,
initCost=35.0, functionCost=RegionCountSkewCostFunction : (500.0, 0.0);
PrimaryRegionCountSkewCostFunction : (not needed); MoveCostFunction : (7.0,
0.0); RackLocalityCostFunction : (15.0, 0.0); TableSkewCostFunction : (35.0,
1.0); RegionReplicaHostCostFunction : (not needed);
RegionReplicaRackCostFunction : (not needed); ReadRequestCostFunction : (5.0,
0.0); CPRequestCostFunction : (5.0, 0.0); WriteRequestCostFunction : (5.0,
0.0); MemStoreSizeCostFunction : (5.0, 0.0); StoreFileCostFunction : (5.0,
0.0); computedMaxSteps: 3200
{code}
> Overhaul of test cluster set up for table skew
> ----------------------------------------------
>
> Key: HBASE-26023
> URL: https://issues.apache.org/jira/browse/HBASE-26023
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, test
> Environment: {code:java}
> {code}
> Reporter: Clara Xiong
> Priority: Major
>
> There is another bug in the original tableSkew cost function for aggregation
> of the cost per table:
> If we have 10 regions, one per table, evenly distributed on 10 nodes, the
> cost is scale to 1.0.
> The more tables we have, the closer the value will be to 1.0. The cost
> function becomes useless.
> All the balancer tests were set up with large numbers of tables with minimal
> regions per table. This artificially inflates the total cost and trigger
> balancer runs. With this fix on TableSkewFunction, we need to overhaul the
> tests too. We also need to add tests that reflect more diversified scenarios
> for table distribution such as large tables with large numbers of regions.
> {code:java}
> protected double cost() {
> double max = cluster.numRegions;
> double min = ((double) cluster.numRegions) / cluster.numServers;
> double value = 0;
> for (int i = 0; i < cluster.numMaxRegionsPerTable.length; i++) {
> value += cluster.numMaxRegionsPerTable[i];
> }
> LOG.info("min = {}, max = {}, cost= {}", min, max, value);
> return scale(min, max, value);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
