[
https://issues.apache.org/jira/browse/HBASE-26178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17420285#comment-17420285
]
Hudson commented on HBASE-26178:
--------------------------------
Results for branch master
[build #398 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/398/]:
(x) *{color:red}-1 overall{color}*
----
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/398/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/398/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/398/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Improve data structure and algorithm for BalanceClusterState to improve
> computation speed for large cluster
> -----------------------------------------------------------------------------------------------------------
>
> Key: HBASE-26178
> URL: https://issues.apache.org/jira/browse/HBASE-26178
> Project: HBase
> Issue Type: Bug
> Components: Balancer, Performance
> Reporter: Clara Xiong
> Assignee: Clara Xiong
> Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> With ~800 node and ~500 regions per node on our large production cluster,
> balancer cannot complete within hours even after we just add 2% servers after
> maintenance.
> The unit tests with larger number of regions are taking longer and longer
> with changes to balancer with recent changes too, evident with the increment
> of the time limit recent PR's included.
> It is time to replace some of the data structure for better time complexity
> including:
> int[][] regionsPerServer; // serverIndex -> region list
> int[][] regionsPerHost; // hostIndex -> list of regions
> int[][] regionsPerRack; // rackIndex -> region list
> // serverIndex -> sorted list of regions by primary region index
> ArrayList<HashSet<Integer>> primariesOfRegionsPerServer;
> // hostIndex -> sorted list of regions by primary region index
> int[][] primariesOfRegionsPerHost;
> // rackIndex -> sorted list of regions by primary region index
> int[][] primariesOfRegionsPerRack;
> Areas of algorithm improvement include:
> # O(n ) to O(1) time to lookup or update per server/host/rack for every
> move test iteration.(n = number of regions per server/host/rack).
> # O(n ) to O(1) time for reserse lookup of region index from primary index.
> # Recomputation of primaryRegionCountSkewCostFunction reduced from O(n ) to
> O(1)
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
