[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[ https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321952#comment-17321952 ] Viraj Jasani commented on HBASE-25767: -- Oh got it, Thanks > CandidateGenerator.getRandomIterationOrder is too slow on large cluster > --- > > Key: HBASE-25767 > URL: https://issues.apache.org/jira/browse/HBASE-25767 > Project: HBase > Issue Type: Improvement > Components: Balancer, Performance >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6 > > > Similar to HBASE-25759, it is just used to test whether we should skip > calculation, but in production masterServices will never be null. > == > Update, change the title of this issue for removing > CandidateGenerator.getRandomIterationOrder as it is too slow which causes the > CandidateGenerator.getRandomIterationOrder to fail when we remove the > masterServices field in LocalityBasedCandidateGenerator. As this is the most > important change in this issue so change the title of this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[ https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321939#comment-17321939 ] Duo Zhang commented on HBASE-25767: --- It is just our UTs... About 80k regions could make the execution very slow to fail the UT... > CandidateGenerator.getRandomIterationOrder is too slow on large cluster > --- > > Key: HBASE-25767 > URL: https://issues.apache.org/jira/browse/HBASE-25767 > Project: HBase > Issue Type: Improvement > Components: Balancer, Performance >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6 > > > Similar to HBASE-25759, it is just used to test whether we should skip > calculation, but in production masterServices will never be null. > == > Update, change the title of this issue for removing > CandidateGenerator.getRandomIterationOrder as it is too slow which causes the > CandidateGenerator.getRandomIterationOrder to fail when we remove the > masterServices field in LocalityBasedCandidateGenerator. As this is the most > important change in this issue so change the title of this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[ https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321938#comment-17321938 ] Viraj Jasani commented on HBASE-25767: -- [~zhangduo] out of curiosity, what size of the cluster (~ no of nodes and regions) you are observing slowness of the balancer with in general? > CandidateGenerator.getRandomIterationOrder is too slow on large cluster > --- > > Key: HBASE-25767 > URL: https://issues.apache.org/jira/browse/HBASE-25767 > Project: HBase > Issue Type: Improvement > Components: Balancer, Performance >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6 > > > Similar to HBASE-25759, it is just used to test whether we should skip > calculation, but in production masterServices will never be null. > == > Update, change the title of this issue for removing > CandidateGenerator.getRandomIterationOrder as it is too slow which causes the > CandidateGenerator.getRandomIterationOrder to fail when we remove the > masterServices field in LocalityBasedCandidateGenerator. As this is the most > important change in this issue so change the title of this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[
https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321857#comment-17321857
]
Hudson commented on HBASE-25767:
Results for branch branch-2.4
[build #98 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/98/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/98/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/98/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/98/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.4/98/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CandidateGenerator.getRandomIterationOrder is too slow on large cluster
> ---
>
> Key: HBASE-25767
> URL: https://issues.apache.org/jira/browse/HBASE-25767
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Performance
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6
>
>
> Similar to HBASE-25759, it is just used to test whether we should skip
> calculation, but in production masterServices will never be null.
> ==
> Update, change the title of this issue for removing
> CandidateGenerator.getRandomIterationOrder as it is too slow which causes the
> CandidateGenerator.getRandomIterationOrder to fail when we remove the
> masterServices field in LocalityBasedCandidateGenerator. As this is the most
> important change in this issue so change the title of this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[
https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321226#comment-17321226
]
Hudson commented on HBASE-25767:
Results for branch branch-2
[build #229 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/229/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/229/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/229/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/229/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/229/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CandidateGenerator.getRandomIterationOrder is too slow on large cluster
> ---
>
> Key: HBASE-25767
> URL: https://issues.apache.org/jira/browse/HBASE-25767
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Performance
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6
>
>
> Similar to HBASE-25759, it is just used to test whether we should skip
> calculation, but in production masterServices will never be null.
> ==
> Update, change the title of this issue for removing
> CandidateGenerator.getRandomIterationOrder as it is too slow which causes the
> CandidateGenerator.getRandomIterationOrder to fail when we remove the
> masterServices field in LocalityBasedCandidateGenerator. As this is the most
> important change in this issue so change the title of this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[
https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17321188#comment-17321188
]
Hudson commented on HBASE-25767:
Results for branch branch-2.3
[build #206 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/206/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/206/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/206/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/206/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2.3/206/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CandidateGenerator.getRandomIterationOrder is too slow on large cluster
> ---
>
> Key: HBASE-25767
> URL: https://issues.apache.org/jira/browse/HBASE-25767
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Performance
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6
>
>
> Similar to HBASE-25759, it is just used to test whether we should skip
> calculation, but in production masterServices will never be null.
> ==
> Update, change the title of this issue for removing
> CandidateGenerator.getRandomIterationOrder as it is too slow which causes the
> CandidateGenerator.getRandomIterationOrder to fail when we remove the
> masterServices field in LocalityBasedCandidateGenerator. As this is the most
> important change in this issue so change the title of this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[
https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320967#comment-17320967
]
Hudson commented on HBASE-25767:
Results for branch master
[build #263 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/263/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/263/General_20Nightly_20Build_20Report/]
(x) {color:red}-1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/263/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/263/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> CandidateGenerator.getRandomIterationOrder is too slow on large cluster
> ---
>
> Key: HBASE-25767
> URL: https://issues.apache.org/jira/browse/HBASE-25767
> Project: HBase
> Issue Type: Improvement
> Components: Balancer, Performance
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.3, 2.3.6
>
>
> Similar to HBASE-25759, it is just used to test whether we should skip
> calculation, but in production masterServices will never be null.
> ==
> Update, change the title of this issue for removing
> CandidateGenerator.getRandomIterationOrder as it is too slow which causes the
> CandidateGenerator.getRandomIterationOrder to fail when we remove the
> masterServices field in LocalityBasedCandidateGenerator. As this is the most
> important change in this issue so change the title of this issue.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[ https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320482#comment-17320482 ] Andrew Kyle Purtell commented on HBASE-25767: - Agree for 2.4 too. > CandidateGenerator.getRandomIterationOrder is too slow on large cluster > --- > > Key: HBASE-25767 > URL: https://issues.apache.org/jira/browse/HBASE-25767 > Project: HBase > Issue Type: Improvement > Components: Balancer, Performance >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > Similar to HBASE-25759, it is just used to test whether we should skip > calculation, but in production masterServices will never be null. > == > Update, change the title of this issue for removing > CandidateGenerator.getRandomIterationOrder as it is too slow which causes the > CandidateGenerator.getRandomIterationOrder to fail when we remove the > masterServices field in LocalityBasedCandidateGenerator. As this is the most > important change in this issue so change the title of this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-25767) CandidateGenerator.getRandomIterationOrder is too slow on large cluster
[ https://issues.apache.org/jira/browse/HBASE-25767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17320442#comment-17320442 ] Nick Dimiduk commented on HBASE-25767: -- We have noticed very slow balancer operation on a large cluster running 2.3. It would be nice to have this performance improvement :) > CandidateGenerator.getRandomIterationOrder is too slow on large cluster > --- > > Key: HBASE-25767 > URL: https://issues.apache.org/jira/browse/HBASE-25767 > Project: HBase > Issue Type: Improvement > Components: Balancer, Performance >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > Similar to HBASE-25759, it is just used to test whether we should skip > calculation, but in production masterServices will never be null. > == > Update, change the title of this issue for removing > CandidateGenerator.getRandomIterationOrder as it is too slow which causes the > CandidateGenerator.getRandomIterationOrder to fail when we remove the > masterServices field in LocalityBasedCandidateGenerator. As this is the most > important change in this issue so change the title of this issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
