[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713809#comment-13713809 ] Lars Hofhansl commented on HBASE-8432: -- Would like another +1 before I commit. a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Labels: patch Fix For: 0.98.0, 0.95.2, 0.94.10 Attachments: 8432-0.94.txt, 8432-trunk.txt, patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713830#comment-13713830 ] Lars Hofhansl commented on HBASE-8432: -- Wait wrong issue. Ted's presumably OK with his patch. Committing now. a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Labels: patch Fix For: 0.98.0, 0.95.2, 0.94.10 Attachments: 8432-0.94.txt, 8432-trunk.txt, patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713262#comment-13713262 ] Wang Qiang commented on HBASE-8432: --- To [~lhofhansl] yes, ted's patch is ok, it's almost the same with mine, I am not familiar with the patch submition process, thanks to [~yuzhih...@gmail.com] a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Labels: patch Fix For: 0.94.11 Attachments: 8432-trunk.txt, patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713270#comment-13713270 ] Lars Hofhansl commented on HBASE-8432: -- Cool. Let's pull it into 0.94.10. a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Labels: patch Fix For: 0.98.0, 0.95.2, 0.94.10 Attachments: 8432-trunk.txt, patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13713328#comment-13713328
]
Hadoop QA commented on HBASE-8432:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12593115/8432-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6405//console
This message is automatically generated.
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Labels: patch
Fix For: 0.98.0, 0.95.2, 0.94.10
Attachments: 8432-0.94.txt, 8432-trunk.txt, patch_20130716.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13711239#comment-13711239 ] Lars Hofhansl commented on HBASE-8432: -- [~aaronwq] Does Ted's patch work for you? a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Labels: patch Fix For: 0.94.10 Attachments: 8432-trunk.txt, patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709643#comment-13709643 ] Wang Qiang commented on HBASE-8432: --- [~Jean-Marc Spaggiari] yes, I have tested all the 3 different scenarios in our cluster a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709646#comment-13709646 ] Wang Qiang commented on HBASE-8432: --- [~jmspaggi] yes, I have tested all the 3 different scenarios in our cluster a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709741#comment-13709741 ] Jean-Marc Spaggiari commented on HBASE-8432: Good. If you think your patch is correct you need to click on submit patch to trigger the QA system. If it pass, I will apply your patch locally and give it a try. a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130716.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13709804#comment-13709804
]
Ted Yu commented on HBASE-8432:
---
{code}
-maxToTake--;
}
+ maxToTake -= regionsToPut;
{code}
Is the above change intentional ?
{code}
if(regionCount = max) {
break;
}
+BalanceInfo balanceInfo =
serverBalanceInfo.get(server.getKey().getServerName());
+if(balanceInfo != null) {
+regionCount += balanceInfo.getNumRegionsAdded();
+}
+if(regionCount = max) {
+continue;
+}
{code}
The check below can be omitted if serverBalanceInfo.get() is lifted above the
original 'if(regionCount = max)':
{code}
+if(regionCount = max) {
+continue;
+}
{code}
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Attachments: patch_20130716.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 2
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 2
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 2
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 2
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
from the above logs, we can also find that some regions needn't move, but
they moved. follow into
'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I
found that 'maxToTake' is error calculated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710625#comment-13710625
]
Wang Qiang commented on HBASE-8432:
---
{code:xml}
if (regionsToPut == 0)
{
regionsToPut = 1;
-maxToTake--;
}
+ maxToTake -= regionsToPut;
{code}
[~yuzhih...@gmail.com] yes, the above is intentional, because when 'int
regionsToPut = min - regionCount' is greater than 0, the 'maxToTake' doesn't
decrease correctly
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Attachments: patch_20130716.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 2
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 2
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 2
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 2
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
from the above logs, we can also find that some regions needn't move, but
they moved. follow into
'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I
found that 'maxToTake' is error calculated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710626#comment-13710626
]
Wang Qiang commented on HBASE-8432:
---
[~yuzhih...@gmail.com] if the check below is omitted, the scenario in my case
will work terribly
{code:xml}
+if(regionCount = max) {
+continue;
+}
{code}
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Attachments: patch_20130716.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 2
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 2
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 2
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 2
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
from the above logs, we can also find that some regions needn't move, but
they moved. follow into
'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I
found that 'maxToTake' is error calculated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710639#comment-13710639
]
Hadoop QA commented on HBASE-8432:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12592513/patch_20130716.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6370//console
This message is automatically generated.
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Labels: patch
Fix For: 0.94.10
Attachments: patch_20130716.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 2
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 2
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 2
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 2
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
from the above logs, we can also find that some regions needn't move, but
they moved. follow into
'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I
found that 'maxToTake' is error calculated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13710751#comment-13710751
]
Hadoop QA commented on HBASE-8432:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12592712/8432-trunk.txt
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop
2.0 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.rest.client.TestRemoteAdmin
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/6372//console
This message is automatically generated.
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Labels: patch
Fix For: 0.94.10
Attachments: 8432-trunk.txt, patch_20130716.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708406#comment-13708406 ] Jean-Marc Spaggiari commented on HBASE-8432: Hi Wang, Have you tried this change with the 3 different scenarios: 1) More regions than RS; 2) More regions than RS/2 but less than RS; 3) Less regions than RS/2. ? a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130425_01.txt, patch.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708506#comment-13708506
]
Ted Yu commented on HBASE-8432:
---
Second patch should have been taken at the root of workspace.
{code}
if(regionCount = max) {
break;
}
+BalanceInfo balanceInfo =
serverBalanceInfo.get(server.getKey().getServerName());
+if(balanceInfo != null) {
+ regionCount += balanceInfo.getNumRegionsAdded();
+}
+if(regionCount = max) {
+ continue;
+}
{code}
If accounting for BalanceInfo is moved to above 'if(regionCount = max)', we
can save one check for 'if(regionCount = max)'
Please attach trunk patch as well.
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Attachments: patch_20130425_01.txt, patch.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 2
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 2
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 2
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 2
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
from the above logs, we can also find that some regions needn't move, but
they moved. follow into
'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I
found that 'maxToTake' is error calculated.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[
https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13708251#comment-13708251
]
Wang Qiang commented on HBASE-8432:
---
// If we still have regions to dish out, assign underloaded to max
if (0 regionsToMove.size()) {
for (Map.EntryServerAndLoad, ListHRegionInfo server :
serversByLoad.entrySet()) {
int regionCount = server.getKey().getLoad();
if(regionCount = max) {
break;
}
addRegionPlan(regionsToMove, fetchFromTail,
server.getKey().getServerName(), regionsToReturn);
if (emptyRegionServerPresent) {
fetchFromTail = !fetchFromTail;
}
if (regionsToMove.isEmpty()) {
break;
}
}
}
the code above should change to the below, it hasn't check the balance info
// If we still have regions to dish out, assign underloaded to max
if (0 regionsToMove.size()) {
for (Map.EntryServerAndLoad, ListHRegionInfo server :
serversByLoad.entrySet()) {
int regionCount = server.getKey().getLoad();
if(regionCount = max) {
break;
}
BalanceInfo balanceInfo =
serverBalanceInfo.get(server.getKey().getServerName());
if(balanceInfo != null) {
regionCount += balanceInfo.getNumRegionsAdded();
}
if(regionCount = max) {
continue;
}
addRegionPlan(regionsToMove, fetchFromTail,
server.getKey().getServerName(), regionsToReturn);
if (emptyRegionServerPresent) {
fetchFromTail = !fetchFromTail;
}
if (regionsToMove.isEmpty()) {
break;
}
}
}
a table with unbalanced regions will balance indefinitely with the
'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
---
Key: HBASE-8432
URL: https://issues.apache.org/jira/browse/HBASE-8432
Project: HBase
Issue Type: Bug
Components: Balancer
Affects Versions: 0.94.5
Environment: Linux 2.6.32-el5.x86_64
Reporter: Wang Qiang
Priority: Critical
Attachments: patch_20130425_01.txt
it happened that a table with unbalanced regions, as follows in my
cluster(the cluster has 20 regionservers, the table has 12 regions):
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 2
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev9.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times
load-balances, the table are still unbalanced:
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 2
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 2
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 2
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev19.cm6:60030/ 1
http://hadoopdev8.cm6:60030/ 1
http://hadoopdev17.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev5.cm6:60030/ 1
http://hadoopdev22.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev7.cm6:60030/ 1
http://hadoopdev21.cm6:60030/ 2
http://hadoopdev16.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
http://hadoopdev3.cm6:60030/ 1
http://hadoopdev20.cm6:60030/ 1
http://hadoopdev18.cm6:60030/ 1
http://hadoopdev4.cm6:60030/ 1
http://hadoopdev12.cm6:60030/ 1
http://hadoopdev15.cm6:60030/ 1
http://hadoopdev14.cm6:60030/ 2
http://hadoopdev6.cm6:60030/ 1
http://hadoopdev13.cm6:60030/ 1
http://hadoopdev11.cm6:60030/ 1
http://hadoopdev10.cm6:60030/ 1
from the above logs, we can also find that some regions needn't move, but
they moved. follow into
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642785#comment-13642785 ] Jean-Marc Spaggiari commented on HBASE-8432: Thanks for the followup [~aaronwq]. Have you tried also the other scenarios? Like regions# RS#/2 and regions# RS#? Are they all still working fine? a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130425_01.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641727#comment-13641727 ] Jean-Marc Spaggiari commented on HBASE-8432: [~qian725] have you tested with your patch? Are you sure about the += in maxToTake += regionCount? Should it not be =? a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130425_01.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13642599#comment-13642599 ] Wang Qiang commented on HBASE-8432: --- @Jean-Marc Spaggiari I have run the unit test, and deployed in my cluster, it works ok. 'maxToTake' is cumulated in the 'for' loop, so the opertaor should be '+=' a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130425_01.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641386#comment-13641386 ] Anoop Sam John commented on HBASE-8432: --- Looks like HBASE-7060?? a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641406#comment-13641406 ] Ted Yu commented on HBASE-8432: --- HBASE-7060 went into 0.94.3 [~aaronwq]: Can you confirm that the problem still exists in 0.94.5 ? a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641417#comment-13641417 ] Wang Qiang commented on HBASE-8432: --- yes, the problem is existed in 0.94.5. We found the problem in our production cluster, and then can do replay the problem in our development cluster. a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-8432) a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer'
[ https://issues.apache.org/jira/browse/HBASE-8432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13641451#comment-13641451 ] Wang Qiang commented on HBASE-8432: --- In my test case, it's less than half of the regionserver number, after several balances(two or three), the table went balanced. But when the region number is larger than half of the regionserver number and less than the regionserver number, the table was still unbalanced after many times of balances. the attachment is my patch. a table with unbalanced regions will balance indefinitely with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer' --- Key: HBASE-8432 URL: https://issues.apache.org/jira/browse/HBASE-8432 Project: HBase Issue Type: Bug Components: Balancer Affects Versions: 0.94.5 Environment: Linux 2.6.32-el5.x86_64 Reporter: Wang Qiang Priority: Critical Attachments: patch_20130425_01.txt it happened that a table with unbalanced regions, as follows in my cluster(the cluster has 20 regionservers, the table has 12 regions): http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 2 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev9.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 with the 'org.apache.hadoop.hbase.master.DefaultLoadBalancer', after 5 times load-balances, the table are still unbalanced: http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 2 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 2 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 2 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev19.cm6:60030/ 1 http://hadoopdev8.cm6:60030/ 1 http://hadoopdev17.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev5.cm6:60030/ 1 http://hadoopdev22.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev7.cm6:60030/ 1 http://hadoopdev21.cm6:60030/ 2 http://hadoopdev16.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 http://hadoopdev3.cm6:60030/ 1 http://hadoopdev20.cm6:60030/ 1 http://hadoopdev18.cm6:60030/ 1 http://hadoopdev4.cm6:60030/ 1 http://hadoopdev12.cm6:60030/ 1 http://hadoopdev15.cm6:60030/ 1 http://hadoopdev14.cm6:60030/ 2 http://hadoopdev6.cm6:60030/ 1 http://hadoopdev13.cm6:60030/ 1 http://hadoopdev11.cm6:60030/ 1 http://hadoopdev10.cm6:60030/ 1 from the above logs, we can also find that some regions needn't move, but they moved. follow into 'org.apache.hadoop.hbase.master.DefaultLoadBalancer.balanceCluster()', I found that 'maxToTake' is error calculated. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
