[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452520#comment-13452520 ] Lars Hofhansl commented on HBASE-6720: -- Hmm... Maybe this is a none-issue now. I'm happy to close as won't fix. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452523#comment-13452523 ] Ted Yu commented on HBASE-6720: --- That's fine. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452525#comment-13452525 ] Elliott Clark commented on HBASE-6720: -- The above code won't fix the described issue if regions are moving quickly but the load balancer is moving a lot. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452544#comment-13452544 ] Elliott Clark commented on HBASE-6720: -- [~lhofhansl] Sorry what I wrote was unclear. I meant to say: The code Ted posted won't stop the load balancer from moving lots of regions if the region moves happen quickly. cutoffTime is some time way in the future assuming that things are going normally. So since applying the plans is usually very quick. All of the plans will be applied. If anything I would stopping the LoadBalancer from moving after a certain number of regions is more useful than stopping it after a number of milliseconds. It's easier to for users to know a good starting value that applies to their cluster. There's no way I could tell you right now how long a balance on our cluster takes when a RegionServer goes down. However I could say the max number of regions that I would want to move at a time. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13452551#comment-13452551 ] Lars Hofhansl commented on HBASE-6720: -- Hey Elliot... We had comment crossing. :) Ok, so I take it, the patch is good after all, and we should apply (with the move of the constant as Stack suggests). (Didn't spend much time actually looking at the code... Should probably do so). Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449459#comment-13449459 ] stack commented on HBASE-6720: -- Ok, then I change to a +1 on patch (with amendment of HConstant addition on commit) given Elliott's comment. bq. Imo way too much balancing logic has leaked into the HMaster class. I would think that timing and per table balancing should be moved into the Balancer. Yes. We need to add more to the Balancer Interface? Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449463#comment-13449463 ] Elliott Clark commented on HBASE-6720: -- I'm -1 The patch doesn't work if per table balancing is turned off. The patch also fails if the number of regions that need to be moved in one table is larger than the limit. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449811#comment-13449811 ] stack commented on HBASE-6720: -- Thanks Elliott. How so? Won't all be well on the next go around of the balancer? Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449870#comment-13449870 ] Elliott Clark commented on HBASE-6720: -- No you're right it will work the next time around. I just mistook where the if statement was located. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[
https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449889#comment-13449889
]
Ted Yu commented on HBASE-6720:
---
We already have cutoff time for region balancing:
{code}
rpCount++;
if (rpCount plans.size()
// if performing next balance exceeds cutoff time, exit the loop
(System.currentTimeMillis() + (totalRegPlanExecTime / rpCount))
cutoffTime) {
LOG.debug(No more balancing till next balance run;
maximumBalanceTime= +
maximumBalanceTime);
break;
}
{code}
Is the above not enough ?
Optionally limit number of regions balanced in each balancer run
Key: HBASE-6720
URL: https://issues.apache.org/jira/browse/HBASE-6720
Project: HBase
Issue Type: Bug
Reporter: Lars Hofhansl
Fix For: 0.96.0, 0.94.3
Attachments: 6720-0.96-v1.txt
See discussion on HBASE-3866
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13448975#comment-13448975 ] Elliott Clark commented on HBASE-6720: -- When this is put in we need to make sure to change the StochasticLoadBalancer as well. It right now has a setting hbase.master.balancer.stochastic.maxMoveRegions that sets the maximum number of regions to move at a time. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449404#comment-13449404 ] Lars Hofhansl commented on HBASE-6720: -- Thanks Elliot. We'll just make hbase.master.balancer.stochastic.maxMoveRegion default to the new general setting. Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-6720) Optionally limit number of regions balanced in each balancer run
[ https://issues.apache.org/jira/browse/HBASE-6720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13449440#comment-13449440 ] stack commented on HBASE-6720: -- Consider moving the HConstant change down into the master package since that is only place it is used. Also, maybe Ted can answer this, but why are we applying the balance filter outside of a balancer in the HMaster code and not inside the balance code as is done here w/ the Stochastic balancer? Optionally limit number of regions balanced in each balancer run Key: HBASE-6720 URL: https://issues.apache.org/jira/browse/HBASE-6720 Project: HBase Issue Type: Bug Reporter: Lars Hofhansl Fix For: 0.96.0, 0.94.3 Attachments: 6720-0.96-v1.txt See discussion on HBASE-3866 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
