[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16815920#comment-16815920
]
Biju Nair commented on HBASE-9741:
--
Current
[hbase-default.xml|https://github.com/apache/hbase/blob/f22b8ade631367d584b7065c7c76b3e0b6eca97b/hbase-common/src/main/resources/hbase-default.xml#L624-L630]
has the smaller {{slop}} for SLB which is enabled by default and HBase book
documents about the higher {{slop}} value used by SimpleLoadBalancer. Can we
close this ticket?
> Remove hbase.regions.slop from hbase-default.xml
>
>
> Key: HBASE-9741
> URL: https://issues.apache.org/jira/browse/HBASE-9741
> Project: HBase
> Issue Type: Bug
> Components: Balancer
>Reporter: Elliott Clark
>Priority: Major
> Labels: beginner
> Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch,
> HBASE-9741-v3.patch, HBASE-9741-v4.patch
>
>
> Different balancers have different slop default values. We should remove
> hbase.regions.slop from hbase-default.xml
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14150977#comment-14150977 ] Ranjana Rajendran commented on HBASE-9741: -- The test was written for the old balancer and took the default value in defaults.xml which was 0.2f to prevent the stochastic balancer from searching for a new base. With this value of 0.2f removed from defaults.xml, was then taking the default value of 0.1f specified in the line (float slop = (float)UTIL.getConfiguration().getFloat(hbase.regions.slop, 0.1f);) which made the test fail as this made the balancer search for a better fit which was not the intention of this test. I would like to understand why this test is required, i.e. why is this test required to assert that regions are balanced using the function assertRegionsAreBalanced() ? If this test need to be rewritten, I would like to take up this task. Highly appreciate any input how to rewrite it different from the patch above. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: beginner Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch, HBASE-9741-v4.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109441#comment-14109441
]
Jonathan Hsieh commented on HBASE-9741:
---
Something doesn't make sense, why does TestRegionRebalancing break with the
values is removed from the conf? Do you have an explanation [~eclark],
[~ranjana_rajendran]? (is this the expect thing that makes this confusing or
was this unexpected?)
{coe}diff --git
a/hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java
b/hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java
index fdcb00c..9252da9 100644
---
a/hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java
+++
b/hbase-server/src/test/java/org/apache/hadoop/hbase/TestRegionRebalancing.java
@@ -165,7 +165,8 @@ public class TestRegionRebalancing {
// TODO: Fix this test. Old balancer used to run with 'slop'. New
// balancer does not.
boolean success = false;
-float slop = (float)UTIL.getConfiguration().getFloat(hbase.regions.slop,
0.1f);
+float slop =
(float)UTIL.getConfiguration().getFloat(HConstants.LOAD_BALANCER_SLOP_KEY,
0.2f);
+
if (slop = 0) slop = 1;
for (int i = 0; i 5; i++) {
{code}
Remove hbase.regions.slop from hbase-default.xml
Key: HBASE-9741
URL: https://issues.apache.org/jira/browse/HBASE-9741
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Labels: beginner
Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch,
HBASE-9741-v3.patch, HBASE-9741-v4.patch
Different balancers have different slop default values. We should remove
hbase.regions.slop from hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14109771#comment-14109771 ] Elliott Clark commented on HBASE-9741: -- This is part of what makes this confusing. The stochastic balancer would work better with a smaller slop, however the test was written using the default slop value for the old balancer. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: beginner Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch, HBASE-9741-v4.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110079#comment-14110079 ] Jonathan Hsieh commented on HBASE-9741: --- I'm still missing something -- we don't change the default balancer with this patch so I'd assume the test would be the same still. Does the test's have a problem where the value isn't passed in (unlikely) or the value is gathered by the balancers out of band? Is this a case where that configuration setting should be named differently for the two balancers but is conflated and causes very different behaviors? Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: beginner Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch, HBASE-9741-v4.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14110278#comment-14110278 ] Elliott Clark commented on HBASE-9741: -- The stochastic balancer should be using the smaller slop value. Removing the value from the defaults.xml just allows the smaller value that is specified as the default for the stochastic balancer to be used. However this test was not written with the smaller value in mind. bq.we don't change the default balancer with this patch so I'd assume the test would be the same still. The test asserts that nothing should move; when in reality there might actually be a better fit. The larger slop stops the stochastic balancer from searching for the better fit. Removing the slop from the defaults just means that the test, which was previously written with the old balancer in mind, needs to be tweaked to pass with the new effective default. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: beginner Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch, HBASE-9741-v4.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091431#comment-14091431 ] Ranjana Rajendran commented on HBASE-9741: -- With the default of 0.2f removed, the function TestRegionReBalancing.java, sets the slop to 0.1f which makes this test fail. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: beginner Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14091473#comment-14091473 ] Ranjana Rajendran commented on HBASE-9741: -- I changed TestRegionReBalancing.java to use 0.2f as the default value and this passes the test. TestRegionReBalancing.java need to be rewritten for the new balancers which do not use the slop. Will appreciate any input how to rewrite this function for the new balancers. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: beginner Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14083665#comment-14083665 ] Elliott Clark commented on HBASE-9741: -- Still need to remove it from the defaults xml. Otherwise it's confusing to users on what the default actually it. The default will look to be whatever is in the xml but will actually be dependent on what balancer is being used. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: noob Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch, HBASE-9741-v3.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078049#comment-14078049
]
Esteban Gutierrez commented on HBASE-9741:
--
Why not rename the {{hbase.regions.slop}} initially to
{{hbase.regions.balancer.base.slop}} and add a new property for the
StochasticLoadBalancer {{hbase.regions.balancer.stochastic.slop}}. Then we can
deprecate gracefully {{hbase.regions.slop}} with a warning and still support
the property for a while.
Remove hbase.regions.slop from hbase-default.xml
Key: HBASE-9741
URL: https://issues.apache.org/jira/browse/HBASE-9741
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Labels: noob
Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch
Different balancers have different slop default values. We should remove
hbase.regions.slop from hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078153#comment-14078153 ] Elliott Clark commented on HBASE-9741: -- Because the stochastic balancer doesn't really use the slop in the same way. It computes costs, so slop isn't really used to determine when a balance plan is good. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: noob Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[
https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14078156#comment-14078156
]
Esteban Gutierrez commented on HBASE-9741:
--
[~eclark] then it might be better to rename {{hbase.regions.slop}} the the
StochasticLoadBalancer to {{hbase.regions.stochastic.load.balancer.cost}} and
do not rename {{hbase.regions.slop}} does it sounds good?
Remove hbase.regions.slop from hbase-default.xml
Key: HBASE-9741
URL: https://issues.apache.org/jira/browse/HBASE-9741
Project: HBase
Issue Type: Bug
Reporter: Elliott Clark
Labels: noob
Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch
Different balancers have different slop default values. We should remove
hbase.regions.slop from hbase-default.xml
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077105#comment-14077105 ] Ranjana Rajendran commented on HBASE-9741: -- We are examining the different tests and the codebase at large for uses of this property. One notable example we found was: TestRegionRebalancing.java, and there is a note suggesting that we rewrite the test to not incorporate slop. We will continue looking through the code for other places where this may impact things. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: noob Attachments: HBASE-9741-v0.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077263#comment-14077263 ] Elliott Clark commented on HBASE-9741: -- Why does TestRegionRebalancing not work ? The set config in TestRegionRebalancing should override what's in the xml. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: noob Attachments: HBASE-9741-v0.patch, HBASE-9741-v1.patch Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-9741) Remove hbase.regions.slop from hbase-default.xml
[ https://issues.apache.org/jira/browse/HBASE-9741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13791438#comment-13791438 ] Jean-Marc Spaggiari commented on HBASE-9741: If we remove it from the hbase-default.xml then we need, at least, to add it into the documentation because I think it's not there today. Remove hbase.regions.slop from hbase-default.xml Key: HBASE-9741 URL: https://issues.apache.org/jira/browse/HBASE-9741 Project: HBase Issue Type: Bug Reporter: Elliott Clark Labels: noob Different balancers have different slop default values. We should remove hbase.regions.slop from hbase-default.xml -- This message was sent by Atlassian JIRA (v6.1#6144)
