[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13902260#comment-13902260 ] Hudson commented on HBASE-10498: FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #90 (See [https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/90/]) HBASE-10498 Add new APIs to load balancer interface(Rajesh) (rajeshbabu: rev 1568188) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901245#comment-13901245 ] rajeshbabu commented on HBASE-10498: [~stack] bq. When a region goes online/offline, we now update in two places? Why not have the one place read the other so we keep one list only, an authoritative one? Some times we need to update in two places only for regions to colocate. If we read from one list some times regions may not be colocated. For example lets take two regions regionA, regionB to be colocated and currently assigned to same region server RS1. Now lets suppose they need to move to other region server. 1) first region regionA started moving to RS2 2) region regionB started moving, while selecting destination if we refer regionsAssignments list then we will get RS1 as the desination, so regionB will be assinged to RS1. 3) By the time assignments completed the two regions will be opened at different servers which is not expected. These kind of things can happen during master startup,balancing,move or assignment failure cases etc.. To avoid the mismatches some times we need to refer multiple data structures like regionsInTransitions or regionPlans and regionAssignments in AM. Still some times we may not ensure same region server has been selected for the regions to co-locate. bq. Inject into the balancer an Interface it can pull on when it needs to know what is online? Any way balancer has reference to master,we need not add new APIs to get the online regions from AM. Thanks. > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901244#comment-13901244 ] Hudson commented on HBASE-10498: SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #147 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/147/]) HBASE-10498 Add new APIs to load balancer interface(Rajesh) (rajeshbabu: rev 1568189) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901107#comment-13901107 ] Hudson commented on HBASE-10498: SUCCESS: Integrated in HBase-TRUNK #4920 (See [https://builds.apache.org/job/HBase-TRUNK/4920/]) HBASE-10498 Add new APIs to load balancer interface(Rajesh) (rajeshbabu: rev 1568188) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901096#comment-13901096 ] stack commented on HBASE-10498: --- When a region goes online/offline, we now update in two places? Why not have the one place read the other so we keep one list only, an authoritative one? Inject into the balancer an Interface it can pull on when it needs to know what is online? St.Ack > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901092#comment-13901092 ] Hudson commented on HBASE-10498: SUCCESS: Integrated in HBase-0.98 #158 (See [https://builds.apache.org/job/HBase-0.98/158/]) HBASE-10498 Add new APIs to load balancer interface(Rajesh) (rajeshbabu: rev 1568189) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/AssignmentManager.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/LoadBalancer.java * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13901024#comment-13901024 ] Enis Soztutar commented on HBASE-10498: --- Looks good. > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13900251#comment-13900251 ] ramkrishna.s.vasudevan commented on HBASE-10498: +1 > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899353#comment-13899353 ] Ted Yu commented on HBASE-10498: lgtm > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > Attachments: HBASE-10498.patch > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[
https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899229#comment-13899229
]
Hadoop QA commented on HBASE-10498:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12628487/HBASE-10498.patch
against trunk revision .
ATTACHMENT ID: 12628487
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:red}-1 site{color}. The patch appears to cause mvn site goal to
fail.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.util.TestHBaseFsck
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8669//console
This message is automatically generated.
> Add new APIs to load balancer interface
> ---
>
> Key: HBASE-10498
> URL: https://issues.apache.org/jira/browse/HBASE-10498
> Project: HBase
> Issue Type: Improvement
> Components: Balancer
>Reporter: rajeshbabu
>Assignee: rajeshbabu
> Fix For: 0.98.1, 0.99.0
>
> Attachments: HBASE-10498.patch
>
>
> If a custom load balancer required to maintain region and corresponding
> server locations,
> we can capture this information when we run any balancer algorithm before
> assignment(like random,retain).
> But during master startup we will not call any balancer algorithm if a region
> already assinged
> During split also we open child regions first in RS and then notify to master
> through zookeeper.
> So split regions information cannot be captured into balancer.
> Since balancer has access to master we can get the information from online
> regions or region plan data structures in AM.
> But some use cases we cannot relay on this information(mainly to maintain
> colocation of two tables regions).
> So it's better to add some APIs to load balancer to notify balancer when
> *region is online or offline*.
> These APIs helps a lot to maintain *regions colocation through custom load
> balancer* which is very important in secondary indexing.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899093#comment-13899093 ] rajeshbabu commented on HBASE-10498: bq.Can you not add a new attribute for the Stochastic LB to consider – colocation – and weight it above others rather than add API? Regions colocation involves all the LB algorithms like random,ratain or balanceCluster depends on scenario(like create,enable,balancer,startup). At present adding co-location logic to Stochastic LB give optimal colocation plan only in balanceCluster case right? bq. However, for secondary indexing, co-locating regions should be a "hard constraint" I imagine. Yes co-locating regions is hard constraint, by the time the regions assignment completed they should be in same RS(there may be some time gap in between the assignments). bq. Still it should be possible to implement hard constraints like co-location inside the core LB's, but implement the logic of deciding which regions to co-locate as a pluggable layer. This can be handled in another JIRA right [~enis]? As part of this issue will provide APIs to notify balancer when region is online or offline(these help to avoid co-location mismatches in sec indexes). > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898603#comment-13898603 ] Enis Soztutar commented on HBASE-10498: --- bq. Can you not add a new attribute for the Stochastic LB to consider – colocation – and weight it above others rather than add API? This is kind of the opposite of what we do for not co-locating the region replicas. The patch at HBASE-10351 adds "soft" constraints for ensuring that the replicas are not co-located. I highly suggest taking a look there. However, for secondary indexing, co-locating regions should be a "hard constraint" I imagine. Still it should be possible to implement hard constraints like co-location inside the core LB's, but implement the logic of deciding which regions to co-locate as a pluggable layer. > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898420#comment-13898420 ] stack commented on HBASE-10498: --- Can you not add a new attribute for the Stochastic LB to consider -- colocation -- and weight it above others rather than add API? Or it may be the case that colocating regions cross-cuts how SLB works currently (it having a single region focus). > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898190#comment-13898190 ] rajeshbabu commented on HBASE-10498: bq.Is this for doing all the placement decisions through the LB interfaces? Yes [~enis]. > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898181#comment-13898181 ] Enis Soztutar commented on HBASE-10498: --- Is this for doing all the placement decisions through the LB interfaces? I think it makes sense. > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898059#comment-13898059 ] rajeshbabu commented on HBASE-10498: Added 0.98.1 to fix versions. > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.98.1, 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (HBASE-10498) Add new APIs to load balancer interface
[ https://issues.apache.org/jira/browse/HBASE-10498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13898035#comment-13898035 ] James Taylor commented on HBASE-10498: -- Can this work be done in 0.98? > Add new APIs to load balancer interface > --- > > Key: HBASE-10498 > URL: https://issues.apache.org/jira/browse/HBASE-10498 > Project: HBase > Issue Type: Improvement > Components: Balancer >Reporter: rajeshbabu >Assignee: rajeshbabu > Fix For: 0.99.0 > > > If a custom load balancer required to maintain region and corresponding > server locations, > we can capture this information when we run any balancer algorithm before > assignment(like random,retain). > But during master startup we will not call any balancer algorithm if a region > already assinged > During split also we open child regions first in RS and then notify to master > through zookeeper. > So split regions information cannot be captured into balancer. > Since balancer has access to master we can get the information from online > regions or region plan data structures in AM. > But some use cases we cannot relay on this information(mainly to maintain > colocation of two tables regions). > So it's better to add some APIs to load balancer to notify balancer when > *region is online or offline*. > These APIs helps a lot to maintain *regions colocation through custom load > balancer* which is very important in secondary indexing. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
