[ 
https://issues.apache.org/jira/browse/HBASE-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916830#comment-16916830
 ] 

HBase QA commented on HBASE-22618:
----------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 33m 
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green}  0m  
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green}  0m 
 0s{color} | {color:green} The patch appears to include 2 new or modified test 
files. {color} |
|| || || || {color:brown} branch-1 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  9m 
30s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
59s{color} | {color:green} branch-1 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
35s{color} | {color:green} branch-1 passed {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red}  0m 
18s{color} | {color:red} branch has 10 errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
49s{color} | {color:green} branch-1 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  2m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedjars {color} | {color:red}  0m 
16s{color} | {color:red} patch has 10 errors when building our shaded 
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}  
6m 56s{color} | {color:green} Patch does not cause any errors with Hadoop 2.8.5 
2.9.2. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 
46s{color} | {color:red} hbase-server generated 3 new + 5 unchanged - 0 fixed = 
8 total (was 5) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 36m 37s{color} 
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
21s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 29s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hbase.executor.TestExecutorService |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base: 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/artifact/patchprocess/Dockerfile
 |
| JIRA Issue | HBASE-22618 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12978696/HBASE-22618.branch-1.002.patch
 |
| Optional Tests |  dupname  asflicense  javac  javadoc  unit  findbugs  
shadedjars  hadoopcheck  hbaseanti  checkstyle  compile  |
| uname | Linux a95e8a863176 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | dev-support/hbase-personality.sh |
| git revision | branch-1 / 3d22604 |
| maven | version: Apache Maven 3.0.5 |
| Default Java | 1.8.0_222 |
| shadedjars | 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/artifact/patchprocess/branch-shadedjars.txt
 |
| shadedjars | 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/artifact/patchprocess/patch-shadedjars.txt
 |
| javadoc | 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/artifact/patchprocess/diff-javadoc-javadoc-hbase-server.txt
 |
| unit | 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/artifact/patchprocess/patch-unit-hbase-server.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/testReport/ |
| Max. process+thread count | 757 (vs. ulimit of 10000) |
| modules | C: hbase-server U: hbase-server |
| Console output | 
https://builds.apache.org/job/PreCommit-HBASE-Build/839/console |
| Powered by | Apache Yetus 0.9.0 http://yetus.apache.org |


This message was automatically generated.



> Provide a way to have Heterogeneous deployment
> ----------------------------------------------
>
>                 Key: HBASE-22618
>                 URL: https://issues.apache.org/jira/browse/HBASE-22618
>             Project: HBase
>          Issue Type: Improvement
>    Affects Versions: 3.0.0, 2.2.0, 2.2.1, 2.1.6, 1.4.11, 2.1.7
>            Reporter: Pierre Zemb
>            Assignee: Pierre Zemb
>            Priority: Major
>             Fix For: 3.0.0, 2.3.0
>
>         Attachments: HBASE-22618.branch-1.001.patch, 
> HBASE-22618.branch-1.002.patch, HBASE-22618.branch-2.001.patch, 
> HBASE-22618.master.001.patch
>
>
> Hi,
> We wouls like to open the discussion about bringing the possibility to have 
> regions deployed on {color:#222222}Heterogeneous deployment{color}, i.e Hbase 
> cluster running different kind of hardware.
> h2. Why?
>  * Cloud deployments means that we may not be able to have the same hardware 
> throughout the years
>  * Some tables may need special requirements such as SSD whereas others 
> should be using hard-drives
>  * {color:#222222} {color}*in our usecase*{color:#222222}(single table, 
> dedicated HBase and Hadoop tuned for our usecase, good key 
> distribution){color}*, the number of regions per RS was the real limit for 
> us*{color:#222222}.{color}
> h2. Our usecase
> We found out that *in our usecase*(single table, dedicated HBase and Hadoop 
> tuned for our usecase, good key distribution)*, the number of regions per RS 
> was the real limit for us*.
> Over the years, due to historical reasons and also the need to benchmark new 
> machines, we ended-up with differents groups of hardware: some servers can 
> handle only 180 regions, whereas the biggest can handle more than 900. 
> Because of such a difference, we had to disable the LoadBalancing to avoid 
> the {{roundRobinAssigmnent}}. We developed some internal tooling which are 
> responsible for load balancing regions across RegionServers. That was 1.5 
> year ago.
> h2. Our Proof-of-concept
> We did work on a Proof-of-concept 
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/HeterogeneousBalancer.java],
>  and some early tests 
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/HeterogeneousBalancer.java],
>  
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestHeterogeneousBalancerBalance.java],
>  and 
> [here|https://github.com/PierreZ/hbase/blob/dev/hbase14/balancer/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestHeterogeneousBalancerRules.java].
>  We wrote the balancer for our use-case, which means that:
>  * there is one table
>  * there is no region-replica
>  * good key dispersion
>  * there is no regions on master
> A rule file is loaded before balancing. It contains lines of rules. A rule is 
> composed of a regexp for hostname, and a limit. For example, we could have:
>  
> {quote}rs[0-9] 200
> rs1[0-9] 50
> {quote}
>  
> RegionServers with hostname matching the first rules will have a limit of 
> 200, and the others 50. If there's no match, a default is set.
> Thanks to the rule, we have two informations: the max number of regions for 
> this cluster, and the rules for each servers. {{HeterogeneousBalancer}} will 
> try to balance regions according to their capacity.
> Let's take an example. Let's say that we have 20 RS:
>  * 10 RS, named through {{rs0}} to {{rs9}} loaded with 60 regions each, and 
> each can handle 200 regions.
>  * 10 RS, named through {{rs10}} to {{rs19}} loaded with 60 regions each, and 
> each can support 50 regions.
> Based on the following rules:
>  
> {quote}rs[0-9] 200
> rs1[0-9] 50
> {quote}
>  
> The second group is overloaded, whereas the first group has plenty of space.
> We know that we can handle at maximum *2500 regions* (200*10 + 50*10) and we 
> have currently *1200 regions* (60*20). {{HeterogeneousBalancer}} will 
> understand that the cluster is *full at 48.0%* (1200/2500). Based on this 
> information, we will then *try to put all the RegionServers to ~48% of load 
> according to the rules.* In this case, it will move regions from the second 
> group to the first.
> The balancer will:
>  * compute how many regions needs to be moved. In our example, by moving 36 
> regions on rs10, we could go from 120.0% to 46.0%
>  * select regions with lowest data-locality
>  * try to find an appropriate RS for the region. We will take the lowest 
> available RS.
> h2. Other implementations and ideas
> Clay Baenziger proposed this idea on the dev ML:
> {quote}{color:#222222}Could it work to have the stochastic load balancer use 
> [pluggable cost functions instead of this static list of cost 
> functions|[https://github.com/apache/hbase/blob/baf3ae80f5588ee848176adefc9f56818458a387/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java#L198]]?
>  Then, could this type of a load balancer be implemented simply as a new cost 
> function which folks could choose to load and mix with the others?{color}
> {quote}
> {color:#222222}I think this could be an interesting way to include 
> user-functions in the mix. As you know your hardawre and the pattern access, 
> you can easily know which metrics is important for balancing, for us, it will 
> only be the number of regions, but we could mix-it with the incoming 
> writes!{color}
>  
> bhupendra.jain proposed also the ideas of "labels"
>  
> {quote}{color:#222222}Internally, we are also having discussion to develop 
> similar solution. In our approach, We were also thinking of adding "RS Label" 
> Feature similar to Hadoop Node Label feature. {color}
> {color:#222222}Each RS can have a label to denote its capabilities / 
> resources . When user create table, there can be extra attributes with its 
> descriptor. The balancer can decide to host region of table based on RS label 
> and these attributes further.  {color}
>  {color:#222222}With RS label feature, Balancer can be more intelligent.  
> Example tables with high read load needs more cache backed by SSDs , So such 
> table regions should be hosted on RS having SSDs ... {color}
> {quote}
> {color:#222222}I love the idea, but I think Clay's idea is better for a 
> better and faster first set of commits on the subject! What do you think? 
> {color}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Reply via email to