[
https://issues.apache.org/jira/browse/HBASE-11780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14104184#comment-14104184
]
Jean-Marc Spaggiari commented on HBASE-11780:
---------------------------------------------
I will let Esteban provide its view on that, but my initial thinking was:
{quote}
How will heterogeneous hardware node attributes be discovered and communicated
to the balancer?
Is static node weighting in clusters of ~hundreds or ~thousands viable? Assume
some hardware is always in churn. How would updates be communicated to the
master? If by flat file somewhere, will we have to restart the master for
changes to take effect?
{quote}
Each node will put its load into an HDFS file so no need to send that to the
balancer. Then balancer will read from the load directory to work on the
distribution. I thought about this because my 2 clusters are heterogeneous...
Then balancer just need to read this directory to decide what need to be
balanced and how. No need for any restart.
{quote}
Elsewhere it has been discussed that we'd like to slim down and ultimately
remove RegionLoad and replace that duplication of our metrics publishing with a
metrics listener on the master that accomplishes the same end. Would this issue
work towards or against that goal?
{quote}
As this balancer might be able to benefit from this idea. Instead of using a
load file, get the metrics on the master. Concern is the load this will create
on the master. Pushing load files on HDFS don't put any load on the master
which will read them only when required. On the other side, pushing tones of
metrics to the master can have an impact.
> Add a resource based load balancer
> ----------------------------------
>
> Key: HBASE-11780
> URL: https://issues.apache.org/jira/browse/HBASE-11780
> Project: HBase
> Issue Type: Improvement
> Reporter: Esteban Gutierrez
> Assignee: Esteban Gutierrez
>
> In some situations with heterogeneous hardware it might be helpful to take in
> account the available resources such as RAM, cores or spindles before for the
> region assignment. A form of weight per RS might be an initial approach but
> for also a limit of open regions in a RS or maximum open store files could be
> used.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
