[
https://issues.apache.org/jira/browse/HBASE-25894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17347126#comment-17347126
]
Bryan Beaudreault commented on HBASE-25894:
-------------------------------------------
I'm on a very old EOL'd version of HBase, but recently looked into this there.
One thing I noticed is that
[CostFromRegionLoadFunction#cost|https://github.com/apache/hbase/blob/741b4b4674be0c500bc7a474739d6481834c5e4a/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/CostFromRegionLoadFunction.java#L55-L67]
we are calculating the full cost of every region for every server on every
iteration of the balancer. On the other hand, our CandidateGenerators typically
generate actions which only affect 1 or 2 regions on 1 or 2 servers. As such we
should really only need to re-calculate cost for the 2 servers which are
affected by an action. This is a lot of wasted work for big clusters with 1000s
of regions or 100s of regionservers.
I improved this dramatically by caching the costs and invalidating them in
tryAction.
Here is the patch I wrote for my version of hbase:
[https://gist.github.com/bbeaudreault/fca47f8694792be8516049c9d883a581.] It
will not apply to HBase 2+, but wanted to contribute it to this Jira incase
it's helpful.
> Improve the performance for region load and region count related cost
> functions
> -------------------------------------------------------------------------------
>
> Key: HBASE-25894
> URL: https://issues.apache.org/jira/browse/HBASE-25894
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer, Performance
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Major
> Fix For: 3.0.0-alpha-1, 2.5.0
>
>
> For a large cluster, we have a lot of regions, so computing the whole cost
> will be expensive. We should try to remove the unnecessary calculation as
> much as possible.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
