[
https://issues.apache.org/jira/browse/HBASE-30135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-30135.
------------------------------------------
Release Note:
If an imbalance is found following the changes in the CacheAwareCostFunction
introduced by HBASE-30134, these changes the generate logic in the
CacheAwareCandidateGenerator as follows:
1) For regions with no cache on other servers, if the cache ratio is below the
hbase.master.balancer.cacheaware.lowCacheRatioThreshold value (defaults 0.35),
it will suggest a move plan for a server with enough cache space available to
accommodate the region's hot data size (as determined by DataTieringManager)
times the “hbase.master.balancer.cacheaware.minFreeCacheSpaceFactor” value
(defaults to 1).
2) If there's no server with cache space available for the region, no plan is
suggested for that specific region.
This also changes CacheAwareCostFunction.regionMoved method, which is called
once StochasticLoadBalancer obtains the plan and verifies if it decreases the
overall cost, by simply calculating the individual cache ratio for the region
in the new server, either using the already cached data, or potential cache due
to free space. Then it takes a diff of this from the cache ratio on current
server, normalizing it by the best ever possible overall ratio and increment
this value to the current overall cache ratio, which is ultimately used for
calculating the cost.
Resolution: Fixed
Merged to master, branch-3, branch-2 and branch-2.6. Thanks for reviewing it,
[~psomogyi] !
> Improve CacheAwareLoadBalancer to simulate low cache ratio regions as cached
> in candidate servers with enough cache space
> -------------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-30135
> URL: https://issues.apache.org/jira/browse/HBASE-30135
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer
> Affects Versions: 3.0.0-beta-1, 2.6.5
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.0.0, 2.7.0, 2.6.6
>
>
> When enabling time based priority on CFs that are already cached on clusters
> with datasets larger than the cache capacity, once the now cold data gets
> evicted, hot data that was previously uncached due to no cache space would
> remain uncached unless:
> 1) A client read request read it from the file system and caches it;
> 2) An operator manually disables and reenables tables with hot data, so that
> the prefetch executor can run and cache those blocks.
> Both options are non optimal, leading to temporary performance impacts and/or
> requiring manual interventions.
> The CacheAwareLoadBalancer, currently, only raises the cost of moving highly
> cached regions, when calculating assignment plans, and doesn't consider cache
> ratio at all when calculating potential imbalance, only skewness. So in the
> scenario where regions are evenly distributed and there's no skewness,
> CacheAwareLoadBalancer would not trigger any moves even when there would be
> regions with low cache ratio on many servers and enough cache space to
> accommodate those regions data in region servers cache.
>
> The solution for this problem will be split in two jiras, for ease of review.
> HBASE-30134 should include low cached ratio regions in the imbalance
> calculation, so that CacheAwareLoadBalancer can trigger the computation of
> new assignment plans.
> This ticket should provide the changes needed to recalculate cache ratio on
> the assignment plans, simulating that low cache ratio regions moved to enough
> cache free space servers would get fully cached, so that such plan scores
> higher then the current state.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
