[
https://issues.apache.org/jira/browse/HBASE-27999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahul Agarkar updated HBASE-27999:
----------------------------------
Description:
HBase uses ephemeral cache to cache the blocks by reading them from the slow
storages and storing them to the bucket cache. This cache is warmed up
everytime a region server is started. Depending on the data size and the
configured cache size, the cache warm up can take anywhere between a few
minutes to few hours. Doing this everytime the region server starts can be a
very expensive process. To eliminate this, HBASE-27313 implemented the cache
persistence feature where the region servers periodically persist the blocks
cached in the bucket cache. This persisted information is then used to
resurrect the cache in the event of a region server restart because of normal
restart or crash.
This feature aims at enhancing this capability of HBase to enable the balancer
implementation considers the cache allocation of each region on region servers
when calculating a new assignment plan and uses the region/region server cache
allocation info reported by region servers which takes into account to
calculate the percentage of HFiles cached for each region on the hosting
server, and then use that as another factor when deciding on an optimal, new
assignment plan.
A design document describing the balancer can be found at
https://docs.google.com/document/d/1A8-eVeRhZjwL0hzFw9wmXl8cGP4BFomSlohX2QcaFg4/edit?usp=sharing
was:
HBase uses ephemeral cache to cache the blocks by reading them from the slow
storages and storing them to the bucket cache. This cache is warmed up
everytime a region server is started. Depending on the data size and the
configured cache size, the cache warm up can take anywhere between a few
minutes to few hours. Doing this everytime the region server starts can be a
very expensive process. To eliminate this, HBASE-27313 implemented the cache
persistence feature where the region servers periodically persist the blocks
cached in the bucket cache. This persisted information is then used to
resurrect the cache in the event of a region server restart because of normal
restart or crash.
This feature aims at enhancing this capability of HBase to enable the balancer
implementation considers the cache allocation of each region on region servers
when calculating a new assignment plan and uses the region/region server cache
allocation info reported by region servers which takes into account to
calculate the percentage of HFiles cached for each region on the hosting
server, and then use that as another factor when deciding on an optimal, new
assignment plan..
> Implement cache aware load balancer
> -----------------------------------
>
> Key: HBASE-27999
> URL: https://issues.apache.org/jira/browse/HBASE-27999
> Project: HBase
> Issue Type: Sub-task
> Components: Balancer
> Reporter: Rahul Agarkar
> Assignee: Rahul Agarkar
> Priority: Major
>
> HBase uses ephemeral cache to cache the blocks by reading them from the slow
> storages and storing them to the bucket cache. This cache is warmed up
> everytime a region server is started. Depending on the data size and the
> configured cache size, the cache warm up can take anywhere between a few
> minutes to few hours. Doing this everytime the region server starts can be a
> very expensive process. To eliminate this, HBASE-27313 implemented the cache
> persistence feature where the region servers periodically persist the blocks
> cached in the bucket cache. This persisted information is then used to
> resurrect the cache in the event of a region server restart because of normal
> restart or crash.
> This feature aims at enhancing this capability of HBase to enable the
> balancer implementation considers the cache allocation of each region on
> region servers when calculating a new assignment plan and uses the
> region/region server cache allocation info reported by region servers which
> takes into account to calculate the percentage of HFiles cached for each
> region on the hosting server, and then use that as another factor when
> deciding on an optimal, new assignment plan.
>
> A design document describing the balancer can be found at
> https://docs.google.com/document/d/1A8-eVeRhZjwL0hzFw9wmXl8cGP4BFomSlohX2QcaFg4/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
