[
https://issues.apache.org/jira/browse/HBASE-28044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil updated HBASE-28044:
-----------------------------------------
Resolution: Abandoned
Status: Resolved (was: Patch Available)
> Reduce frequency of saving backing map in persistence cache
> -----------------------------------------------------------
>
> Key: HBASE-28044
> URL: https://issues.apache.org/jira/browse/HBASE-28044
> Project: HBase
> Issue Type: Sub-task
> Reporter: Wellington Chevreuil
> Assignee: Wellington Chevreuil
> Priority: Major
>
> Currently we always write the whole cache mapping into the persistent map
> file. This is not a lightweight operation, on a full 1.6TB cache with ten
> millions of block, this can grow as much as 10GB. In the current persistent
> cache implementation, we flush it to disk every 1s. If we raise the
> "checkpoint" period, we risk lose more cache events in the event of a
> recovery.
> This proposes reduce the frequency needed to save the backing map as follows:
> 1) Save every block addition/eviction into a single file in disk;
> 2) Checkpoint at higher intervals, consolidating all transactions into the
> larger map file;
> 3) In the event of failure, recovery would consist of loading the latest map
> file, then applying all the transactions files sequentially;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
