Wellington Chevreuil created HBASE-28044:
--------------------------------------------
Summary: Reduce frequency of saving backing map in persistence
cache
Key: HBASE-28044
URL: https://issues.apache.org/jira/browse/HBASE-28044
Project: HBase
Issue Type: Sub-task
Reporter: Wellington Chevreuil
Assignee: Wellington Chevreuil
Currently we always write the whole cache mapping into the persistent map file.
This is not a lightweight operation, on a full 1.6TB cache with ten millions of
block, this can grow as much as 10GB. In the current persistent cache
implementation, we flush it to disk every 1s. If we raise the "checkpoint"
period, we risk lose more cache events in the event of a recovery.
This proposes reduce the frequency needed to save the backing map as follows:
1) Save every block addition/eviction into a single file in disk;
2) Checkpoint at higher intervals, consolidating all transactions into the
larger map file;
3) In the event of failure, recovery would consist of loading the latest map
file, then applying all the transactions files sequentially;
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
