Janardhan Hungund created HBASE-28805:
-----------------------------------------

             Summary: Implement chunked persistence of backing map for 
persistent bucket cache.
                 Key: HBASE-28805
                 URL: https://issues.apache.org/jira/browse/HBASE-28805
             Project: HBase
          Issue Type: Task
          Components: BucketCache
            Reporter: Janardhan Hungund
            Assignee: Janardhan Hungund


The persistent bucket cache implementation feature relies on the persistence of 
backing map to a persistent file. the protobuf APIs are used to serialise the 
backing map and its related structures into the file. An asynchronous thread 
periodically flushes the contents of backing map to the persistence file.

The protobuf library has a limitation of 2GB on the size of protobuf messages. 
If the size of backing map increases beyond 2GB, an unexpected exception is 
reported in the asynchronous thread and stops the persister thread. This causes 
the persistent file go out of sync with the actual bucket cache. Due to this, 
the bucket cache shrinks to a smaller size after a cache restart.

This Jira tracks the implementation of introducing chunking of the backing map 
to persistence such that every protobuf is smaller than 2GB in size.

Thanks,
Janardhan



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

Reply via email to