[
https://issues.apache.org/jira/browse/HBASE-28805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Janardhan Hungund updated HBASE-28805:
--------------------------------------
Description:
The persistent bucket cache implementation feature relies on the persistence of
backing map to a persistent file. the protobuf APIs are used to serialise the
backing map and its related structures into the file. An asynchronous thread
periodically flushes the contents of backing map to the persistence file.
The protobuf library has a limitation of 2GB on the size of protobuf messages.
If the size of backing map increases beyond 2GB, an unexpected exception is
reported in the asynchronous thread and stops the persister thread. This causes
the persistent file go out of sync with the actual bucket cache. Due to this,
the bucket cache shrinks to a smaller size after a cache restart. Checksum
errors are also reported.
This Jira tracks the implementation of introducing chunking of the backing map
to persistence such that every protobuf is smaller than 2GB in size.
Thanks,
Janardhan
was:
The persistent bucket cache implementation feature relies on the persistence of
backing map to a persistent file. the protobuf APIs are used to serialise the
backing map and its related structures into the file. An asynchronous thread
periodically flushes the contents of backing map to the persistence file.
The protobuf library has a limitation of 2GB on the size of protobuf messages.
If the size of backing map increases beyond 2GB, an unexpected exception is
reported in the asynchronous thread and stops the persister thread. This causes
the persistent file go out of sync with the actual bucket cache. Due to this,
the bucket cache shrinks to a smaller size after a cache restart.
This Jira tracks the implementation of introducing chunking of the backing map
to persistence such that every protobuf is smaller than 2GB in size.
Thanks,
Janardhan
> Implement chunked persistence of backing map for persistent bucket cache.
> -------------------------------------------------------------------------
>
> Key: HBASE-28805
> URL: https://issues.apache.org/jira/browse/HBASE-28805
> Project: HBase
> Issue Type: Task
> Components: BucketCache
> Reporter: Janardhan Hungund
> Assignee: Janardhan Hungund
> Priority: Major
>
> The persistent bucket cache implementation feature relies on the persistence
> of backing map to a persistent file. the protobuf APIs are used to serialise
> the backing map and its related structures into the file. An asynchronous
> thread periodically flushes the contents of backing map to the persistence
> file.
> The protobuf library has a limitation of 2GB on the size of protobuf
> messages. If the size of backing map increases beyond 2GB, an unexpected
> exception is reported in the asynchronous thread and stops the persister
> thread. This causes the persistent file go out of sync with the actual bucket
> cache. Due to this, the bucket cache shrinks to a smaller size after a cache
> restart. Checksum errors are also reported.
> This Jira tracks the implementation of introducing chunking of the backing
> map to persistence such that every protobuf is smaller than 2GB in size.
> Thanks,
> Janardhan
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
