[
https://issues.apache.org/jira/browse/HBASE-26067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-26067:
-------------------------------
Release Note:
Introduces the StoreFileTracker interface to HBase. This is a server-side
interface which abstracts how a Store (column family) knows what files should
be included in that Store. Previously, HBase relied on a listing the directory
a Store used for storage to determine the files which should make up that Store.
After this feature, there are two implementations of StoreFileTrackers. The
first (and default) implementation is listing the Store directory. The second
is a new implementation which records files which belong to a Store within each
Store. Whenever the list of files that make up a Store change, this metadata
file will be updated.
This feature is notable in that it better enables HBase to function on storage
systems which do not provide the typical posix filesystem semantics, most
importantly, those which do not implement a file rename operation which is
atomic. Storage systems which do not implement atomic renames often implement a
rename as a copy and delete operation which amplifies the I/O costs by 2x.
At scale, this feature should have a 2x reduction in I/O costs when using
storage systems that do not provide atomic renames, most importantly in HBase
compactions and memstore flushes. See the corresponding section, "Store File
Tracking", in the HBase book for more information on how to use this feature.
> Change the way on how we track store file list
> ----------------------------------------------
>
> Key: HBASE-26067
> URL: https://issues.apache.org/jira/browse/HBASE-26067
> Project: HBase
> Issue Type: Umbrella
> Components: HFile
> Reporter: Duo Zhang
> Assignee: Duo Zhang
> Priority: Major
>
> Open a separated jira to track the work since it can not be fully included in
> HBASE-24749.
> I think this could be a landed prior to HBASE-24749, as if this works, we
> could have different implementations for tracking store file list.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
