Josh Elser created HBASE-18135:
----------------------------------
Summary: Track file archival for low latency space quota with
snapshots
Key: HBASE-18135
URL: https://issues.apache.org/jira/browse/HBASE-18135
Project: HBase
Issue Type: Improvement
Reporter: Josh Elser
Assignee: Josh Elser
Related to the work proposed on HBASE-17748 and building on the same idea as
HBASE-18133, we can make the space quota tracking for HBase snapshots faster to
respond.
When snapshots are in play, the location of a file (whether in the {{data}} or
{{archive}} directory) plays a factor in the realized size of a table. Like
flushes, compactions, etc, moving files from the data directory to the archive
directory is done by the RegionServer. We can hook into this call and send the
necessary information to the Master so that it can more quickly update the size
of a table when there are snapshots in play.
This will require the RegionServer to report the full coordinates of the file
being moved (table+region+family+file) so that the SnapshotQuotaObserverChore
running in the master can avoid HDFS lookups in partial or total to compute the
location of a Region's hfiles.
This may also require some refactoring of the SnapshotQuotaObserverChore to
de-couple the receipt of these file archival reports from RegionServers (e.g.
{{HRegionFileSystem.removeStoreFiles(..)}}, and the Master processing the sizes
of snapshots.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
