[
https://issues.apache.org/jira/browse/HBASE-18135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Elser updated HBASE-18135:
-------------------------------
Attachment: HBASE-18135.001.patch
.001 Parking an initial implementation for reviews to catch up.
This moves the logic for computing the size of a snapshot out of a Chore and
into its own in-memory state in the Master. This state can be updated via the
Chore or via RPCs from a RS. The computed snapshot sizes are still persisted to
the hbase:quota table. The actual code change is much smaller than the patch
lends itself because a bit of code was just moved directly.
> Track file archival for low latency space quota with snapshots
> --------------------------------------------------------------
>
> Key: HBASE-18135
> URL: https://issues.apache.org/jira/browse/HBASE-18135
> Project: HBase
> Issue Type: Improvement
> Reporter: Josh Elser
> Assignee: Josh Elser
> Fix For: 3.0.0
>
> Attachments: HBASE-18135.001.patch
>
>
> Related to the work proposed on HBASE-17748 and building on the same idea as
> HBASE-18133, we can make the space quota tracking for HBase snapshots faster
> to respond.
> When snapshots are in play, the location of a file (whether in the {{data}}
> or {{archive}} directory) plays a factor in the realized size of a table.
> Like flushes, compactions, etc, moving files from the data directory to the
> archive directory is done by the RegionServer. We can hook into this call and
> send the necessary information to the Master so that it can more quickly
> update the size of a table when there are snapshots in play.
> This will require the RegionServer to report the full coordinates of the file
> being moved (table+region+family+file) so that the SnapshotQuotaObserverChore
> running in the master can avoid HDFS lookups in partial or total to compute
> the location of a Region's hfiles.
> This may also require some refactoring of the SnapshotQuotaObserverChore to
> de-couple the receipt of these file archival reports from RegionServers (e.g.
> {{HRegionFileSystem.removeStoreFiles(..)}}, and the Master processing the
> sizes of snapshots.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
