Josh Elser updated HBASE-18135:
    Attachment: HBASE-18135.004.patch

> Track file archival for low latency space quota with snapshots
> --------------------------------------------------------------
>                 Key: HBASE-18135
>                 URL: https://issues.apache.org/jira/browse/HBASE-18135
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Major
>             Fix For: 3.0.0
>         Attachments: HBASE-18135.001.patch, HBASE-18135.002.patch, 
> HBASE-18135.004.patch
> Related to the work proposed on HBASE-17748 and building on the same idea as 
> HBASE-18133, we can make the space quota tracking for HBase snapshots faster 
> to respond.
> When snapshots are in play, the location of a file (whether in the {{data}} 
> or {{archive}} directory) plays a factor in the realized size of a table. 
> Like flushes, compactions, etc, moving files from the data directory to the 
> archive directory is done by the RegionServer. We can hook into this call and 
> send the necessary information to the Master so that it can more quickly 
> update the size of a table when there are snapshots in play.
> This will require the RegionServer to report the full coordinates of the file 
> being moved (table+region+family+file) so that the SnapshotQuotaObserverChore 
> running in the master can avoid HDFS lookups in partial or total to compute 
> the location of a Region's hfiles.
> This may also require some refactoring of the SnapshotQuotaObserverChore to 
> de-couple the receipt of these file archival reports from RegionServers (e.g. 
> {{HRegionFileSystem.removeStoreFiles(..)}}, and the Master processing the 
> sizes of snapshots.

This message was sent by Atlassian JIRA

Reply via email to