[
https://issues.apache.org/jira/browse/HBASE-18135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387186#comment-16387186
]
Hudson commented on HBASE-18135:
--------------------------------
FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #4697 (See
[https://builds.apache.org/job/HBase-Trunk_matrix/4697/])
HBASE-18135 Implement mechanism for RegionServers to report file (elserj: rev
4a4c0120494757539d680c2d7d44fe6ab3d71d27)
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/MasterQuotaManager.java
* (add)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifier.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
* (add)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierFactoryImpl.java
* (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/master/HMaster.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/SnapshotQuotaObserverChore.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestSnapshotQuotaObserverChore.java
* (add)
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestFileArchiverNotifierImpl.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HStore.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/MockRegionServerServices.java
* (edit)
hbase-client/src/main/java/org/apache/hadoop/hbase/quotas/QuotaTableUtil.java
* (edit) hbase-protocol-shaded/src/main/protobuf/RegionServerStatus.proto
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/RegionServerServices.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/quotas/TestLowLatencySpaceQuotas.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/master/MasterRpcServices.java
* (add)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierFactory.java
* (edit)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/RegionServerSpaceQuotaManager.java
* (add)
hbase-server/src/main/java/org/apache/hadoop/hbase/quotas/FileArchiverNotifierImpl.java
* (edit)
hbase-server/src/test/java/org/apache/hadoop/hbase/master/MockRegionServer.java
> Track file archival for low latency space quota with snapshots
> --------------------------------------------------------------
>
> Key: HBASE-18135
> URL: https://issues.apache.org/jira/browse/HBASE-18135
> Project: HBase
> Issue Type: Improvement
> Reporter: Josh Elser
> Assignee: Josh Elser
> Priority: Major
> Fix For: 3.0.0
>
> Attachments: HBASE-18135.001.patch, HBASE-18135.002.patch,
> HBASE-18135.004.patch, HBASE-18135.005.patch
>
>
> Related to the work proposed on HBASE-17748 and building on the same idea as
> HBASE-18133, we can make the space quota tracking for HBase snapshots faster
> to respond.
> When snapshots are in play, the location of a file (whether in the {{data}}
> or {{archive}} directory) plays a factor in the realized size of a table.
> Like flushes, compactions, etc, moving files from the data directory to the
> archive directory is done by the RegionServer. We can hook into this call and
> send the necessary information to the Master so that it can more quickly
> update the size of a table when there are snapshots in play.
> This will require the RegionServer to report the full coordinates of the file
> being moved (table+region+family+file) so that the SnapshotQuotaObserverChore
> running in the master can avoid HDFS lookups in partial or total to compute
> the location of a Region's hfiles.
> This may also require some refactoring of the SnapshotQuotaObserverChore to
> de-couple the receipt of these file archival reports from RegionServers (e.g.
> {{HRegionFileSystem.removeStoreFiles(..)}}, and the Master processing the
> sizes of snapshots.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
