Josh Elser updated HBASE-18133:
    Attachment: HBASE-18133.004.patch

> Low-latency space quota size reports
> ------------------------------------
>                 Key: HBASE-18133
>                 URL: https://issues.apache.org/jira/browse/HBASE-18133
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>            Priority: Major
>             Fix For: 3.0.0
>         Attachments: HBASE-18133.001.patch, HBASE-18133.002.patch, 
> HBASE-18133.003.patch, HBASE-18133.004.patch
> Presently space quota enforcement relies on RegionServers sending reports to 
> the master about each Region that they host. This is done by periodically, 
> reading the cached size of each HFile in each Region (which was ultimately 
> computed from HDFS).
> This means that the Master is unaware of Region size growth until the the 
> next time this chore in a RegionServer fires which is a fair amount of 
> latency (a few minutes, by default). Operations like flushes, compactions, 
> and bulk-loads are delayed even though the RegionServer is running those 
> operations locally.
> Instead, we can create an API which these operations could invoke that would 
> automatically update the size of the Region being operated on. For example, a 
> successful flush can report that the size of a Region increased by the size 
> of the flush. A compaction can subtract the size of the input files of the 
> compaction and add in the size of the resulting file.
> This de-couples the computation of a Region's size from sending the Region 
> sizes to the Master, allowing us to send reports more frequently, increasing 
> the responsiveness of the cluster to size changes.

This message was sent by Atlassian JIRA

Reply via email to