Jinglun commented on HDFS-14813:

My bad, I miss [~ayushtkn] comments in HDFS-14814. Let's discuss inherit quota 
in HDFS-14814, I'll move my comments there.

> RBF: Make Global quota and Remote quota consistent.
> ---------------------------------------------------
>                 Key: HDFS-14813
>                 URL: https://issues.apache.org/jira/browse/HDFS-14813
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Jinglun
>            Assignee: Jinglun
>            Priority: Major
> Make Global quota and remote quota consistent.
>  (Global quota: the quota on mount table, Remote quota: the quota on 
> namespace)
> HDFS administrator can use global quota to simplify the management for 
> federation paths. But there is no consistent constraint for the global quota 
> and the remote quota. As an hdfs administrator the inconsistent has 3 
> disadvantages on management:
>      1. The quota part of getQuotaUsage() on a federation path is not 
> helpful. It's neither the global quota nor one of the remote quotas.
>      2. The global quota could be different with the remote quota. When a 
> QuotaExceedException happens it needs the administrator to find out whether 
> it's a violation of the global quota or the remote quota.
>      3. For management simplicity, it's always a good idea to keep the global 
> quota and the remote quota the same. Now we need the administrator to keep 
> the consistent manually.
>  My proposal is to add a constraint for global quota: 
>      1. For federation paths, global quota could be inherited from parent 
> federation path.
>      2. For all remote paths in mount tables, the remote quotas must be 
> consistent with the global quotas.
>  To implement this, my idea is:
>      1. Global quota could be inherited. Add a method getGlobalQuota(String 
> path) to Quota.java returning the global quota.
>      2. Each time RouterQuotaUpdateService updates the quota usage for mount 
> table entries, it also checks and updates the remote quota.
>      3. When getQuotaUsage() on a federation path, return the global quota.
>      4. When setQuota() on a federation path, first update the global quota 
> in mount table, then recompute global quota for the current path and its 
> children paths, finally update all the federation paths.
> Implement 1+2 in HDFS-14814
> Implement 4 in HDFS-14815

This message was sent by Atlassian Jira

To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

Reply via email to