[
https://issues.apache.org/jira/browse/HDFS-12934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16315781#comment-16315781
]
Yiqun Lin commented on HDFS-12934:
----------------------------------
Thanks for the patient review, [~elgoiri].
bq. For the refactor, I was referring to doing something like this:..
The change looks good, I have changed this.
{quote}
A single usage.getQuota() == -1 && usage.getSpaceQuota() == -1, sets the whole
thing to builder.quota(HdfsConstants.QUOTA_DONT_SET);.
Is that the expected behavior?
{quote}
Yes, it's expected behavior. {{HdfsConstants.QUOTA_DONT_SET}} flag set in the
aggregated quota just used for trigger one time setQuota RPC call rather than
passing this flag in new generated RouterQuotaUsage. That is say, we found some
sub-clusters haven't set quota in required path. Just do the initial settings.
bq. For the isNsViolated, we may want to just reference
DirectoryWithQuotaFeature if we cannot use the verify.
Maybe have the same name.
I have updated this with the same name.
Other minor comments are all addressed except this one:
{quote}
In HDFS-12919, I'm doing a merge for the locations with a minor refactor the
invokeConcurrent(), you may want to take a look there as I think your code
would get a little simplified when doing aggregateQuota().
{quote}
Seeing this and the content you mentioned in HDFS-12919, I have tried to make
this change as you suggested. But found It depends on some adjustment in
HDFS-12919 (some method visible change, etc.). Can we do this after HDFS-12919
being committed?
Attach the updated patch.
> RBF: Federation supports global quota
> -------------------------------------
>
> Key: HDFS-12934
> URL: https://issues.apache.org/jira/browse/HDFS-12934
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 3.0.0
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Labels: RBF
> Attachments: HDFS-12934.001.patch, HDFS-12934.002.patch,
> HDFS-12934.003.patch, HDFS-12934.004.patch, HDFS-12934.005.patch, RBF support
> global quota.pdf
>
>
> Now federation doesn't support set the global quota for each folder.
> Currently the quota will be applied for each subcluster under the specified
> folder via RPC call.
> It will be very useful for users that federation can support setting global
> quota and exposing the command of this.
> In a federated environment, a folder can be spread across multiple
> subclusters. For this reason, we plan to solve this by following way:
> # Set global quota across each subcluster. We don't allow each subcluster can
> exceed maximun quota value.
> # We need to construct one <Path, QuotaUsage> cache map for storing the sum
> quota usage of these subclusters under federation folder. Every time we want
> to do WRITE operation under specified folder, we will get its quota usage
> from cache and verify its quota. If quota exceeded, throw exception,
> otherwise update its quota usage in cache when finishing operations.
> The quota will be set to mount table and as a new field in mount table. The
> set/unset command will be like:
> {noformat}
> hdfs dfsrouteradmin -setQuota -ns <nsQuota> -ss <ssQuota> <mount table>
> hdfs dfsrouteradmin -clrQuota <mount table>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
