[
https://issues.apache.org/jira/browse/HDFS-12934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16314861#comment-16314861
]
Íñigo Goiri commented on HDFS-12934:
------------------------------------
For the refactor, I was referring to doing something like this:
{code}
RouterQuotaUsage quotaUsage = this.cache.get(path);
if (quotaUsage != null && isQuotaSet(quotaUsage)) {
return quotaUsage;
}
int pos = path.lastIndexOf(Path.SEPARATOR);
if (pos != -1) {
String parentPath = path.substring(0, pos);
return getQuotaUsage(parentPath);
}
return null;
{code}
If I didn't miss anything, it's the same behavior as you had.
For the merging comment, I was referring to what you currently do in
{{RouterRpcServer#aggregateQuota()}}.
A single {{usage.getQuota() == -1 && usage.getSpaceQuota() == -1}}, sets the
whole thing to {{builder.quota(HdfsConstants.QUOTA_DONT_SET);}}.
Is that the expected behavior?
For the {{isNsViolated}}, we may want to just reference
{{DirectoryWithQuotaFeature}} if we cannot use the verify.
Maybe have the same name.
Minor comments on [^HDFS-12934.005.patch]:
* Rename {{isQuotaServiceEnabled()}} to {{isQuotaEnabled()}}?
* {{RouterRpcServer#getRemoteLoctions()}} has a typo right now.
* Both {{getRemoteLoctions()}} and {{getQuotaLocations()}} in
{{RouterRpcServer}} deal with quotas, so we should have quota in the name.
* Could we extract {{this.router.getQuotaManager()}} in
{{RouterRpcServer#getRemoteLoctions()}}?
* In HDFS-12919, I'm doing a merge for the locations with a minor refactor the
invokeConcurrent(), you may want to take a look there as I think your code
would get a little simplified when doing {{aggregateQuota()}}.
* In {{RouterQuotaUpdateService }}, what about wrapping in a function the part
that does "write out updated mount table entries into State Store"? To follow
the pattern of the other mount table store functions.
In general, the code looks pretty clean; I'll try to get some time to test it
thoroughly.
I haven't checked the unit tests carefully, I'll check if all the corner cases
are covered there.
> RBF: Federation supports global quota
> -------------------------------------
>
> Key: HDFS-12934
> URL: https://issues.apache.org/jira/browse/HDFS-12934
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 3.0.0
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Labels: RBF
> Attachments: HDFS-12934.001.patch, HDFS-12934.002.patch,
> HDFS-12934.003.patch, HDFS-12934.004.patch, HDFS-12934.005.patch, RBF support
> global quota.pdf
>
>
> Now federation doesn't support set the global quota for each folder.
> Currently the quota will be applied for each subcluster under the specified
> folder via RPC call.
> It will be very useful for users that federation can support setting global
> quota and exposing the command of this.
> In a federated environment, a folder can be spread across multiple
> subclusters. For this reason, we plan to solve this by following way:
> # Set global quota across each subcluster. We don't allow each subcluster can
> exceed maximun quota value.
> # We need to construct one <Path, QuotaUsage> cache map for storing the sum
> quota usage of these subclusters under federation folder. Every time we want
> to do WRITE operation under specified folder, we will get its quota usage
> from cache and verify its quota. If quota exceeded, throw exception,
> otherwise update its quota usage in cache when finishing operations.
> The quota will be set to mount table and as a new field in mount table. The
> set/unset command will be like:
> {noformat}
> hdfs dfsrouteradmin -setQuota -ns <nsQuota> -ss <ssQuota> <mount table>
> hdfs dfsrouteradmin -clrQuota <mount table>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
