[
https://issues.apache.org/jira/browse/HDFS-12934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16316598#comment-16316598
]
Íñigo Goiri edited comment on HDFS-12934 at 1/8/18 5:11 PM:
------------------------------------------------------------
Thanks [~linyiqun], I think this is pretty much good to go.
A few minor comments on [^HDFS-12934.006.patch]:
* Check styles in the test can be solved.
* In {{Router}}, we could get rid of the field for {{isQuotaEnabled}} (keep it
as var) and use {{this.quotaManager == null}} in {{isQuotaEnabled()}}.
* {{Router#getQuotaManager()}} should just return {{this.quotaManager}}; no
need on doing the {{isQuotaEnabled()}}.
* For the comment about {{DirectoryWithQuotaFeature}}, I was talking about
linking to this class method in the javadoc of
{{RouterQuotaUsage#verifyNamespaceQuota}}.
* No need for {{timeOutMs}} in the internal {{getQuotaUsage()}}, just call it
{{getQuotaUsageInternal()}} or something (not sure it can be made protected).
It'd be also good if we could move this into the module approach (more on this
at the end of this comment).
For the relation with HDFS-12919, I agree that the method returning is too
convoluted and no need to carry it over; we cna leave it for that JIRA.
However, in the latets patches, I made the erasure coding in
{{RouterRpcServer}} to be a module.
I think we should do this here too, it only requires changing the visibility of
a couple methods in {{RouterRpcServer}} and starts what I think is the right
direction.
BTW, review appreciated on HDFS-12919 (I'm relying on Jenkins for tests because
I'm having issues for testing EC policies on Windows from 3.0).
was (Author: elgoiri):
Thanks [~linyiqun], I think this is pretty much good to go.
A few minor comments on [^HDFS-12934.006.patch]:
* Check styles in the test can be solved.
* In {{Router}}, we could get rid of the field for {{isQuotaEnabled}} (keep it
as var) and use {{this.quotaManager == null}} in {{isQuotaEnabled()}}.
* {{Router#getQuotaManager()}} should just return {{this.quotaManager}}; no
need on doing the {{isQuotaEnabled()}}.
* For the comment about {{DirectoryWithQuotaFeature}}, I was talking about
linking to this class method in the javadoc of
{{RouterQuotaUsage#verifyNamespaceQuota}}.
For the relation with HDFS-12919, I agree that the method returning is too
convoluted and no need to carry it over; we cna leave it for that JIRA.
However, in the latets patches, I made the erasure coding in
{{RouterRpcServer}} to be a module.
I think we should do this here too, it only requires changing the visibility of
a couple methods in {{RouterRpcServer}} and starts what I think is the right
direction.
BTW, review appreciated on HDFS-12919 (I'm relying on Jenkins for tests because
I'm having issues for testing EC policies on Windows from 3.0).
> RBF: Federation supports global quota
> -------------------------------------
>
> Key: HDFS-12934
> URL: https://issues.apache.org/jira/browse/HDFS-12934
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 3.0.0
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Labels: RBF
> Attachments: HDFS-12934.001.patch, HDFS-12934.002.patch,
> HDFS-12934.003.patch, HDFS-12934.004.patch, HDFS-12934.005.patch,
> HDFS-12934.006.patch, RBF support global quota.pdf
>
>
> Now federation doesn't support set the global quota for each folder.
> Currently the quota will be applied for each subcluster under the specified
> folder via RPC call.
> It will be very useful for users that federation can support setting global
> quota and exposing the command of this.
> In a federated environment, a folder can be spread across multiple
> subclusters. For this reason, we plan to solve this by following way:
> # Set global quota across each subcluster. We don't allow each subcluster can
> exceed maximun quota value.
> # We need to construct one <Path, QuotaUsage> cache map for storing the sum
> quota usage of these subclusters under federation folder. Every time we want
> to do WRITE operation under specified folder, we will get its quota usage
> from cache and verify its quota. If quota exceeded, throw exception,
> otherwise update its quota usage in cache when finishing operations.
> The quota will be set to mount table and as a new field in mount table. The
> set/unset command will be like:
> {noformat}
> hdfs dfsrouteradmin -setQuota -ns <nsQuota> -ss <ssQuota> <mount table>
> hdfs dfsrouteradmin -clrQuota <mount table>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
