[
https://issues.apache.org/jira/browse/HDFS-12934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Lin updated HDFS-12934:
-----------------------------
Description:
Now ferderation doesn't support set the global quota for each folder. Currently
the qupta will be applied for each subcluster under the specified folder via
RPC call.
It will be very useful for users that ferderation can support setting global
quota and exposing the command of this.
In a federated environment, a folder can be spread across multiple subclusters.
For this reason, we plan to solve this by following way:
# Set global quota across each subcluster. We don't allow each subcluster cab
exceed maximun quota value.
# We need to contruct one <Path, QuataUsage> cache map for storing current
quota usage of these subclusters, Every time we want to do WRITE operation
under specifed folder, we will get its quota usage from cache and verify its
quota. If quota exceeded, throw exception, otherwise update its quota usage in
cache when finishing operations.
The quota will be set to mount table and as a new field in mount table. The
set/unset command will be like:
{noformat}
hdfs dfsrouteradmin -setQuota <nsQuota> <ssQuota> <mount table>
hdfs dfsrouteradmin -clrQuota <mount table>
{noformat}
> RBF: Federation supports global quota
> -------------------------------------
>
> Key: HDFS-12934
> URL: https://issues.apache.org/jira/browse/HDFS-12934
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Affects Versions: 3.0.0
> Reporter: Yiqun Lin
> Assignee: Yiqun Lin
> Labels: RBF
>
> Now ferderation doesn't support set the global quota for each folder.
> Currently the qupta will be applied for each subcluster under the specified
> folder via RPC call.
> It will be very useful for users that ferderation can support setting global
> quota and exposing the command of this.
> In a federated environment, a folder can be spread across multiple
> subclusters. For this reason, we plan to solve this by following way:
> # Set global quota across each subcluster. We don't allow each subcluster cab
> exceed maximun quota value.
> # We need to contruct one <Path, QuataUsage> cache map for storing current
> quota usage of these subclusters, Every time we want to do WRITE operation
> under specifed folder, we will get its quota usage from cache and verify its
> quota. If quota exceeded, throw exception, otherwise update its quota usage
> in cache when finishing operations.
> The quota will be set to mount table and as a new field in mount table. The
> set/unset command will be like:
> {noformat}
> hdfs dfsrouteradmin -setQuota <nsQuota> <ssQuota> <mount table>
> hdfs dfsrouteradmin -clrQuota <mount table>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
