nijel created YARN-3796:
---------------------------
Summary: Support User level Quota for space and Name (count)
Key: YARN-3796
URL: https://issues.apache.org/jira/browse/YARN-3796
Project: Hadoop YARN
Issue Type: New Feature
Reporter: nijel
Assignee: nijel
I would like to have one feature in HDFS to have quota management at user
level.
Background :
When the customer uses a multi tenant solution it will have many Hadoop eco
system components like HIVE, HBASE, yarn etc. The base folder of these
components are different like /hive - Hive , /hbase -HBase.
Now if a user creates some file or table these will be under the folder
specific to component. If the user name is taken into account it looks like
{code}
/hive/user1/table1
/hive/user2/table1
/hbase/user1/Htable1
/hbase/user2/Htable1
Same for yarn/map-reduce data and logs
{code}
In this case restricting the user to use a certain amount of disk/file is very
difficult since the current quota management is at folder level.
Requirement: User level Quota for space and Name (count). Say user1 can have
100G irrespective of the folder or location used.
Here the idea to consider the file owner ad the key and attribute the quota to
it. So the current quota system can have a initial check for the user quota if
defined, before validating the folder quota.
Note:
This need a change in fsimage to store the user and quota information
Please have a look on this scenario. If it sounds good, i will create the tasks
and the update the design and prototype.
Thanks
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
