Ashwin Shankar created YARN-1864:
------------------------------------
Summary: Fair Scheduler Dynamic Hierarchical User Queues
Key: YARN-1864
URL: https://issues.apache.org/jira/browse/YARN-1864
Project: Hadoop YARN
Issue Type: New Feature
Components: scheduler
Reporter: Ashwin Shankar
Fix For: 2.4.0
In Fair Scheduler, we want to be able to create user queues under any parent
queue in the hierarchy. For eg. Say user1 submits a job to a parent queue
called root.allUserQueues, we want be able to create a new queue called
root.allUserQueues.user1 and run user1's job in it.Any further jobs submitted
by this user to root.allUserQueues will be run in this newly created
root.allUserQueues.user1.
This is very similar to the 'user-as-default' feature in Fair Scheduler which
creates user queues under root queue. But we want the ability to create user
queues under ANY parent queue.
Why do we want this ?
1. Preemption : these dynamically created user queues can preempt each other if
its fair share is not met. So there is fairness among users.
2. Allocation to user queues : we want all the user queries(adhoc) to consume
only a fraction of resources in the shared cluster. By creating this feature,we
could do that by giving a fair share to the parent user queue which is then
redistributed to all the dynamically created user queues.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
