[
https://issues.apache.org/jira/browse/YARN-1921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Stepachev updated YARN-1921:
-----------------------------------
Attachment: YARN-1921.patch
> Allow to override queue prefix, where new queues created
> --------------------------------------------------------
>
> Key: YARN-1921
> URL: https://issues.apache.org/jira/browse/YARN-1921
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
> Affects Versions: 2.3.0
> Environment: Yarn 2.3.0
> Reporter: Andrey Stepachev
> Attachments: YARN-1921.patch
>
>
> Fair scheduler has a couple of QueuePlacementRules. Those rules can create
> queues, if they not exists with hardcoded prefix "root.".
> Consider an example: we have a placement rule, which creates user's queue if
> it not exists. Current implementation creates it at "root." prefix Suppose
> that this user runs a big job. In that case it will get a fair share of
> resources because queue will be created at 'root.' with default settings, and
> that affects all other users of the cluster.
> Of course, FairScheduler can place such users to default queue, but in that
> case if user submits a big queue it will eats resources of whole queue, and
> we know that no preemption can be done within one queue (Or i'm wrong?). So
> effectively one user can usurp all default queue resources.
> To solve that I created a patch, which allows to override "root." prefix in
> QueuePlacementRules. Thats gives us flexibility to automatically create
> queues for users or group of users under predefined queue. So, every user
> will get a separate queue and will share parent queue resources and can't
> usurp all resources, because parent node can be configured to preempt tasks.
> Consider example (parent queue specified for each rule):
> {code:title=policy.xml|borderStyle=solid}
> <queuePlacementPolicy>
> <rule name='specified' parent='granted'/>
> <rule name='user' parent='guests'/>
> </queuePlacementPolicy>
> {code}
> With such definition queue requirements will give us:
> {code:title=Example.java|borderStyle=solid}
> "root.granted.specifiedq" == policy.assignAppToQueue("specifiedq",
> "someuser");
> "root.guests.someuser" == policy.assignAppToQueue("default", "someuser");
> "root.guests.otheruser" == policy.assignAppToQueue("default", "otheruser");
> {code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
