Xuan Gong updated YARN-1961:
    Component/s: fairscheduler

> Fair scheduler preemption doesn't work for non-leaf queues
> ----------------------------------------------------------
>                 Key: YARN-1961
>                 URL: https://issues.apache.org/jira/browse/YARN-1961
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler, scheduler
>    Affects Versions: 2.4.0
>            Reporter: Ashwin Shankar
>            Assignee: Ashwin Shankar
>              Labels: scheduler
> Setting minResources and minSharePreemptionTimeout to a non-leaf queue 
> doesn't cause preemption to happen when that non-leaf queue is below 
> minResources and there are outstanding demands in that non-leaf queue.
> Here is an example fs allocation config(partial) :
> {code:xml}
> <queue name="abc">
>   <minResources>3072 mb,0 vcores</minResources>
>   <minSharePreemptionTimeout>30</minSharePreemptionTimeout>
>     <queue name="childabc1">
>     </queue>
>     <queue name="childabc2">
>     </queue>
>  </queue>
>  {code}
> With the above configs,preemption doesn't seem to happen if queue abc is 
> below minShare and it has outstanding unsatisfied demands from apps in its 
> child queues. Ideally in such cases we would like preemption to kick off and 
> reclaim resources from other queues(not under queue abc).
> Looking at the code it seems like preemption checks for starvation only at 
> the leaf queue level and not at the parent level.
> {code:title=FairScheduler.java|borderStyle=solid}
> boolean isStarvedForMinShare(FSLeafQueue sched)
> boolean isStarvedForFairShare(FSLeafQueue sched)
> {code}
> This affects our use case where we have a parent queue with probably a 100 
> unconfigured leaf queues under it.We want to give a minshare to the parent 
> queue to protect all the leaf queues under it,but we cannot do it due to this 
> bug.

This message was sent by Atlassian JIRA

Reply via email to