[ https://issues.apache.org/jira/browse/YARN-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ashwin Shankar reassigned YARN-1961: ------------------------------------ Assignee: Ashwin Shankar > Fair scheduler preemption doesn't work for non-leaf queues > ---------------------------------------------------------- > > Key: YARN-1961 > URL: https://issues.apache.org/jira/browse/YARN-1961 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler > Affects Versions: 2.4.0 > Reporter: Ashwin Shankar > Assignee: Ashwin Shankar > Labels: scheduler > > Setting minResources and minSharePreemptionTimeout to a non-leaf queue > doesn't cause preemption to happen when that non-leaf queue is below > minResources and there are outstanding demands in that non-leaf queue. > Here is an example fs allocation config(partial) : > {code:xml} > <queue name="abc"> > <minResources>3072 mb,0 vcores</minResources> > <minSharePreemptionTimeout>30</minSharePreemptionTimeout> > <queue name="childabc1"> > </queue> > <queue name="childabc2"> > </queue> > </queue> > {code} > With the above configs,preemption doesn't seem to happen if queue abc is > below minShare and it has outstanding unsatisfied demands from apps in its > child queues. Ideally in such cases we would like preemption to kick off and > reclaim resources from other queues(not under queue abc). > Looking at the code it seems like preemption checks for starvation only at > the leaf queue level and not at the parent level. > {code:title=FairScheduler.java|borderStyle=solid} > boolean isStarvedForMinShare(FSLeafQueue sched) > boolean isStarvedForFairShare(FSLeafQueue sched) > {code} > This affects our use case where we have a parent queue with probably a 100 > unconfigured leaf queues under it.We want to give a minshare to the parent > queue to protect all the leaf queues under it,but we cannot do it due to this > bug. -- This message was sent by Atlassian JIRA (v6.2#6252)