[
https://issues.apache.org/jira/browse/YARN-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated YARN-1961:
-----------------------------------
Issue Type: Sub-task (was: Bug)
Parent: YARN-4752
> Fair scheduler preemption doesn't work for non-leaf queues
> ----------------------------------------------------------
>
> Key: YARN-1961
> URL: https://issues.apache.org/jira/browse/YARN-1961
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler, scheduler
> Affects Versions: 2.4.0
> Reporter: Ashwin Shankar
> Assignee: Ashwin Shankar
> Labels: scheduler
>
> Setting minResources and minSharePreemptionTimeout to a non-leaf queue
> doesn't cause preemption to happen when that non-leaf queue is below
> minResources and there are outstanding demands in that non-leaf queue.
> Here is an example fs allocation config(partial) :
> {code:xml}
> <queue name="abc">
> <minResources>3072 mb,0 vcores</minResources>
> <minSharePreemptionTimeout>30</minSharePreemptionTimeout>
> <queue name="childabc1">
> </queue>
> <queue name="childabc2">
> </queue>
> </queue>
> {code}
> With the above configs,preemption doesn't seem to happen if queue abc is
> below minShare and it has outstanding unsatisfied demands from apps in its
> child queues. Ideally in such cases we would like preemption to kick off and
> reclaim resources from other queues(not under queue abc).
> Looking at the code it seems like preemption checks for starvation only at
> the leaf queue level and not at the parent level.
> {code:title=FairScheduler.java|borderStyle=solid}
> boolean isStarvedForMinShare(FSLeafQueue sched)
> boolean isStarvedForFairShare(FSLeafQueue sched)
> {code}
> This affects our use case where we have a parent queue with probably a 100
> unconfigured leaf queues under it.We want to give a minshare to the parent
> queue to protect all the leaf queues under it,but we cannot do it due to this
> bug.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
