[
https://issues.apache.org/jira/browse/YARN-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14073921#comment-14073921
]
Karthik Kambatla commented on YARN-2214:
----------------------------------------
Patch looks mostly good. One nit: can we move the check in
FSLeafQueue#preemptContainer before the debug logging.
> preemptContainerPreCheck() in FSParentQueue delays convergence towards
> fairness
> -------------------------------------------------------------------------------
>
> Key: YARN-2214
> URL: https://issues.apache.org/jira/browse/YARN-2214
> Project: Hadoop YARN
> Issue Type: Bug
> Components: scheduler
> Affects Versions: 2.5.0
> Reporter: Ashwin Shankar
> Assignee: Ashwin Shankar
> Attachments: YARN-2214-v1.txt
>
>
> preemptContainerPreCheck() in FSParentQueue rejects preemption requests if
> the parent queue is below fair share. This can cause a delay in converging
> towards fairness when the starved leaf queue and the queue above fairshare
> belong under a non-root parent queue(ie their least common ancestor is a
> parent queue which is not root).
> Here is an example :
> root.parent has fair share = 80% and usage = 80%
> root.parent.child1 has fair share =40% usage = 80%
> root.parent.child2 has fair share=40% usage=0%
> Now a job is submitted to child2 and the demand is 40%.
> Preemption will kick in and try to reclaim all the 40% from child1.
> When it preempts the first container from child1,the usage of root.parent
> will become <80%, which is less than root.parent's fair share,causing
> preemption to stop.So only one container gets preempted in this round
> although the need is a lot more. child2 would eventually get to half its fair
> share but only after multiple rounds of preemption.
> Solution is to remove preemptContainerPreCheck() in FSParentQueue and keep it
> only in FSLeafQueue(which is already there).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
