[jira] [Updated] (YARN-2214) preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness
[ https://issues.apache.org/jira/browse/YARN-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Shankar updated YARN-2214: - Attachment: YARN-2214-v2.txt > preemptContainerPreCheck() in FSParentQueue delays convergence towards > fairness > --- > > Key: YARN-2214 > URL: https://issues.apache.org/jira/browse/YARN-2214 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Affects Versions: 2.5.0 >Reporter: Ashwin Shankar >Assignee: Ashwin Shankar > Attachments: YARN-2214-v1.txt, YARN-2214-v2.txt > > > preemptContainerPreCheck() in FSParentQueue rejects preemption requests if > the parent queue is below fair share. This can cause a delay in converging > towards fairness when the starved leaf queue and the queue above fairshare > belong under a non-root parent queue(ie their least common ancestor is a > parent queue which is not root). > Here is an example : > root.parent has fair share = 80% and usage = 80% > root.parent.child1 has fair share =40% usage = 80% > root.parent.child2 has fair share=40% usage=0% > Now a job is submitted to child2 and the demand is 40%. > Preemption will kick in and try to reclaim all the 40% from child1. > When it preempts the first container from child1,the usage of root.parent > will become <80%, which is less than root.parent's fair share,causing > preemption to stop.So only one container gets preempted in this round > although the need is a lot more. child2 would eventually get to half its fair > share but only after multiple rounds of preemption. > Solution is to remove preemptContainerPreCheck() in FSParentQueue and keep it > only in FSLeafQueue(which is already there). -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (YARN-2214) preemptContainerPreCheck() in FSParentQueue delays convergence towards fairness
[ https://issues.apache.org/jira/browse/YARN-2214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashwin Shankar updated YARN-2214: - Attachment: YARN-2214-v1.txt Attached patch removes preemptContainerPreCheck from FSParentQueue.Added a unit test which proves the case mentioned in the description. > preemptContainerPreCheck() in FSParentQueue delays convergence towards > fairness > --- > > Key: YARN-2214 > URL: https://issues.apache.org/jira/browse/YARN-2214 > Project: Hadoop YARN > Issue Type: Bug > Components: scheduler >Affects Versions: 2.5.0 >Reporter: Ashwin Shankar >Assignee: Ashwin Shankar > Attachments: YARN-2214-v1.txt > > > preemptContainerPreCheck() in FSParentQueue rejects preemption requests if > the parent queue is below fair share. This can cause a delay in converging > towards fairness when the starved leaf queue and the queue above fairshare > belong under a non-root parent queue(ie their least common ancestor is a > parent queue which is not root). > Here is an example : > root.parent has fair share = 80% and usage = 80% > root.parent.child1 has fair share =40% usage = 80% > root.parent.child2 has fair share=40% usage=0% > Now a job is submitted to child2 and the demand is 40%. > Preemption will kick in and try to reclaim all the 40% from child1. > When it preempts the first container from child1,the usage of root.parent > will become <80%, which is less than root.parent's fair share,causing > preemption to stop.So only one container gets preempted in this round > although the need is a lot more. child2 would eventually get to half its fair > share but only after multiple rounds of preemption. > Solution is to remove preemptContainerPreCheck() in FSParentQueue and keep it > only in FSLeafQueue(which is already there). -- This message was sent by Atlassian JIRA (v6.2#6252)
