[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16119471#comment-16119471 ] Sunil G commented on YARN-2099: --- [~ywskycn], please check following [comment|https://issues.apache.org/jira/browse/YARN-2098?focusedCommentId=16119468=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16119468] > Preemption in fair scheduler should consider app priorities > --- > > Key: YARN-2099 > URL: https://issues.apache.org/jira/browse/YARN-2099 > Project: Hadoop YARN > Issue Type: Sub-task > Components: api, resourcemanager >Affects Versions: 2.5.0 >Reporter: Ashwin Shankar >Assignee: Wei Yan > Attachments: YARN-2099.patch > > > Fair scheduler should take app priorities into account while > preempting containers. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010244#comment-14010244 ] Ashwin Shankar commented on YARN-2099: -- Ah, I didn't know about YARN-596,this is very nice ! I agree with Sandy's comment. Keeping app preemption based on leaf queue's scheduling policy and having a separate policy which is purely based on priority makes sense to me. Preemption in fair scheduler should consider app priorities --- Key: YARN-2099 URL: https://issues.apache.org/jira/browse/YARN-2099 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.5.0 Reporter: Ashwin Shankar Fair scheduler should take app priorities into account while preempting containers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010251#comment-14010251 ] Wei Yan commented on YARN-2099: --- Hey, [~ashwinshankar77], Are you working on this one? If not, I would like to take it. Preemption in fair scheduler should consider app priorities --- Key: YARN-2099 URL: https://issues.apache.org/jira/browse/YARN-2099 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.5.0 Reporter: Ashwin Shankar Fair scheduler should take app priorities into account while preempting containers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14010269#comment-14010269 ] Ashwin Shankar commented on YARN-2099: -- Hey [~ywskycn], please go ahead. Preemption in fair scheduler should consider app priorities --- Key: YARN-2099 URL: https://issues.apache.org/jira/browse/YARN-2099 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.5.0 Reporter: Ashwin Shankar Fair scheduler should take app priorities into account while preempting containers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007789#comment-14007789 ] Wei Yan commented on YARN-2099: --- Hi, Ashwin. We have discussed a little bit about the priority and preemption in YARN-596. The old preemption implementation is based on container's priority, and YARN-596 starts to consider fairshare. Have one question here, consider all queues deploy fair share policy. By taking app priorities, if we want to preempt a container, which option do we deploy? (1) We collect all running apps from over-fairshare queues, and select the one with the lowest priority as the candidate, or (2) We firstly choose the queue which is most over its fair share, then the select the app with the lowest priority inside that queue as the candidate. Preemption in fair scheduler should consider app priorities --- Key: YARN-2099 URL: https://issues.apache.org/jira/browse/YARN-2099 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.5.0 Reporter: Ashwin Shankar Fair scheduler should take app priorities into account while preempting containers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007841#comment-14007841 ] Sandy Ryza commented on YARN-2099: -- I think the desired behavior here should be that a leaf queue's scheduling policy determines which apps get containers preempted inside of it. For fair share and DRF, the app to get a container preempted should be that which most exceeds its fair share. Priorities come in to play by providing weightings on the fair share. It would also be possible to add a leaf-level scheduling policy that always preempts the app with the lowest priority. I think that once YARN-596 is in we will actually have this behavior. Thoughts? Anything I'm missing? Preemption in fair scheduler should consider app priorities --- Key: YARN-2099 URL: https://issues.apache.org/jira/browse/YARN-2099 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.5.0 Reporter: Ashwin Shankar Fair scheduler should take app priorities into account while preempting containers. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2099) Preemption in fair scheduler should consider app priorities
[ https://issues.apache.org/jira/browse/YARN-2099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14007844#comment-14007844 ] Wei Yan commented on YARN-2099: --- Add a PriorityPolicy besides the existing fifo/fair/drf should work fine. Also, we also need to think about the resource allocation part. For a queue using PriorityPolicy, resource should be assigned to the applications from the highest proority. right? Preemption in fair scheduler should consider app priorities --- Key: YARN-2099 URL: https://issues.apache.org/jira/browse/YARN-2099 Project: Hadoop YARN Issue Type: Sub-task Components: api, resourcemanager Affects Versions: 2.5.0 Reporter: Ashwin Shankar Fair scheduler should take app priorities into account while preempting containers. -- This message was sent by Atlassian JIRA (v6.2#6252)
