[ https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974679#comment-15974679 ]
Jason Lowe commented on YARN-2113: ---------------------------------- The more I think about this, I believe it is completely correct to preempt containers youngest to oldest until the next container would put us at or below the user limit. Essentially what we're doing is "rewinding" the scheduler decisions for this user until the last container that was legitimately allocated given the current user limit. The scheduler always allocates one container beyond the user limit since it checks if the user is currently <= the limit _before_ it tacks on the new container. I don't think we should consider older containers since the order the user allocated them in (i.e.: oldest to youngest) was "legal" given their current user limit. It's only the containers that started beyond the user limit that are "bonus" and are candidates for preemption. So I don't see the need for a configurable deadzone or checking something with the minimum allocation. It looks like we simply kill youngest to oldest until killing the next container would put the user <= their limit. > Add cross-user preemption within CapacityScheduler's leaf-queue > --------------------------------------------------------------- > > Key: YARN-2113 > URL: https://issues.apache.org/jira/browse/YARN-2113 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler > Reporter: Vinod Kumar Vavilapalli > Assignee: Sunil G > Attachments: > TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt, > YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch, > YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch, > YARN-2113.0007.patch, YARN-2113.v0.patch > > > Preemption today only works across queues and moves around resources across > queues per demand and usage. We should also have user-level preemption within > a queue, to balance capacity across users in a predictable manner. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org