[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974679#comment-15974679
]
Jason Lowe commented on YARN-2113:
----------------------------------
The more I think about this, I believe it is completely correct to preempt
containers youngest to oldest until the next container would put us at or below
the user limit. Essentially what we're doing is "rewinding" the scheduler
decisions for this user until the last container that was legitimately
allocated given the current user limit. The scheduler always allocates one
container beyond the user limit since it checks if the user is currently <= the
limit _before_ it tacks on the new container. I don't think we should consider
older containers since the order the user allocated them in (i.e.: oldest to
youngest) was "legal" given their current user limit. It's only the containers
that started beyond the user limit that are "bonus" and are candidates for
preemption.
So I don't see the need for a configurable deadzone or checking something with
the minimum allocation. It looks like we simply kill youngest to oldest until
killing the next container would put the user <= their limit.
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Sunil G
> Attachments:
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
