[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15974454#comment-15974454
]
Sunil G commented on YARN-2113:
-------------------------------
Its an interesting corner case. Ideally deadzone will be applicable per queue
level, however there could be apps which are asking containers of various
sizes. So a fat container could still force preemption to kick in if its above
deadzone limit.
bq.However it brings up an interesting scenario where killling the youngest
container would lower the utilization below the user's limit but killing older,
smaller containers would not.
We are already trying to keep a preemption order for apps/containers. If we are
planning to configure deadzone as _no: of containers_, it will be trickier. In
whole preemption module, we were selecting containers (in its preemption order)
as last step where we have a X resource to be preempted from this queue/app. At
that time only we were trying to order apps/containers and then preempt X
resource from it.
To do like deadzone as _no: of containers_, we have to iterate the whole
containers (arranged from lowest priority to high) in apps (which are also
sorted in low -> high priority/fifo order for given *user*) to find that exact
container which will bring down to user-limit. now assume the containers are
arranged from {{container 1 ..... (m-1) m (m+1) ..... n}}.
After selecting _container m_, userX limit will be come down to its UL. So
possible preemption candidates are from {{container 1 ... (m-1)}}. Assume we
did this calculation in calculate module, but its possible that some containers
may be added/removed when we really do container selection later. (there is no
writelock for whole preemption module)
This is implementation point of view. [~jlowe] [~leftnoteasy] [~eepayne], could
u pls confirm whether I saw this in different way.
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Sunil G
> Attachments:
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
