[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sunil G updated YARN-2113:
--------------------------
Attachment: YARN-2113.0012.patch
To summarize the change in this patch, we were earlier trying to find the
container which might bring user's resource usage down to its user-limit. This
works well when there is an anomaly of resources among many users. We were
using USERLIMIT_FIRST order to handle same.
Now within each user, there might be resource demand from high priority apps.
Hence I improved the check (to select the fat container which push user's usage
under UL) as below
1. This check must be done only when preemption order is USERLIMIT_FIRST
2. By selecting container "c", check whether this user's resource usage is
going below its user-limit.
3. Used resource of user must be always greater than user-limit to skip some
containers as per this check. If used resource is under user limit, then these
containers of this user has to be preempted as demand might be due to high
priority apps running in same user.
With the help of this, we can handle user-limit and priority based preemption
together by giving weight over user-limit first. Thoughts?
Ref Code: {{IntraQueueCandidatesSelector.isResourceUsageUnderUserLimit}}
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Sunil G
> Attachments: IntraQueue Preemption-Impact Analysis.pdf,
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch,
> YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch,
> YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
