[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15991126#comment-15991126
]
Eric Payne commented on YARN-2113:
----------------------------------
Thanks [~leftnoteasy].
{quote}
1) Priority first:
- App1 can preempt App2 if:
-- When App1.priority > App2.priority
--- User of app1 not reached user-limit
{quote}
I believe that the above statement is too broad. The above statement sounds
like preemption will bring App2.user down below its user limit even when other
users are above theirs. I would like to ensure that App2.user is not brought
below its user limit unless all other users are already at or below their user
limit.
{quote}
2) User limit first:
- App1 can preempt app2 if:
-- app1.user != app2.user
{quote}
Even in USERLIMIT_FIRST mode, shouldn't App1 preempt from App2 if App1.priority
> App2.priority AND App1.user == App2.user?
bq. No matter if app1.user is outside of #100/MULP or not, no change to
preemption behavior
You are right.
{quote}
If I understand Jason's comment correctly: above behavior should support both
scenarios via different policies:
{quote}
I would like to hear [~jlowe]'s comments, but IIUC, he would have the same
concern as above. I would also like his analysis of the following.
I see preemption requirements as follows (and maybe from the other direction):
For both USERLIMIT_FIRST and PRIORITY_FIRST:
- preemption for App1 should happen while App1.user is below its user limit AND
in the following order of importance:
-- Preempt from apps owned by over-user-limit users:
--- Preempt lowest priority containers in youngest-to-oldest order of
over-user-limit users (maybe even put containers first from
highest-over-user-limit users).
-- Preempt from lower-priority apps owned by App1.user
--- When there are no more users over their user limit AND lower priority apps
owned by App1.user exist, preempt lowest-priority containers in
youngest-to-oldest order
For PRIORITY_FIRST
- Continue to preempt lower priority non-AM containers in lowest-to-highest and
youngest-to-oldest order until App1.user has reached its user limit.
For USERLIMIT_FIRST
- no further preemption occurs
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Sunil G
> Attachments: IntraQueue Preemption-Impact Analysis.pdf,
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.0005.patch, YARN-2113.0006.patch,
> YARN-2113.0007.patch, YARN-2113.0008.patch, YARN-2113.0009.patch,
> YARN-2113.0010.patch, YARN-2113.0011.patch, YARN-2113.0012.patch,
> YARN-2113.0013.patch, YARN-2113.apply.onto.0012.ericp.patch,
> YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
