[
https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951538#comment-15951538
]
Jason Lowe commented on YARN-2113:
----------------------------------
The answer is no to both questions. Both users are below the 100% MULP so
nobody should be cross-user preempted. Note that in your example, U2 will be
the bully when it comes to getting more user limit since they have the highest
priority app. Until that app stops asking, it will have first right of refusal
for any resources offered to the queue. The key difference is that we aren't
shooting resources to help satisfy the request, so U2 will still have to wait
for resources to be freed voluntarily since no other users in the queue are
above their MULP.
To be clear, I definitely see why someone may want the proposed behavior. It
all comes down to what people think priority means in light of multiple users.
In some cases, we may just want the priority to reorder the apps but not
actively shoot things as part of that reordering (i.e.: no shooting below
MULP). In other cases I could see users expecting the scheduler to make room
for the app regardless of what other users are doing with respect to their
limit (i.e.: shooting even if below MULP). I was chatting about this with
[~nroberts] and he mentioned it was in some ways similar to timesharing
priorities and realtime priorities in Linux. Timesharing priorities are
somewhat nice to each other, but realtime ones are not. One way to have both
behaviors is to partition the priorities into those two spaces, i.e.: so
important that we will shoot *anything* lower to make room priorities and
"regular" priorities where we don't shoot users below the MULP. I'm not sure
that's really the way to go, but it is one solution that I believe would allow
users/admins to choose which behavior they want for priorities vs. user limits.
> Add cross-user preemption within CapacityScheduler's leaf-queue
> ---------------------------------------------------------------
>
> Key: YARN-2113
> URL: https://issues.apache.org/jira/browse/YARN-2113
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: scheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Sunil G
> Attachments:
> TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt,
> YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch,
> YARN-2113.0004.patch, YARN-2113.v0.patch
>
>
> Preemption today only works across queues and moves around resources across
> queues per demand and usage. We should also have user-level preemption within
> a queue, to balance capacity across users in a predictable manner.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
