[ https://issues.apache.org/jira/browse/YARN-2113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15951538#comment-15951538 ]
Jason Lowe commented on YARN-2113: ---------------------------------- The answer is no to both questions. Both users are below the 100% MULP so nobody should be cross-user preempted. Note that in your example, U2 will be the bully when it comes to getting more user limit since they have the highest priority app. Until that app stops asking, it will have first right of refusal for any resources offered to the queue. The key difference is that we aren't shooting resources to help satisfy the request, so U2 will still have to wait for resources to be freed voluntarily since no other users in the queue are above their MULP. To be clear, I definitely see why someone may want the proposed behavior. It all comes down to what people think priority means in light of multiple users. In some cases, we may just want the priority to reorder the apps but not actively shoot things as part of that reordering (i.e.: no shooting below MULP). In other cases I could see users expecting the scheduler to make room for the app regardless of what other users are doing with respect to their limit (i.e.: shooting even if below MULP). I was chatting about this with [~nroberts] and he mentioned it was in some ways similar to timesharing priorities and realtime priorities in Linux. Timesharing priorities are somewhat nice to each other, but realtime ones are not. One way to have both behaviors is to partition the priorities into those two spaces, i.e.: so important that we will shoot *anything* lower to make room priorities and "regular" priorities where we don't shoot users below the MULP. I'm not sure that's really the way to go, but it is one solution that I believe would allow users/admins to choose which behavior they want for priorities vs. user limits. > Add cross-user preemption within CapacityScheduler's leaf-queue > --------------------------------------------------------------- > > Key: YARN-2113 > URL: https://issues.apache.org/jira/browse/YARN-2113 > Project: Hadoop YARN > Issue Type: Sub-task > Components: scheduler > Reporter: Vinod Kumar Vavilapalli > Assignee: Sunil G > Attachments: > TestNoIntraQueuePreemptionIfBelowUserLimitAndDifferentPrioritiesWithExtraUsers.txt, > YARN-2113.0001.patch, YARN-2113.0002.patch, YARN-2113.0003.patch, > YARN-2113.0004.patch, YARN-2113.v0.patch > > > Preemption today only works across queues and moves around resources across > queues per demand and usage. We should also have user-level preemption within > a queue, to balance capacity across users in a predictable manner. -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org