[
https://issues.apache.org/jira/browse/YARN-3275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14345290#comment-14345290
]
Jason Lowe commented on YARN-3275:
----------------------------------
Thanks for the patch, Eric!
bq. the expectation of our users is that if they are running a job on a
non-preemptable queue, their containers should never be preempted.
I completely agree with this. IMHO the whole point of the preemption disable
feature is to guarantee a queue marked as such will never be preempted. It's
as if the entire preemption feature was turned off from that queue's
perspective.
Looking at the patch, I'm a bit worried about this part:
{code}
+ Resource absMaxCapIdealAssignedDelta = Resource.newInstance(0, 0);
+ if (Resources.greaterThanOrEqual(
+ rc, clusterResource, maxCapacity, idealAssigned)) {
+ absMaxCapIdealAssignedDelta = Resources.subtract(maxCapacity,
idealAssigned);
+ }
{code}
If the intent of this calculation is to guarantee none of the components of
absMaxCapIdealAssignedDelta are negative then I don't believe this accomplishes
that goal. I believe it's possible for Resources.greaterThanOrEqual to return
true yet subtracting the values will result in one of the components to be
negative. For example, what if both resources are memory dominant, maxCapacity
has more memory than idealAssigned, but the opposite is true for vcores?
Subtracting idealAssigned from maxCapacity will result in a positive memory
component but a negative vcore component. If we need to make sure neither
component goes negative then I think we need to do a component-wise max with
the zero resource rather than a comparision.
Also one style nit: we normally don't do one-liner conditionals without braces,
so I'd like to see the continue explicitly put in a block. It might be useful
to put a debug log statement with the continue to note that we wanted to
preempt this queue for some reason (and by how much) but it was marked with
preemption disabled.
> CapacityScheduler: Preemption happening on non-preemptable queues
> -----------------------------------------------------------------
>
> Key: YARN-3275
> URL: https://issues.apache.org/jira/browse/YARN-3275
> Project: Hadoop YARN
> Issue Type: Bug
> Affects Versions: 2.7.0
> Reporter: Eric Payne
> Assignee: Eric Payne
> Labels: capacity-scheduler
> Attachments: YARN-3275.v1.txt
>
>
> YARN-2056 introduced the ability to turn preemption on and off at the queue
> level. In cases where a queue goes over its absolute max capacity (YARN-3243,
> for example), containers can be preempted from that queue, even though the
> queue is marked as non-preemptable.
> We are using this feature in large, busy clusters and seeing this behavior.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
