[
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140295#comment-14140295
]
Wangda Tan commented on YARN-2056:
----------------------------------
Hi [~eepayne],
Thanks for updating, I've took a look at your patch and completely re-thought
about it. Some ideas:
In existing preemption logic, the {{resetCapacity}} is a static calculation,
irrelatated to resource usage of any queue.
And with result calculated by {{resetCapacity}}, {{computeFixpointAllocation}}
will make sure that: *If there's any resource isn't allocated, it will be
assigned to existing queues by its normalized_guarantee calculated*. This is
foundation of following calculation, like get {{toBePreempted}} of each queue,
we shouldn't break it.
IMHO, the right place to put reserving resource logic for un-preemptable queue
is not {{resetCapacity}}, it should in {{computeFixpointAllocation}}.
I think adding {{preemptableExtra}} in each TempQueue, and ParentQueue will
accumulate {{preemptableExtra}} of its children is a good idea.
What I proposed to do in {{computeFixpointAllocation}}
{code}
computeFixpointAllocation {
...
+ for (q in qAlloc) {
+ if (q.disablePreempt) {
+ q.ideal_assigned = q.current
+ unassigned = unassigned - q.ideal_assigned
+ }
+ }
while (!qAlloc.isEmpty()
&& Resources.greaterThan(rc, tot_guarant, unassigned, Resources.none())
&& Resources.greaterThan(rc, tot_guarant, wQassigned,
Resources.none())) {
...
}
}
{code}
And in the while (!qAlloc.isEmpty()) loop above, we need take care the fact:
some queue's ideal_assigned is started from 0, and some queue's ideal_assigned
is started from current.
Does this make sense to you?
Wangda
> Disable preemption at Queue level
> ---------------------------------
>
> Key: YARN-2056
> URL: https://issues.apache.org/jira/browse/YARN-2056
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Mayank Bansal
> Assignee: Eric Payne
> Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt,
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt,
> YARN-2056.201409181916.txt
>
>
> We need to be able to disable preemption at individual queue level
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
