Wangda Tan commented on YARN-2056:

Hi [~eepayne],
Thanks for updating, I've took a look at your patch and completely re-thought 
about it. Some ideas:

In existing preemption logic, the {{resetCapacity}} is a static calculation, 
irrelatated to resource usage of any queue.
And with result calculated by {{resetCapacity}}, {{computeFixpointAllocation}} 
will make sure that: *If there's any resource isn't allocated, it will be 
assigned to existing queues by its normalized_guarantee calculated*. This is 
foundation of following calculation, like get {{toBePreempted}} of each queue, 
we shouldn't break it.

IMHO, the right place to put reserving resource logic for un-preemptable queue 
is not {{resetCapacity}}, it should in {{computeFixpointAllocation}}.

I think adding {{preemptableExtra}} in each TempQueue, and ParentQueue will 
accumulate {{preemptableExtra}} of its children is a good idea.

What I proposed to do in {{computeFixpointAllocation}}

computeFixpointAllocation {

+       for (q in qAlloc) {
+               if (q.disablePreempt) {
+                       q.ideal_assigned = q.current
+           unassigned = unassigned - q.ideal_assigned
+               }
+       }

    while (!qAlloc.isEmpty()
        && Resources.greaterThan(rc, tot_guarant, unassigned, Resources.none())
        && Resources.greaterThan(rc, tot_guarant, wQassigned, 
Resources.none())) {

And in the while (!qAlloc.isEmpty()) loop above, we need take care the fact: 
some queue's ideal_assigned is started from 0, and some queue's ideal_assigned 
is started from current.

Does this make sense to you?


> Disable preemption at Queue level
> ---------------------------------
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, 
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt, 
> YARN-2056.201409181916.txt
> We need to be able to disable preemption at individual queue level

This message was sent by Atlassian JIRA

Reply via email to