Eric Payne updated YARN-2056:
    Attachment: YARN-2056.201410132225.txt

[~leftnoteasy], Thanks for all of your help.

After looking through your suggested algorithm and thinking about it some more, 
I think it is important to have a model where the most underserved queues are 
given first chance at the unassigned resources. I think the algorithm should 
build up each queue, reassessing its needs on every pass. Based on this, I have 
rewritten the patch with the following algorithm. Plese let me know what you 
- Prior to assigning the unused resources, process each queue as follows:
  - If current > guaranteed, idealAssigned = guaranteed + untouchable extra
    Else idealAssigned = current;
  - Subtract idealAssigned resources from unassigned.
  - If the queue has all of its needs met (that is, if idealAssigned >= current 
+ pending), remove the queue from consideration.
  - Sort queues from most under-guaranteed to most over-guaranteed. Call the 
this queue orderedByNeed
- While there are unsatisfied queues and some unassigned resources exist
  - calculate normalized guaranteed (as today) for all remaining queues at this 
hierarchical level
  - Pull off the underserved queue(s) from orderedByNeed
  - For each underserved queue (or set of queues if multiple are equally 
underserved), offer its share of
    the unassigned resources based on its normalized guarantee.
      - After the offer, if the queue is not satisfied, place it back in the 
ordered list of queues (orderedByNeed),
      recalculating its place  in the order of most under-guaranteed to most 
In this  way, the most underserved queue(s) are always handled first.

> Disable preemption at Queue level
> ---------------------------------
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, 
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt, 
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt, 
> YARN-2056.201409232329.txt, YARN-2056.201409242210.txt, 
> YARN-2056.201410132225.txt
> We need to be able to disable preemption at individual queue level

This message was sent by Atlassian JIRA

Reply via email to