[
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-2056:
-----------------------------
Attachment: YARN-2056.201410132225.txt
[~leftnoteasy], Thanks for all of your help.
After looking through your suggested algorithm and thinking about it some more,
I think it is important to have a model where the most underserved queues are
given first chance at the unassigned resources. I think the algorithm should
build up each queue, reassessing its needs on every pass. Based on this, I have
rewritten the patch with the following algorithm. Plese let me know what you
think.
{code}
- Prior to assigning the unused resources, process each queue as follows:
- If current > guaranteed, idealAssigned = guaranteed + untouchable extra
Else idealAssigned = current;
- Subtract idealAssigned resources from unassigned.
- If the queue has all of its needs met (that is, if idealAssigned >= current
+ pending), remove the queue from consideration.
- Sort queues from most under-guaranteed to most over-guaranteed. Call the
this queue orderedByNeed
- While there are unsatisfied queues and some unassigned resources exist
- calculate normalized guaranteed (as today) for all remaining queues at this
hierarchical level
- Pull off the underserved queue(s) from orderedByNeed
- For each underserved queue (or set of queues if multiple are equally
underserved), offer its share of
the unassigned resources based on its normalized guarantee.
- After the offer, if the queue is not satisfied, place it back in the
ordered list of queues (orderedByNeed),
recalculating its place in the order of most under-guaranteed to most
over-guaranteed.
In this way, the most underserved queue(s) are always handled first.
{code}
> Disable preemption at Queue level
> ---------------------------------
>
> Key: YARN-2056
> URL: https://issues.apache.org/jira/browse/YARN-2056
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Mayank Bansal
> Assignee: Eric Payne
> Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt,
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt,
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt,
> YARN-2056.201409232329.txt, YARN-2056.201409242210.txt,
> YARN-2056.201410132225.txt
>
>
> We need to be able to disable preemption at individual queue level
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
