[
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-2056:
-----------------------------
Attachment: YARN-2056.201409210049.txt
Hi [~leftnoteasy]. Thank you for spending the time to look at this patch and
provide helpful suggestions.
{quote}
IMHO, the right place to put reserving resource logic for un-preemptable queue
is not {{resetCapacity}}, it should in {{computeFixpointAllocation}}.
...
Does this make sense to you?
{quote}
Yes, that makes sense, and I think it is a simpler algorithm. I updated the
patch, so please have a look.
I have made a conscious decision to only allow disable preemption at the leaf
queue level. This is because there may be a use case where you want to disable
preemption at the parent level, and have other queue hierarchies leave it
alone, but then allow preemption between children of the disabled parent. So,
rather than solve that problem with this fix, I only allow leaf queues to
disable preemption. Even if a leaf queue could inherit it's parent's disable
preemption value, there will likely be cases where part of the parent queue's
over-capacity resources are untouchable and part of them are preemptable.
So, I adjusted your suggested algorithm somewhat.
- I collected untouchableExtra instead of preemptableExtra at the TempQueue
level. in {{computeFixpointAllocation}},
- I looped through each queue, and if one has any untouchableExtra, then the
queue's {{idealAssigned = guaranteed + untouchableExtra}}
- In {{TempQueue#offer}}, one of the calculations is {{current + pending -
idealAssigned}}. I had to take into consideration that if the queue has over
capacity, some of it may be untouchable and some may be preemptable. If some of
it is preemptable, then {{current}} could be greater than {{idealAssigned}},
and {{TempQueue#offer}} would end up assigning more to that queue than it
should.
> Disable preemption at Queue level
> ---------------------------------
>
> Key: YARN-2056
> URL: https://issues.apache.org/jira/browse/YARN-2056
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Mayank Bansal
> Assignee: Eric Payne
> Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt,
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt,
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt
>
>
> We need to be able to disable preemption at individual queue level
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
