[
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne updated YARN-2056:
-----------------------------
Attachment: YARN-2056.201409232329.txt
[~leftnoteasy],
This patch now works in all cases, except IF queue preemption is disabled for a
particular queue, it has a different behavior than previously. That is:
{code}
root has A,B,C, total capacity = 90
A.guaranteed = 30, A.pending = 20, A.current = 40
B.guaranteed = 30, B.pending = 0, B.current = 50
C.guaranteed = 30, C.pending = 0, C.current = 0
{code}
In the above case, if all queues are preemptable, this patch works the same as
it did before. That is, A and B will both end up with 45.
However, with this patch
{code}
IF (A is not preemptable)
AND (A is already over capacity)
AND (all resources are used)
AND (A is asking for more resources) {
A will remain at 40 and B will remain at 50
}
{code}
I believe that there is a way to make the this patch maintain the old behavior,
even if A is not preemptable, but it would require something like the following
algorithm:
{code}
In ProportionalCapacityPreemptionPolicy#computeFixpointAllocation:
FOR each queue {
IF queue has untouchableExtra {
queue.idalAssigned = queue.guaranteed + queue.untouchable
unassigned -= queue.idealAssigned
remove queue from qAlloc
add queue to list of queues that were removed
}
}
Assign the remaining unassigned resources, computing idealAssigned for the
remaining queues, with the following modification:
IF (queues at this level go over their capacity)
AND (they are over by the same percentage as the queue(s) that were
removed) {
put the removed queue(s) back into qAlloc and continue to compute
idealAssigned
}
{code}
> Disable preemption at Queue level
> ---------------------------------
>
> Key: YARN-2056
> URL: https://issues.apache.org/jira/browse/YARN-2056
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Mayank Bansal
> Assignee: Eric Payne
> Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt,
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt,
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt,
> YARN-2056.201409232329.txt
>
>
> We need to be able to disable preemption at individual queue level
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
