[
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142404#comment-14142404
]
Wangda Tan commented on YARN-2056:
----------------------------------
Hi [~eepayne],
Thanks for updating, I took a look at your latest patch, some comments,
bq. Even if a leaf queue could inherit it's parent's disable preemption value,
there will likely be cases where part of the parent queue's over-capacity
resources are untouchable and part of them are preemptable.
I'm not quite understand about this, is there any issue if we let
"disable-preemption" of parent queue is only a default value, and its children
can overwrite it?
bq. I collected untouchableExtra instead of preemptableExtra at the TempQueue
level. in computeFixpointAllocation,
Make sense to me
With
bq. In TempQueue#offer, one of the calculations is current + pending -
idealAssigned ... and TempQueue#offer would end up assigning more to that queue
than it should.
bq. I looped through each queue, and if one has any untouchableExtra, then the
queue's idealAssigned = guaranteed + untouchableExtra
A problem I can see is, a negtive example:
{code}
root has A,B,C, total capacity = 90
A.guaranteed = 30, A.pending = 20, A.current = 40
B.guaranteed = 30, B.pending = 0, B.current = 50
C.guaranteed = 30, C.pending = 0, C.current = 0
initial:
A.idealAssigned = 40
B.idealAssigned = 0
unassigned = 50 (90 - 40)
1st round (A/B/C's normalized_guarantee = 1/3)
A.idealAssigned = 40 (unchanged because changes of TempQueue#offer
implementation, A will be removed)
B.idealAssigned = 17 (50/3)
C.idealAssigned = 0 (satisfied already, C will be removed)
unassigned = 33
2nd round (A/B's normalized_guarantee = 0.5)
B.idealAssigned = 17 + 33 = 50
unassigned = 0
{code}
However, with A/B has same guarantee, they should end up with both
ideal_assigned = 45. And A will be able to preempt 5 from B. So in a short
word, the problem is, a disable_preemption queue cannot preempt more resource
than its current when some queues in the cluster don't consume resource. Do you
have any idea to solve this problem?
Thanks,
Wangda
> Disable preemption at Queue level
> ---------------------------------
>
> Key: YARN-2056
> URL: https://issues.apache.org/jira/browse/YARN-2056
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: resourcemanager
> Affects Versions: 2.4.0
> Reporter: Mayank Bansal
> Assignee: Eric Payne
> Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt,
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt,
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt
>
>
> We need to be able to disable preemption at individual queue level
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
