[ 
https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142404#comment-14142404
 ] 

Wangda Tan commented on YARN-2056:
----------------------------------

Hi [~eepayne],
Thanks for updating, I took a look at your latest patch, some comments,
bq. Even if a leaf queue could inherit it's parent's disable preemption value, 
there will likely be cases where part of the parent queue's over-capacity 
resources are untouchable and part of them are preemptable.
I'm not quite understand about this, is there any issue if we let 
"disable-preemption" of parent queue is only a default value, and its children 
can overwrite it?

bq. I collected untouchableExtra instead of preemptableExtra at the TempQueue 
level. in computeFixpointAllocation,
Make sense to me

With
bq. In TempQueue#offer, one of the calculations is current + pending - 
idealAssigned ... and TempQueue#offer would end up assigning more to that queue 
than it should.
bq. I looped through each queue, and if one has any untouchableExtra, then the 
queue's idealAssigned = guaranteed + untouchableExtra
A problem I can see is, a negtive example:

{code}
root has A,B,C, total capacity = 90
A.guaranteed = 30, A.pending = 20, A.current = 40
B.guaranteed = 30, B.pending = 0, B.current = 50
C.guaranteed = 30, C.pending = 0, C.current = 0

initial:
A.idealAssigned = 40
B.idealAssigned = 0
unassigned = 50 (90 - 40)

1st round (A/B/C's normalized_guarantee = 1/3)
A.idealAssigned = 40 (unchanged because changes of TempQueue#offer 
implementation, A will be removed)
B.idealAssigned = 17 (50/3)
C.idealAssigned = 0 (satisfied already, C will be removed)
unassigned = 33

2nd round (A/B's normalized_guarantee = 0.5)
B.idealAssigned = 17 + 33 = 50
unassigned = 0
{code}

However, with A/B has same guarantee, they should end up with both 
ideal_assigned = 45. And A will be able to preempt 5 from B. So in a short 
word, the problem is, a disable_preemption queue cannot preempt more resource 
than its current when some queues in the cluster don't consume resource. Do you 
have any idea to solve this problem?

Thanks,
Wangda

> Disable preemption at Queue level
> ---------------------------------
>
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt, YARN-2056.201408260128.txt, 
> YARN-2056.201408310117.txt, YARN-2056.201409022208.txt, 
> YARN-2056.201409181916.txt, YARN-2056.201409210049.txt
>
>
> We need to be able to disable preemption at individual queue level



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to