Sunil G commented on YARN-2009:

Hi [~eepayne]

Couple of doubts on the comments.

bq.Shouldn't the following be tq.getUsed() - tq.getActuallyToBePreempted()? 
tq.getGuaranteed() only returns the queue's guaranteed capacity but if apps in 
the queue are using extra resources, then you want to subtract from the total 
If we pick on getUsed, we will be working on from a variable base value "used". 
So it will be hard to predict or calculate the preemption logic for analysis. 
But i think i am agreeing on the factor that the *used* can more that 
*guaranteed* and *unallocated* could even go negative in specific cases. So we 
could take a *max(guaranteed, used)*. Will this be fine?

bq.Shouldn't this also take into consideration used capacity of all parent 
queues as well?
I think we might need to consider only LeafQueue as we are working on each 
LeafQueue's one by one for intra-queue preemption.

bq.f user1 has used the entire queue with a low priority app, user1's headroom 
will be 0. But, if that same user starts a higher priority app, that higher 
priority app needs to preempt from the lower priority app, doesn't it?
I am doing this check in the first loop which runs from high priroity app to 
low priority app to calculate its idealAssigned. If an app's user-limit is 
already met, then we need not have to consider any more demand from that app. 
Hence such apps idealAssigned can be kept  as 0.  On the same line, assume if 
we are doing preemption here. app1 of user1 used entire queue. app2 of user2 
asks more resources. if we preempt some container from app1, will scheduler 
allocate to app2? (provided there are some demand from other users). If i am 
not wrong, it will not got. pls correct me if I am wrong.

I think this same scenario is mentioned by you in above comments. May be I 
could consider app2's demand provided there are no other user's waiting in 
queue with apps.

> Priority support for preemption in ProportionalCapacityPreemptionPolicy
> -----------------------------------------------------------------------
>                 Key: YARN-2009
>                 URL: https://issues.apache.org/jira/browse/YARN-2009
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Devaraj K
>            Assignee: Sunil G
>         Attachments: YARN-2009.0001.patch, YARN-2009.0002.patch
> While preempting containers based on the queue ideal assignment, we may need 
> to consider preempting the low priority application containers first.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org

Reply via email to