Wangda Tan commented on YARN-2069:

Hi [~mayank_bansal],
Thanks for your comments,
I think the change of title/description should be correct, this patch is 
targeted to solve cross-queue preemption should respect user-limit.

I think your other comments all make sense to me. Only below one,
bq. We need to maintian the reverse order of application submission which only 
can be done iterating through applications as we want to preempt applications 
which are last submitted.
IMHO, this is reasonable but conflict with this JIRA's scope, let me give you 
an example. 
Assume a queue has 10 apps, each app has 5 containers (1G for each container, 
so queue has 50G mem used). There're two apps, each app has 5 apps. User-limit 
is 15G, queue's absolute capacity is 30G.
And first 5 apps belongs to user-A, last 5 apps belongs to user-B.
In your correct method, user-B will be preempted 20 containers and user-A will 
be preempted nothing.
After preemption, only 5 container left for user-B, and 25 containers left for 
user-A. User-limit is respected here.

Does this make sense to you?


> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>                 Key: YARN-2069
>                 URL: https://issues.apache.org/jira/browse/YARN-2069
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch, 
> YARN-2069-trunk-3.patch
> This is different from (even if related to, and likely share code with) 
> YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed 
> capacity, it's individual users are treated in-line with their limits 
> irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to 
> balance queue capacities.

This message was sent by Atlassian JIRA

Reply via email to