[
https://issues.apache.org/jira/browse/YARN-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074940#comment-14074940
]
Mayank Bansal commented on YARN-2069:
-------------------------------------
HI [~wangda] ,
Thanks for the review.
Let me explain what this algo is doing .
Lets say you have queueA in your cluster with capacity 30% allocated to it.
Now Queue A is using 50% resources. Queue A has 5 users with 20% user limit.
That means with each user is using 10% of the capacity of the cluster.
Now Another queueB is there with allocated capacity 70%.
Used capacity of queue B is 50%. Now if another application gets submitted to
Queue B which needs 10% capacity.
Now 10% capacity has to be claimed back from queue A .
So restoobtain = 10%
Targated user limit will be = 8% (This is always calculated based on how much
we need to calim back from user)
So based on the current alogorithm , it will take out 2% resources from every
user and will leave behind the balance for each users.
This will also be true if all the users are not using same number of resources
so based on this algo it will take out more from the users
which are using more to balance till targated user limit.
Other thing this algo also does is it preempt application which is submitted
last that means if user1 has 2 application, it will try to take the maximum
containers from the last application submitted leaving behind the AM container
however user limit will be honoured with combined all applications in the queue.
This algo does not remove AM continer if its not absolutely needed, it goes to
get all the tasks containers first and then consider AM containers.to be
preempted.
Thanks,
Mayank
> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>
> Key: YARN-2069
> URL: https://issues.apache.org/jira/browse/YARN-2069
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: capacityscheduler
> Reporter: Vinod Kumar Vavilapalli
> Assignee: Mayank Bansal
> Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch,
> YARN-2069-trunk-3.patch, YARN-2069-trunk-4.patch, YARN-2069-trunk-5.patch,
> YARN-2069-trunk-6.patch, YARN-2069-trunk-7.patch
>
>
> This is different from (even if related to, and likely share code with)
> YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed
> capacity, it's individual users are treated in-line with their limits
> irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to
> balance queue capacities.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
