[ 
https://issues.apache.org/jira/browse/YARN-2022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14028667#comment-14028667
 ] 

Wangda Tan commented on YARN-2022:
----------------------------------

Thanks for this summary, [~vinodkv]
As you mentioned, we cannot do without preempting AMs, some other restrictions 
will be violated if we do so. I'm also agree to solve YARN-2074.
You could check my comment: 
https://issues.apache.org/jira/browse/YARN-2074?focusedCommentId=14019510&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14019510.
 This is the minimum effort I can think to solve YARN-2074. For each time AM 
registering, a number of failure count (doesn't consider preemption-caused 
failure)  will be responded to AM, and AM can check if itself is the last 
attempt.
Any thoughts? 

> Preempting an Application Master container can be kept as least priority when 
> multiple applications are marked for preemption by 
> ProportionalCapacityPreemptionPolicy
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2022
>                 URL: https://issues.apache.org/jira/browse/YARN-2022
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: YARN-2022-DesignDraft.docx, YARN-2022.2.patch, 
> YARN-2022.3.patch, YARN-2022.4.patch, Yarn-2022.1.patch
>
>
> Cluster Size = 16GB [2NM's]
> Queue A Capacity = 50%
> Queue B Capacity = 50%
> Consider there are 3 applications running in Queue A which has taken the full 
> cluster capacity. 
> J1 = 2GB AM + 1GB * 4 Maps
> J2 = 2GB AM + 1GB * 4 Maps
> J3 = 2GB AM + 1GB * 2 Maps
> Another Job J4 is submitted in Queue B [J4 needs a 2GB AM + 1GB * 2 Maps ].
> Currently in this scenario, Jobs J3 will get killed including its AM.
> It is better if AM can be given least priority among multiple applications. 
> In this same scenario, map tasks from J3 and J2 can be preempted.
> Later when cluster is free, maps can be allocated to these Jobs.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to