[ https://issues.apache.org/jira/browse/YARN-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571423#comment-14571423 ]
Sunil G commented on YARN-3510: ------------------------------- Hi [~leftnoteasy] and [~cwelch] Thank you for reply. As Craig mentioned, I didnt mean the application priority here. I agree to your point that this is not a new issue as per this patch. The new change will a make more fair approach in selecting containers across applications, and not specifically from last submitted application. The scenario which I mentioned above was a case happened in our test bed. Now with new patch, App2 will have to relaunch map because it got preempted. In above example, assume we need to preempt 4 containers. Earlier App1 will kill 4 of its reducers and App2 was not loosing any container. Now we will loose a map from App2. This is a corner scenario, and in general if the new approach gives a more fair preemption, then we can move to that. :) > Create an extension of ProportionalCapacityPreemptionPolicy which preempts a > number of containers from each application in a way which respects fairness > -------------------------------------------------------------------------------------------------------------------------------------------------------- > > Key: YARN-3510 > URL: https://issues.apache.org/jira/browse/YARN-3510 > Project: Hadoop YARN > Issue Type: Sub-task > Components: yarn > Reporter: Craig Welch > Assignee: Sunil G > Attachments: YARN-3510.2.patch, YARN-3510.3.patch, YARN-3510.5.patch, > YARN-3510.6.patch > > > The ProportionalCapacityPreemptionPolicy preempts as many containers from > applications as it can during it's preemption run. For fifo this makes > sense, as it is prempting in reverse order & therefore maintaining the > primacy of the "oldest". For fair ordering this does not have the desired > effect - instead, it should preempt a number of containers from each > application which maintains a fair balance /close to a fair balance between > them -- This message was sent by Atlassian JIRA (v6.3.4#6332)