[
https://issues.apache.org/jira/browse/YARN-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14570664#comment-14570664
]
Sunil G commented on YARN-3510:
-------------------------------
Hi [~cwelch]
Thanks for taking this optimization up. I have few doubts here.
Here an evenly distributed preempting policy across applications are tried. But
each application internally has containers from different priorities, and least
priority container is selected first from an application for preemption.
Now consider a scenario where we have 2 applications (assuming map reduce).
{noformat}
App1 has containers <10 containers:Priority 10, 5 containers:Priority 20> Old
timestamp
App2 has containers <10 containers:Priority 10, 2 containers:Priority 20> New
timestamp
{noformat}
As per new implementation, after 2 rounds, some containers of priority 10(maps)
may get preempted if I am not wrong. Is this intentional, because killing maps
is costlier.
I feel, we can group containers based on priority among all applications, and
then can do this preemption at each container priority level. It may be more
better but we may have more buckets of priorities. Please share your thoughts.
> Create an extension of ProportionalCapacityPreemptionPolicy which preempts a
> number of containers from each application in a way which respects fairness
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: YARN-3510
> URL: https://issues.apache.org/jira/browse/YARN-3510
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: yarn
> Reporter: Craig Welch
> Assignee: Craig Welch
> Attachments: YARN-3510.2.patch, YARN-3510.3.patch, YARN-3510.5.patch,
> YARN-3510.6.patch
>
>
> The ProportionalCapacityPreemptionPolicy preempts as many containers from
> applications as it can during it's preemption run. For fifo this makes
> sense, as it is prempting in reverse order & therefore maintaining the
> primacy of the "oldest". For fair ordering this does not have the desired
> effect - instead, it should preempt a number of containers from each
> application which maintains a fair balance /close to a fair balance between
> them
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
