[
https://issues.apache.org/jira/browse/YARN-3054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15192683#comment-15192683
]
Peng Zhang commented on YARN-3054:
----------------------------------
Thanks [~kasha]
I agreed with "have a preemption priority or even a preemption cost per
container".
And in my temporary fix, I preempt latest scheduled container instead of low or
high priority containers.
I think this will make containers for a mount of resources(at least steady fair
share) steady. So MapReduce job progress will proceed.
> Preempt policy in FairScheduler may cause mapreduce job never finish
> --------------------------------------------------------------------
>
> Key: YARN-3054
> URL: https://issues.apache.org/jira/browse/YARN-3054
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: fairscheduler
> Affects Versions: 2.6.0
> Reporter: Peng Zhang
>
> Preemption policy is related with schedule policy now. Using comparator of
> schedule policy to find preemption candidate cannot guarantee a subset of
> containers never be preempted. And this may cause tasks to be preempted
> periodically before they finish. So job cannot make any progress.
> I think preemption in YARN should got below assurance:
> 1. Mapreduce jobs can get additional resources when others are idle;
> 2. Mapreduce jobs for one user in one queue can still progress with its min
> share when others preempt resources back.
> Maybe always preempt the latest app and container can get this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
