Peng Zhang created YARN-3054:
--------------------------------
Summary: Preempt policy in FairScheduler may cause mapreduce job
never finish
Key: YARN-3054
URL: https://issues.apache.org/jira/browse/YARN-3054
Project: Hadoop YARN
Issue Type: Bug
Components: fairscheduler
Affects Versions: 2.6.0
Reporter: Peng Zhang
Preemption policy is related with schedule policy now. Using comparator of
schedule policy to find preemption candidate cannot guarantee a subset of
containers never be preempted. And this may cause tasks to be preempted
periodically before they finish. So job cannot make any progress.
I think preemption in YARN should got below assurance:
1. Mapreduce jobs can get additional resources when others are idle;
2. Mapreduce jobs for one user in one queue can still progress with its min
share when others preempt resources back.
Maybe always preempt the latest app and container can get this?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
