Jason Lowe created YARN-6191:
--------------------------------
Summary: CapacityScheduler preemption by container priority can be
problematic for MapReduce
Key: YARN-6191
URL: https://issues.apache.org/jira/browse/YARN-6191
Project: Hadoop YARN
Issue Type: Bug
Components: capacityscheduler
Reporter: Jason Lowe
A MapReduce job with thousands of reducers and just a couple of maps left to go
was running in a preemptable queue. Periodically other queues would get busy
and the RM would preempt some resources from the job, but it _always_ picked
the job's map tasks first because they use the lowest priority containers.
Even though the reducers had a shorter running time, most were spared but the
maps were always shot. Since the map tasks ran for a longer time than the
preemption period, the job was in a perpetual preemption loop.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
