Hello, YARN's FairScheduler does not have maxMaps/maxReducers fields like its MR1 counterpart did, given YARN's generic Container based architecture. Please see http://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-yarn/hadoop-yarn-site/FairScheduler.html#Configuration, sub-section "Allocation file format" for the actual configurable elements in it.
Preemption marks the tasks as killed, and not failed, so they do not count towards your job failure thresholds. If your task attempts are truly failing, their task attempt logs (syslog, stderr, stdout) will have the reason they failed. On Tue, May 20, 2014 at 12:23 PM, Du Lam <[email protected]> wrote: > Hi > some questions on yarn+fairscheduler: > 1. is the maxMaps and maxReduces in allocations.xml actually work? notice it > is not working in my setup. > 2. my job always fail with diagnostics such as: Task > task_1400033851458_4824_m_000006 failed 4 times . > is it possible that this is due to preempted too many times? or any other > issue. At the same job, there are also tasks get killed with note: Attmpt > state missing from History : marked as KILLED > > any help would be appreciated. Thanks. -- Harsh J
