Zhe Zhang created MAPREDUCE-6870:
------------------------------------
Summary: Add configuration for MR job to finish when all reducers
are complete (even with unfinished mappers)
Key: MAPREDUCE-6870
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6870
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.6.1
Reporter: Zhe Zhang
Even with MAPREDUCE-5817, there could still be cases where mappers get
scheduled before all reducers are complete, but those mappers run for long
time, even after all reducers are complete. This could hurt the performance of
large MR jobs.
In some cases, mappers don't have any materialize-able outcome other than
providing intermediate data to reducers. In that case, the job owner should
have the config option to finish the job once all reducers are complete.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
