[
https://issues.apache.org/jira/browse/MAPREDUCE-1924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Adam Kramer resolved MAPREDUCE-1924.
------------------------------------
Resolution: Duplicate
Yes, this is a dup of 1060.
> Mappers running when reducers have finished
> -------------------------------------------
>
> Key: MAPREDUCE-1924
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-1924
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Adam Kramer
>
> Occasionally, I will run jobs for which some reducers are able to finish but
> there are still mappers running. I understand why sometimes mappers restart
> themselves even after the reduce phase has begun--too many fetch-failures,
> for example. But in today's case, ALL of the reducers have succeeded and are
> done, so these mappers really ARE unnecessary...so it is a bug that they are
> running.
> Then, I killed one of them to see what was up--it just restarted itself. So,
> it is another bug that mappers don't know they're unnecessary when they're
> killed.
> My guess is that if one of these jobs, which clearly finished at least once,
> were to die randomly a few times, it would take the whole job with it--even
> though the job has completed.
> Whenever all reduce tasks are complete, Hadoop should kill ALL remaining map
> tasks and immediately move to finish the job.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
