[
https://issues.apache.org/jira/browse/MAPREDUCE-11?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Amar Kamat resolved MAPREDUCE-11.
---------------------------------
Resolution: Invalid
Job history is no longer used for recovery and hence this issue is no longer
valid.
> Cleanup JobHistory file naming to do with job recovery
> ------------------------------------------------------
>
> Key: MAPREDUCE-11
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-11
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: Devaraj Das
> Attachments: MAPREDUCE-11-v1.8.patch
>
>
> The JobTracker uses the job history files for doing job recovery upon
> startup. To handle cases where JobTracker goes down again while the recovered
> job is running, there is some logic that plays with files and it ends up
> having two history files for some window of time during the life of the job -
> actual history file, .recover file. The idea being that upon the next restart
> we should be able to the maximal number of events for the job. It led to
> performance problems in the job submission / recovery (part of which got
> addressed in HADOOP-4372). It also looks pretty unlikely that a running job
> will traverse across multiple JT restarts. Even if it did, without the
> .recover file, it'd only mean that we lose some tasks that got completed in a
> subsequent restart. I propose that we remove the .recover file logic and base
> the recovery on only the original job history file.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
