[
https://issues.apache.org/jira/browse/MAPREDUCE-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13120212#comment-13120212
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2702:
----------------------------------------------------
Looked at the patch. Some comments:
- I think AppAttemptId is part of the API one way or the other. If we are not
making the appAttemptId as part of the API explicitly, we should document
recoverTask() about the Id being passed as part of the context/config so that
folks with their own OuputCommitters can know.
- We should probably drop the boolean return value also and rely explicitly on
the exception, it isn't adding any value anyways.
- We need to document that if recoverTask() throws an exception, that task's
output won't be used in the new generation.
- You missed changes to the old-api that you hinted at?
- The method names got a little confusing with so many usages of 'temp', how
about simply calling them getJobDirName() and getTaskDirName()?
(Funny we never had a test-case for FileOutputCommitter, good test Sharad :) )
> [MR-279] OutputCommitter changes for MR Application Master recovery
> -------------------------------------------------------------------
>
> Key: MAPREDUCE-2702
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2702
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Components: mrv2
> Reporter: Sharad Agarwal
> Assignee: Sharad Agarwal
> Priority: Blocker
> Attachments: MAPREDUCE-2702.patch, mr2702_v1.patch, mr2702_v2.patch,
> mr2702_v3.patch, mr2702_v4.patch
>
>
> In MR AM recovers from a crash, it only reruns the non completed tasks. The
> completed tasks (along with their output, if any) needs to be recovered from
> the previous life. This would require some changes in OutputCommitter.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
