[
https://issues.apache.org/jira/browse/PIG-3065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13696154#comment-13696154
]
Rohini Palaniswamy commented on PIG-3065:
-----------------------------------------
Look at HadoopShims classes. You can add a method like
public boolean isRecoverySupported(OutputCommitter committer) {
return committer.isRecoverySupported();
}
In 0.20/1.x HadoopShims, the method can just return false always. The AM
restart/recovery is still not as stable as one would hope to be at the moment
and we have seen it lead to data corruption/loss. So please also add a
configuration property for this and have it turned off by default.
> pig output format/committer should support recovery for hadoop 0.23
> -------------------------------------------------------------------
>
> Key: PIG-3065
> URL: https://issues.apache.org/jira/browse/PIG-3065
> Project: Pig
> Issue Type: New Feature
> Reporter: Rohini Palaniswamy
> Priority: Minor
>
> In hadoop 0.23 the output committer can optionally support recovery to handle
> the application master getting restarted (failing some # of attempts). If its
> possible the pig outputformat/committer should support recovery.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
