[
https://issues.apache.org/jira/browse/HADOOP-1558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alejandro Abdelnur updated HADOOP-1558:
---------------------------------------
Attachment: hadoop-1558-JUN0907-1620.txt
new testcase was missing in previous patch.
the initialize() and commit() methods now check if the output dir is present,
if missing (could be the case when MR job by implementation is designed to
produce not ouput) it does a no-operation in both methods.
> changes to OutputFormat to work on temporary directory to enable re-running
> crashed jobs (Issue: 1121)
> ------------------------------------------------------------------------------------------------------
>
> Key: HADOOP-1558
> URL: https://issues.apache.org/jira/browse/HADOOP-1558
> Project: Hadoop
> Issue Type: Improvement
> Components: mapred
> Environment: all
> Reporter: Alejandro Abdelnur
> Fix For: 0.14.0
>
> Attachments: hadoop-1558-JUN0907-1550.txt,
> hadoop-1558-JUN0907-1620.txt
>
>
> Add OutputFormat methods like:
> /** Called to initialize output for this job. */
> void initialize(JobConf job) throws IOException;
> /** Called to finalize output for this job. */
> void commit(JobConf job) throws IOException;
> In the base implemenation for FileSystem output, initialize() might then
> create a temporary directory for the job, removing any that already exists,
> and commit could rename the temporary output directory to the final name.
> The existing checkOutputSpecs() would continue to throw an exception if the
> final output already exists.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
