[
https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610124#action_12610124
]
Devaraj Das commented on HADOOP-3150:
-------------------------------------
I think that we should just treat the side files as a special case. That is, it
doesn't really need to inherit FileOutputFormat. It could use that internally.
So we could have a class called _SideFileWriter_ that provides the methods
_getSideFileDirectory_ . The implementation of _getSideFileDirectory_ could
internally use the FileOutputFormat API to get the workoutputpath. In the case
where the outputformat of the job is a FileOutputFormat, the side files can be
created in the workoutputpath as is done today. In the case where the
outputformat is something else, the _getSideFileDirectory_ api returns a
directory that it synthesizes (something like _jobid/_temporary). The commit is
done for both the SideFileWriter as well as the job's OutputFormat.
Does that work? Thoughts?
> Move task file promotion into the task
> --------------------------------------
>
> Key: HADOOP-3150
> URL: https://issues.apache.org/jira/browse/HADOOP-3150
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.19.0
>
> Attachments: 3150.patch
>
>
> We need to move the task file promotion from the JobTracker to the Task and
> move it down into the output format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
