[
https://issues.apache.org/jira/browse/HADOOP-3150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12610563#action_12610563
]
Devaraj Das commented on HADOOP-3150:
-------------------------------------
Yes, I missed the point that a record writer might not be able to cater to all
the needs that a user might have. For example, the user wants to write a stream
of data that he generates. So we really need the FSDataOutputStream either
through an API like what Owen suggested, or, let the user create it in the
directory that we provide him.
I'd suggest that HADOOP-3149 be committed in its current form/scope, and
address the side files problem as part of this work. We'd also require the user
to set the final output directory of the side files through the API
FileOutputFormat.setOutputPath even though he might not be using
FileOutputFormat as his job outputformat.
Initially i suggested that we define a class SideFileWriter to get/set the
above, and thereby make it clear that he & the framework is dealing with side
files, but I am ok either way.
> Move task file promotion into the task
> --------------------------------------
>
> Key: HADOOP-3150
> URL: https://issues.apache.org/jira/browse/HADOOP-3150
> Project: Hadoop Core
> Issue Type: Improvement
> Components: mapred
> Reporter: Owen O'Malley
> Assignee: Amareshwari Sriramadasu
> Fix For: 0.19.0
>
> Attachments: 3150.patch
>
>
> We need to move the task file promotion from the JobTracker to the Task and
> move it down into the output format.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
