[
https://issues.apache.org/jira/browse/HADOOP-3702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12615523#action_12615523
]
Owen O'Malley commented on HADOOP-3702:
---------------------------------------
I don't like that each stage in the pipeline has its own configuration and that
you serialize them all and put them into the outer configuration. What is the
use case for that? If you are going to do that, wouldn't it be easier to just
make Configuration Writable and use the writable stringifier?
You should use hadoop.io.Data{In,Out}putBuffer rather than defining your own
DirectBufferByteArrayOutputStream, especially since the name sounds like a
direct buffer.
You should probably convert the task id string to a TaskAttemptID and call the
isMap method rather than parsing the taskid string.
The preferred style is Sun's:
{code}
if (...) {
} else {
}
{code}
Other than that, it seems good.
> add support for chaining Maps in a single Map and after a Reduce [M*/RM*]
> -------------------------------------------------------------------------
>
> Key: HADOOP-3702
> URL: https://issues.apache.org/jira/browse/HADOOP-3702
> Project: Hadoop Core
> Issue Type: New Feature
> Components: mapred
> Environment: all
> Reporter: Alejandro Abdelnur
> Assignee: Alejandro Abdelnur
> Priority: Minor
> Attachments: patch3702.txt, patch3702.txt, patch3702.txt,
> patch3702.txt, patch3702.txt
>
>
> On the same input, we usually need to run multiple Maps one after the other
> without no Reduce. We also have to run multiple Maps after the Reduce.
> If all pre-Reduce Maps are chained together and run as a single Map a
> significant amount of Disk I/O will be avoided.
> Similarly all post-Reduce Maps can be chained together and run in the Reduce
> phase after the Reduce.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
