[
https://issues.apache.org/jira/browse/MAPREDUCE-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13424631#comment-13424631
]
Mariappan Asokan commented on MAPREDUCE-318:
--------------------------------------------
Hi Todd,
Thanks for your comments. I read more positive comments on the performance
improvement that other developers have noticed. I understand your concerns on
the stability of the code as well. However, I see the following drawbacks
keeping the existing code:
* We have to add another configuration parameter to force the execution of new
code.
* There will be some code duplication. Bug fixes have to go into both code
bases until of course 1.x release is obsoleted.
* If there is any enhancement to be done, one has to incorporate it into both
code bases. Just like MAPREDUCE-2454, there is another Jira MAPREDUCE-4049
that requires two different patches one for each code base.
Having said this, I am okay to keep the existing code and bypass it only when a
configuration parameter is set. I want to know whether there is any other
show-stopper along the way. If there is none, I will raise a Jira on this and
start working towards it.
> Refactor reduce shuffle code
> ----------------------------
>
> Key: MAPREDUCE-318
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-318
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: performance, task
> Reporter: Owen O'Malley
> Assignee: Owen O'Malley
> Fix For: 0.21.0
>
> Attachments: HADOOP-5233_api.patch, HADOOP-5233_part0.patch,
> mapred-318-14Aug.patch, mapred-318-20Aug.patch, mapred-318-24Aug.patch,
> mapred-318-3Sep-v1.patch, mapred-318-3Sep.patch, mapred-318-common.patch
>
>
> The reduce shuffle code has become very complex and entangled. I think we
> should move it out of ReduceTask and into a separate package
> (org.apache.hadoop.mapred.task.reduce). Details to follow.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
