[
https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13477565#comment-13477565
]
Mariappan Asokan commented on MAPREDUCE-2454:
---------------------------------------------
Hi Arun,
I stepped back and looked at some of the Jiras that were developed under
branches(like MAPREDUCE-279, HDFS-1623, etc.) They contributed major
enhancements to Hadoop. MAPREDUCE-2454 on the other hand has just refactored
existing code to make some classes more modular and pluggable.
There is no significant new functionality added. There is no change in the way
shuffling is done. The communication between shuffle and merge is kept intact.
The MAPREDUCE-318 refactoring in fact helped a lot. There is only a minor
rearranging of the code in MAPREDUCE-2454.
I followed your advice diligently to work on the trunk first and I picked up
suggestions from some other developers who are watching this Jira. During our
brief meetings, the suggestions you gave were very valuable and I followed them.
I have been testing the changes for the last year or so and I have been keeping
the Jira up to the latest trunk. I have not hit any issues in my testing w.r.t
both functionally and performance.
I understand that your time is very precious. I already updated the design
document for you to make it easier for code review. If you can suggest
anything that will expedite the committing, please let me know.
Thanks.
-- Asokan
> Allow external sorter plugin for MR
> -----------------------------------
>
> Key: MAPREDUCE-2454
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.0.0-alpha, 3.0.0, 2.0.2-alpha
> Reporter: Mariappan Asokan
> Assignee: Mariappan Asokan
> Priority: Minor
> Labels: features, performance, plugin, sort
> Attachments: HadoopSortPlugin.pdf, HadoopSortPlugin.pdf,
> KeyValueIterator.java, MapOutputSorterAbstract.java, MapOutputSorter.java,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mr-2454-on-mr-279-build82.patch.gz, MR-2454-trunkPatchPreview.gz,
> ReduceInputSorter.java
>
>
> Define interfaces and some abstract classes in the Hadoop framework to
> facilitate external sorter plugins both on the Map and Reduce sides.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
