[
https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495660#comment-13495660
]
Alejandro Abdelnur commented on MAPREDUCE-2454:
-----------------------------------------------
Asokan,
Thanks for patience working out the design and implementation of this JIRA.
The latest patch looks good and it addresses the concerns voiced regarding big
changes that could destabilize the MR framework.
There are few minor NITs that should be addressed in the patch (at the end of
this comment).
+1 after these NITs are addresses and jenkins test-patch OKs the new patch.
Patch NITs:
* PreReduceProcessor.java: unused import: Shuffle
* ReduceTasks.java: line 357, PreReduceProcessor merger = (PreReduceProcessor)
ReflectionUtils.newInstance(..., no need for the "(PreReduceProcessor)" casting.
* Shuffle.java, unused imports: FileSystem, LocalDirAllocator,
CompressionCodec, MapOutputFile, RawKeyValueIterator, Reducer,
CombineOutputCollector
* TestLimitNQuery.java, unused imports: FileInputStream, FileNotFoundException,
FileStatus, JobCounter, TaskAttemptID, ReflectionUtils
> Allow external sorter plugin for MR
> -----------------------------------
>
> Key: MAPREDUCE-2454
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.0.0-alpha, 3.0.0, 2.0.2-alpha
> Reporter: Mariappan Asokan
> Assignee: Mariappan Asokan
> Priority: Minor
> Labels: features, performance, plugin, sort
> Attachments: HadoopSortPlugin.pdf, HadoopSortPlugin.pdf,
> KeyValueIterator.java, MapOutputSorterAbstract.java, MapOutputSorter.java,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mapreduce-2454.patch, mapreduce-2454.patch, mapreduce-2454.patch,
> mr-2454-on-mr-279-build82.patch.gz, MR-2454-trunkPatchPreview.gz,
> ReduceInputSorter.java
>
>
> Define interfaces and some abstract classes in the Hadoop framework to
> facilitate external sorter plugins both on the Map and Reduce sides.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
