[
https://issues.apache.org/jira/browse/MAPREDUCE-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mariappan Asokan updated MAPREDUCE-4808:
----------------------------------------
Attachment: mapreduce-4808.patch
I will briefly outline what the patch does and the rationale.
* It makes the {{MergeManager}} pluggable. Rationale: {{MergeManager}} does
the merge sorting on the reduce side and is part of the overall sorting that
happens in MR data flow.
* It makes {{MapOutput}} class overridable.
Rationale: {{MergeManager}} plugin implementations can make efficient use of
JVM memory for data shuffling and provide their own implementation of
{{MapOutput.}}
* It makes local job runs use {{MergeManager}} or plugin implementations to do
the merge sort instead of {{Merger.}}
Rationale: Local job runs should also be able to use the sort plugin on the
reduce side just like they can make use of the sort plugin on the map side.
> Allow reduce-side merge to be pluggable
> ---------------------------------------
>
> Key: MAPREDUCE-4808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4808
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.0.2-alpha
> Reporter: Arun C Murthy
> Assignee: Mariappan Asokan
> Fix For: 2.0.3-alpha
>
> Attachments: COMBO-mapreduce-4809-4812-4808.patch,
> mapreduce-4808.patch, mapreduce-4808.patch
>
>
> Allow reduce-side merge to be pluggable for MAPREDUCE-2454
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
