[
https://issues.apache.org/jira/browse/MAPREDUCE-4808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13556247#comment-13556247
]
Arun C Murthy commented on MAPREDUCE-4808:
------------------------------------------
bq. The goal is to be able to write alternate implementations of the Shuffle
Alejandro - it seems like you understand something about the use-case that I
don't. Maybe you & Asokan have had a private chat?
What are the use-cases for alternate implementations of the Shuffle? Like Chris
also mentioned with MAPREDUCE-4049 we already allow alternate implementations
of Shuffle, is this redundant then?
bq. While some of this logic replacement could be done at Merge level as you
suggested, other, like MapOutput allocation cannot be done there as this is
driven by the MergeManager.
So, a combination of MapOutput re-factor and Merger interface should suffice?
IAC, what are the use-cases for alternate implementations of MapOutput? Or, is
it the MapOutput re-factor merely a code-hygiene issue?
----
I'm not trying to be difficult here. But, I feel like I just don't understand
the use-case. So, I'd appreciate if we could focus on concrete use-cases for
the plugin. I admit I still am having a hard time understanding why we need
this complexity.
Thanks.
> Refactor MapOutput and MergeManager to facilitate reuse by Shuffle
> implementations
> ----------------------------------------------------------------------------------
>
> Key: MAPREDUCE-4808
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4808
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Affects Versions: 2.0.2-alpha
> Reporter: Arun C Murthy
> Assignee: Mariappan Asokan
> Fix For: 2.0.3-alpha
>
> Attachments: COMBO-mapreduce-4809-4812-4808.patch,
> mapreduce-4808.patch, mapreduce-4808.patch, mapreduce-4808.patch,
> mapreduce-4808.patch, mapreduce-4808.patch, mapreduce-4808.patch,
> mapreduce-4808.patch, MergeManagerPlugin.pdf
>
>
> Now that Shuffle is pluggable (MAPREDUCE-4049), it would be convenient for
> alternate implementations to be able to reuse portions of the default
> implementation.
> This would come with the strong caveat that these classes are LimitedPrivate
> and Unstable.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
