[
https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13026354#comment-13026354
]
Owen O'Malley commented on MAPREDUCE-2454:
------------------------------------------
I suspect the right interfaces are a subset of the current
mapreduce.RecordReader and RecordWriter. In particular, they would look like:
{code}
public abstract class RecordWriter {
public abstract void write(Object key, Object value
) throws IOException, InterruptedException;
public abstract void close() throws IOException, InterruptedException;
}
public abstract class RecordReader {
public abstract
boolean nextKeyValue() throws IOException, InterruptedException;
public abstract
Object getCurrentKey() throws IOException, InterruptedException;
public abstract
Object getCurrentValue() throws IOException, InterruptedException;
public abstract void close() throws IOException, InterruptedException;
}
{code}
Making the current shuffle code implement these classes would take work, but be
doable.
I'll also take the chance to suggest Java's ServiceLoader library as the right
way to configure the plugin.
> Allow external sorter plugin for MR
> -----------------------------------
>
> Key: MAPREDUCE-2454
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Mariappan Asokan
> Priority: Minor
> Attachments: MapOutputSorter.java, ReduceInputSorter.java
>
>
> Define interfaces and some abstract classes in the Hadoop framework to
> facilitate external sorter plugins both on the Map and Reduce sides.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
