[
https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13029254#comment-13029254
]
Steve Loughran commented on MAPREDUCE-2454:
-------------------------------------------
I'm also worried about the exception signatures of the interface. One of the
hardest things to do in any Java plugin API is getting the list of possible
exceptions throw right -if not you end up wrapping everything up.
Presumably the list of exceptions thrown is derived from your (private)
implementation. For example, the {{initialize()}} methods throw {{ IOException,
ClassNotFoundException, UnsupportedOperationException }}.
A {{ClassNotFoundException}} ClassCastException implies the implementation is
possibly playing with classloaders -which is a dangerous game and one in which
most Java developers, myself included- shouldn't be doing. At the very least
the exception list should include
{{ClassCastException}} -when the class loads but is the wrong type
{{InstantiationException}} you can't instantiate the class as its constructor
isn't there
{{NoClassDefFoundError}} can't find the class though it used to be there
{{LinkageError}} etc.
The other signatures may work, but the current initialize code is clearly
biased towards one single implementation. We'd need more implementation, and
some test cases that simulate other failures, to make sure whatever changes go
into the MR engine actually handle them, presumably by catching and failing the
job. I don't see any point in trying to fall back to another sorter as that
would just hide a problem.
> Allow external sorter plugin for MR
> -----------------------------------
>
> Key: MAPREDUCE-2454
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454
> Project: Hadoop Map/Reduce
> Issue Type: New Feature
> Reporter: Mariappan Asokan
> Priority: Minor
> Attachments: KeyValueIterator.java, MapOutputSorter.java,
> MapOutputSorterAbstract.java, ReduceInputSorter.java
>
>
> Define interfaces and some abstract classes in the Hadoop framework to
> facilitate external sorter plugins both on the Map and Reduce sides.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
