[
https://issues.apache.org/jira/browse/MAPREDUCE-5974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon updated MAPREDUCE-5974:
-----------------------------------
Attachment: mapreduce-5974.txt
Attached patch implements the improvement as described. I did not include any
new unit test since this code path is exercised by existing paths, and mocking
out this section of the code is really quite difficult. If folks would like to
see a unit test, I can add a full functional test which specifies some bogus
collector class ahead of the real implementation, but figured that it's a
trivial enough change we could get by without.
> Allow map output collector fallback
> -----------------------------------
>
> Key: MAPREDUCE-5974
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5974
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: task
> Affects Versions: 2.6.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Attachments: mapreduce-5974.txt
>
>
> Currently we only allow specifying a single MapOutputCollector implementation
> class in a job. It would be nice to allow a comma-separated list of classes:
> we should try each collector implementation in the user-specified order until
> we find one that can be successfully instantiated and initted.
> This is useful for cases where a particular optimized collector
> implementation cannot operate on all key/value types, or requires native
> code. The cluster administrator can configure the cluster to try to use the
> optimized collector and fall back to the default collector.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
