Hi, Hadoop includes MultipleOutputs class to deal with MR job that needs to write to multiple output formats.
We at Akamai have extended the idea, and allowed multiple mappers to write to multiple reducers, which would use MultipleOutputs to write to multiple output formats. Would that fit to Hadoop's core library? At any rate, we would really appreciate a review of the code and the concept. The main idea is simple. Wrap all key/values sent from mapper to reducer with a class that specify to which mapper does this key belong. Use a custom Comparator, Partitioner and serializer that would understand to which reducer does this key belong, and would direct it to the relevant reducer, use the relevant comparator, etc. Here is the project: https://github.com/elazarl/multireducers Thanks, Elazar
