Totally stuck here, I can't seem to find a way to resolve this, but I
can't use the new API _and_ use the MultipleOutputFormats class.
I found this thread which is related, but doesn't seem to help me (or
I missed something completely, certainly possible):
http://markmail.org/message/u4wz5nbcn5rawydq#query:hadoop%20MultipleTextOutputFormat%20OutputFormat%20Job%20JobConf+page:1+mid:5wy63oqa2vs6bj7b+state:results
My controller Job class is simple, but I get a compile error trying to
add the new MultipleOutputs:
public class ControllerMetricGrinder {
public static class MetricNameMultipleTextOutputFormat extends
MultipleTextOutputFormat<String, ControllerMetric> {
@Override
protected String generateFileNameForKeyValue(String key,
ControllerMetric value, String name) {
return key;
}
}
public static void main(String[] args) throws Exception {
Job job = new Job();
job.setJarByClass(ControllerMetricGrinder.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(ControllerMetric.class);
job.setMapperClass(ControllerMetricMapper.class);
job.setCombinerClass(ControllerMetricReducer.class);
job.setReducerClass(ControllerMetricReducer.class);
// COMPILE ERROR HERE
MultipleOutputs.addMultiNamedOutput(job, "metrics",
MetricNameMultipleTextOutputFormat.class,
Text.class, ControllerMetric.class);
job.setNumReduceTasks(5);
FileInputFormat.addInputPath(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));
System.exit(job.waitForCompletion(true) ? 0 : 1);
}
}
(mappers and reducers are using the new API, and are in separate
classes).
MultipleOutputs doesn't take a Job, it only takes a JobConf. Any
ideas? I'd prefer to use the new API (because I've written it that
way), but I'm guessing now I'll have to go and rework everything to
the OLD API to get this to work.
I'm trying to create a File-per-metric name (there's only 5).
thoughts?
Paul
