Try with number of reducers = 1 . -Amareshwari
On 8/16/10 12:25 PM, "rajgopalv" <[email protected]> wrote: 0 down vote favorite Hi. I'm a newbie in Hadoop. I'm trying out the Wordcount program. Now to try out multiple output files, i use MultipleOutputFormat. this link helped me in doing it. http://hadoop.apache.org/common/docs/r0.19.0/api/org/apache/hadoop/mapred/lib/MultipleOutputs.html in my driver class i had MultipleOutputs.addNamedOutput(conf, "even", org.apache.hadoop.mapred.TextOutputFormat.class, Text.class, IntWritable.class); MultipleOutputs.addNamedOutput(conf, "odd", org.apache.hadoop.mapred.TextOutputFormat.class, Text.class, IntWritable.class);` and my reduce class became this public static class Reduce extends MapReduceBase implements Reducer<Text, IntWritable, Text, IntWritable> { MultipleOutputs mos = null; public void configure(JobConf job) { mos = new MultipleOutputs(job); } public void reduce(Text key, Iterator<IntWritable> values, OutputCollector<Text, IntWritable> output, Reporter reporter) throws IOException { int sum = 0; while (values.hasNext()) { sum += values.next().get(); } if (sum % 2 == 0) { mos.getCollector("even", reporter).collect(key, new IntWritable(sum)); }else { mos.getCollector("odd", reporter).collect(key, new IntWritable(sum)); } //output.collect(key, new IntWritable(sum)); } @Override public void close() throws IOException { // TODO Auto-generated method stub mos.close(); } } Things worked , but i get LOT of files, (one odd and one even for every map-reduce) Question is : How can i have just 2 output files (odd & even) so that every odd output of every reduce gets written into that odd file, and same for even. -- View this message in context: http://old.nabble.com/MultipleOutputFormat-tp29447204p29447204.html Sent from the Hadoop core-user mailing list archive at Nabble.com.
