Dear all,
I got the problem. The input file is set wrongly.
Sorry for sending such an email!
Thanks,
Bing
On Thu, Feb 9, 2012 at 10:05 PM, Bing Li <lbl...@gmail.com> wrote:
> Dear all,
>
> When running some sample codes from Hadoop in Action, I got an
> IOException: Job Failed.
>
> Exception in thread "main" java.io.IOException: Job failed!
> at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1252)
> at com.greatfree.testing.hadoop.CitationHistogram.run(Unknown Source)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at com.greatfree.testing.hadoop.CitationHistogram.main(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
>
> For the following code, everything is fine.
>
> ......
>
> public static class MapClass extends MapReduceBase implements
> Mapper<Text, Text, Text, Text>
> {
> public void map(Text key, Text value,
> OutputCollector<Text, Text> output, Reporter reporter) throws IOException
> {
> output.collect(value, key);
> }
> }
>
> public static class ReduceClass extends MapReduceBase implements
> Reducer<Text, Text, Text, IntWritable>
> {
> public void reduce(Text key, Iterator<Text> values,
> OutputCollector<Text, IntWritable> output, Reporter reporter) throws
> IOException
> {
> int count = 0;
> while (values.hasNext())
> {
> values.next();
> count ++;
> }
> output.collect(key, new IntWritable(count));
> }
> }
>
> public int run(String[] args) throws Exception
> {
> Configuration conf = getConf();
>
> JobConf job = new JobConf(conf, MyJob.class);
>
> Path in = new Path(args[0]);
> Path out = new Path(args[1]);
> FileInputFormat.setInputPaths(job, in);
> FileOutputFormat.setOutputPath(job, out);
>
> job.setJobName("MyJobForCount");
> job.setMapperClass(MapClass.class);
> job.setReducerClass(ReduceClass.class);
>
> job.setInputFormat(KeyValueTextInputFormat.class);
> job.setOutputFormat(TextOutputFormat.class);
> job.setOutputKeyClass(Text.class);
> job.setOutputValueClass(Text.class);
> job.set("key.value.separator.in.input.line", ",");
>
> JobClient.runJob(job);
> return 0;
> }
>
> ......
>
> However, when making some changes on the above code as follows, I got the
> above exception.
>
> ......
>
> public static class MapClass extends MapReduceBase implements
> Mapper<Text, Text, IntWritable, IntWritable>
> {
> private final static IntWritable uno = new IntWritable(1);
> private IntWritable citationCount = new IntWritable();
>
> public void map(Text key, Text value,
> OutputCollector<IntWritable, IntWritable> output, Reporter reporter) throws
> IOException
> {
> citationCount.set(Integer.parseInt(value.toString()));
> output.collect(citationCount, uno);
> }
> }
>
> public static class Reduce extends MapReduceBase implements
> Reducer<IntWritable,IntWritable,IntWritable,IntWritable>
> {
> public void reduce(IntWritable key, Iterator<IntWritable>
> values, OutputCollector<IntWritable, IntWritable>output, Reporter reporter)
> throws IOException
> {
> int count = 0;
> while (values.hasNext())
> {
> count += values.next().get();
> }
> output.collect(key, new IntWritable(count));
> }
> }
>
> public int run(String[] args) throws Exception
> {
> Configuration conf = getConf();
>
> JobConf job = new JobConf(conf, CitationHistogram.class);
>
> Path in = new Path(args[0]);
> Path out = new Path(args[1]);
> FileInputFormat.setInputPaths(job, in);
> FileOutputFormat.setOutputPath(job, out);
>
> job.setJobName("CitationHistogram");
> job.setMapperClass(MapClass.class);
> job.setReducerClass(Reduce.class);
>
> job.setInputFormat(KeyValueTextInputFormat.class);
> job.setOutputFormat(TextOutputFormat.class);
> job.setOutputKeyClass(IntWritable.class);
> job.setOutputValueClass(IntWritable.class);
>
> JobClient.runJob(job);
>
> return 0;
> }
>
> ......
>
> Both of the above code work on the same input file. So I felt weird. Could
> you please help me? Thanks so much!
>
> Best regards,
> Bing
>
