Re: how to define new InputFormat with streaming?

2013-03-17 Thread Harsh J
The issue is that Streaming expects the old/stable MR API (org.apache.hadoop.mapred.InputFormat) as its input format class, but your WholeFileInputFormat is using the new MR API (org.apache.hadoop.mapreduce.lib.input.InputFormat). Using the older form will let you pass. This has nothing to do

Re:Re: how to define new InputFormat with streaming?

2013-03-17 Thread springring
thanks I modify the java file with old mapred API, but there is still error javac -classpath /usr/lib/hadoop/hadoop-core-0.20.2-cdh3u3.jar:/usr/lib/hadoop/lib/* -d class9 ./*.java ./WholeFileInputFormat.java:16: error: package org.apache.hadoop.mapred.lib.input does not exist import

Re: Re: how to define new InputFormat with streaming?

2013-03-17 Thread Harsh J
It isn't as easy as changing that import line: package org.apache.hadoop.mapred.lib.input does not exist The right package is package org.apache.hadoop.mapred. On Mon, Mar 18, 2013 at 7:22 AM, springring springr...@126.com wrote: thanks I modify the java file with old mapred API, but there

Re:Re: Re: how to define new InputFormat with streaming?

2013-03-17 Thread springring
you are right! Now the import path is all right. At 2013-03-18 09:57:33,Harsh J ha...@cloudera.com wrote: It isn't as easy as changing that import line: package org.apache.hadoop.mapred.lib.input does not exist The right package is package org.apache.hadoop.mapred. On Mon, Mar 18,

how to define new InputFormat with streaming?

2013-03-15 Thread springring
Hi, my hadoop version is Hadoop 0.20.2-cdh3u3 and I want to define new InputFormat in hadoop book , but there is error class org.apache.hadoop.streaming.WholeFileInputFormat not org.apache.hadoop.mapred.InputFormat Hadoop version is 0.20, but the streaming still depend on 0.10 mapred

Re: how to define new InputFormat with streaming?

2013-03-15 Thread Steve Loughran
On 15 March 2013 09:18, springring springr...@126.com wrote: Hi, my hadoop version is Hadoop 0.20.2-cdh3u3 and I want to define new InputFormat in hadoop book , but there is error class org.apache.hadoop.streaming.WholeFileInputFormat not org.apache.hadoop.mapred.InputFormat Hadoop