Hi, Peeyush, I guess I didn't make myself clear. I am trying to run a Hadoop pipes job with a combination of Java classes and C++ classes. So the command I am using is like:
hadoop pipes -conf myconf.xml -inputformat MyInputFormat.class -input in -output out And it threw ClassNotFoundException for my InputSplit class. As I understand "hadoop jar" is used to run a jar file, which is not my case. And there is a -jar option in "hadoop pipes". But, unfortunately, it is not working for me. So the question I want to ask is how to include customized Java classes, such as MyInputSplit, in a pipes job? Thanks, Mike ________________________________ From: Peeyush Bishnoi <[EMAIL PROTECTED]> To: [email protected]; [email protected] Sent: Wednesday, October 29, 2008 12:52:18 PM Subject: RE: How do I include customized InputFormat, InputSplit and RecordReader in a C++ pipes job? Hello Zhengguo , Yes , -libjars is the new feature in Hadoop. This feature has been available from Hadoop-0.17.x , but it is more stable from hadoop 0.18.x example to use -libjars... hadoop jar -libjars <comma seperated list of jars> ... Thanks , --- Peeyush -----Original Message----- From: Zhengguo 'Mike' SUN [mailto:[EMAIL PROTECTED] Sent: Wed 10/29/2008 9:22 AM To: [email protected] Subject: Re: How do I include customized InputFormat, InputSplit and RecordReader in a C++ pipes job? Hi, Amareshwari, Is -libjars a new option in Hadoop 0.19? I am using 0.17.2. The only option I see is -jar, which didn't work for me. And besides passing them as jar file, is there any other ways to do that? Thanks Mike ________________________________ From: Amareshwari Sriramadasu <[EMAIL PROTECTED]> To: [email protected] Sent: Tuesday, October 28, 2008 11:58:33 PM Subject: Re: How do I include customized InputFormat, InputSplit and RecordReader in a C++ pipes job? Hi, How are you passing your classes to the pipes job? If you are passing them as a jar file, you can use -libjars option. From branch 0.19, the libjar files are added to the client classpath also. Thanks Amareshwari Zhengguo 'Mike' SUN wrote: > Hi, > > I implemented customized classes for InputFormat, InputSplit and RecordReader > in Java and was trying to use them in a C++ pipes job. The customized > InputFormat class could be included using the -inputformat option, but it > threw ClassNotFoundException for my customized InputSplit class. It seemed > the classpath has not been correctly set. Is there any way that let me > include my customized classes in a pipes job? > > > > >
