OK. I gave up on my code and switched to run some example code from Mahout. I ran the clustering example from quickstart as follow: bin/hadoop jar ../trunk/examples/target/mahout-examples-0.4-SNAPSHOT.job org.apache.mahout.clustering.syntheticcontrol.canopy.Job
Again, it worked on my local machine and gave me ClassNotFoundException in the cluster. java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.mahout.clustering.syntheticcontrol.canopy.InputMapper at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:919) at org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:161) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:532) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.lang.ClassNotFoundException: org.apache.mahout.clustering.syntheticcontrol.canopy.InputMapper at java.net.URLClassLoader$1.run(URLClassLoader.java:200) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:188) at java.lang.ClassLoader.loadClass(ClassLoader.java:307) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:252) at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:247) at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:866) at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:917) ... 4 more From: Sean Owen <[email protected]> To: Zhengguo 'Mike' SUN <[email protected]> Cc: "[email protected]" <[email protected]> Sent: Monday, February 21, 2011 3:34 PM Subject: Re: LanczosSolver and ClassNotFoundException In contrast, what is the scenario where it didn't work then? This may be a dumb question that a Hadoop guru would know better. MapTask.run() occurs in the... master? but not on the "client side". I don't know why the classpath would be different though, but, this is the essential issue. (I've not seen this ever.) On Mon, Feb 21, 2011 at 7:52 PM, Zhengguo 'Mike' SUN <[email protected]> wrote: > Yes. I put the .jar files under the lib/ directory of my own jar. I didn't > set the Class-Path in MANIFEST.MF in the past. I just tried putting all jar > files in the Class-Path of MANIFEST.MF, it still gave me the same exception. > My experiment is a little bit different. It creates a Vector in the map > function of the Mapper class. It worked correctly in the cluster. >
