Peter: I just noticed that there has been a new patch applied by Drew yesterday for this. (https://issues.apache.org/jira/browse/MAHOUT-426)
The last comment on the thread says "Integrated in Mahout-Quality #128 (See http://hudson.zones.apache.org/hudson/job/Mahout-Quality/128/)" Does it mean that if I did a "mvn-U" on the trunk, I should be able to get the fix and then not run into the issue ? Regards gokul On Thu, Jul 1, 2010 at 2:31 PM, Gokul Pillai <[email protected]> wrote: > The version on my instance is 0.20.1+152. > I am guessing if the jar file name is "hadoop-0.20.1+152-fairscheduler.jar" > then that is the version. > > > On Thu, Jul 1, 2010 at 11:11 AM, Peter M. Goldstein < > [email protected]> wrote: > >> Gokul, >> >> What version of Hadoop are you using? I'm running with 0.20+320, and I >> don't run into this issue. >> >> Thanks. >> >> Regards, >> >> Peter >> >> -----Original Message----- >> From: Gokul Pillai [mailto:[email protected]] >> Sent: Thursday, July 01, 2010 10:29 AM >> To: [email protected] >> Subject: Re: No class definition found for org.apache.mahout.math.Vector >> when invoking K-means example >> >> Hello Peter: >> When i run that command, I get the following although different error: >> Exception in thread "main" java.lang.NoClassDefFoundError: >> org/apache/commons/cli2/OptionException >> at java.lang.Class.forName0(Native Method) >> at java.lang.Class.forName(Class.java:247) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:149) >> Caused by: java.lang.ClassNotFoundException: >> org.apache.commons.cli2.OptionException >> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >> >> Digging into this, I find that hadoop 0.20 ships with >> "commons-cli-1.2.jar" >> but mahout 0.40 Snapshot version depends on "commons-cli-2.0-mahout.jar" >> that is there in the $MAHOUT_HOME/trunk/examples/target/ folder once you >> "mvn install" in the "examples" folder. >> >> So i replaced the jar file in hadoop/lib with this one and then it goes >> back >> to the ClassNotFound error for Vector class. >> >> Regards >> Gokul >> >> On Thu, Jul 1, 2010 at 9:29 AM, Peter M. Goldstein < >> [email protected]> wrote: >> >> > Hi Gokul, >> > >> > Using the Mahout on Amazon EC2 directions I'm able to get a little >> farther >> > than this. Specifically, using the job file created when building the >> > examples I don't get any ClassNotFoundExceptions. Instead I get an >> > InvalidInputException (presumably because I didn't prepare any input >> data >> > for the job). If you provide some appropriately formatted candidate >> data >> > I'd be happy to try and run a complete test. >> > >> > In my test I'm using the command line: >> > >> > $HADOOP_HOME/bin/hadoop jar >> > $MAHOUT_HOME/examples/target/mahout-examples-0.4-SNAPSHOT.job >> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job >> > >> > Again, it's critical to build and use the examples Job file. If I >> attempt >> > to use the core Job file instead key classes appear to be absent. >> > >> > Hope that helps. >> > >> > --Peter >> > >> > -----Original Message----- >> > From: Gokul Pillai [mailto:[email protected]] >> > Sent: Wednesday, June 30, 2010 2:35 PM >> > To: [email protected] >> > Subject: Re: No class definition found for org.apache.mahout.math.Vector >> > when invoking K-means example >> > >> > Peter >> > Thank you for your response. However, after making your changes, it >> still >> > does not work for me. >> > In fact I tried another way which is: >> > 1. create a lib folder within my Job Jar-file and put the >> > "org.apache.mahout.math.Vector" and other dependent jars. >> > 2. Launch the command via "hadoop jar ./mahout-examples-with-libs.jar >> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job" >> > >> > However, I still get the error: >> > "Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)" >> > >> > >> > Another thing I tried is modify the $HADOOP_HOME/conf/hadoop-env.sh and >> > edit >> > the HADOOP_CLASSPATH to include the dependent libraries. Even that does >> not >> > work. >> > >> > Regards >> > Gokul >> > >> > On Mon, Jun 28, 2010 at 3:43 PM, Peter M. Goldstein < >> > [email protected]> wrote: >> > >> > > Hi Gokul, >> > > >> > > You may want to see the following bug reports: >> > > >> > > https://issues.apache.org/jira/browse/MAHOUT-426 >> > > >> > > https://issues.apache.org/jira/browse/MAHOUT-427 >> > > >> > > https://issues.apache.org/jira/browse/MAHOUT-428 >> > > >> > > I ran into similar issues as you and had to some minor patching to get >> > > everything to work. >> > > >> > > Also, make sure you run an explicit "mvn install" from within the >> > examples >> > > directory. I had to do this to get the JOB files to build correctly. >> > > Getting the JOB files to build correctly solves (sort of) issues 426 >> and >> > > 427, so you only need to apply the patch described in 428. >> > > >> > > Hope that helps. >> > > >> > > Regards, >> > > >> > > Peter >> > > >> > > -----Original Message----- >> > > From: Gokul Pillai [mailto:[email protected]] >> > > Sent: Monday, June 28, 2010 3:27 PM >> > > To: [email protected] >> > > Subject: No class definition found for org.apache.mahout.math.Vector >> when >> > > invoking K-means example >> > > >> > > I made a new installation of Hadoop 0.20 from Cloudera. >> > > I then installed Mahout 0.4-SNAPSHOT by doing a SVN checkout this >> morning >> > > and ran mvn install on core, utils and examples. >> > > I tried running the KMeans Clustering example by doing the following >> as >> > > mentioned in the tutorial: >> > > hadoop jar >> > ~/mahout/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.jar >> > > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job >> > > >> > > And I keep getting this error although I have explicitly set my >> Classpath >> > > to >> > > >> > > >> > >> > >> >> "/usr/lib/hadoop/lib/mahout-collections-1.0.jar:/usr/lib/hadoop/lib/mahout-m >> > > >> > > >> > >> > >> >> ath-0.4-SNAPSHOT.jar:/usr/lib/hadoop/lib/mahout-utils-0.4-SNAPSHOT.jar:/usr/ >> > > lib/hadoop/lib/mahout-core-0.4-SNAPSHOT.jar": >> > > >> > > 10/06/28 15:19:21 INFO mapred.JobClient: Task Id : >> > > attempt_201006280959_0027_r_000000_1, Status : FAILED >> > > java.lang.RuntimeException: java.lang.ClassNotFoundException: >> > > org.apache.mahout.clustering.kmeans.KMeansCombiner >> > > at >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:868) >> > > at >> > > >> > > >> > >> > >> >> org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:169) >> > > at >> org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1107) >> > > at >> > > >> > > >> > >> > >> >> org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1780 >> > > ) >> > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:375) >> > > at org.apache.hadoop.mapred.Child.main(Child.java:170) >> > > Caused by: java.lang.ClassNotFoundException: >> > > org.apache.mahout.clustering.kmeans.KMeansCombiner >> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >> > > at java.security.AccessController.doPrivileged(Native Method) >> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) >> > > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) >> > > at java.lang.Class.forName0(Native Method) >> > > at java.lang.Class.forName(Class.java:247) >> > > at >> > > >> > >> >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:815) >> > > at >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866) >> > > ... 5 more >> > > >> > > >> > > Any help would be appreciated. >> > > >> > > Regards >> > > Gokul >> > > >> > > >> > > >> > >> > >> > >> >> >> >
