To clarify, do the fixes relate to only Java code or is there any script in the mahout/trunk/bin folder too? If so, how does one get that from the repo?
On Fri, Jul 9, 2010 at 4:16 PM, Gokul Pillai <[email protected]> wrote: > Peter: > I just noticed that there has been a new patch applied by Drew yesterday > for this. (https://issues.apache.org/jira/browse/MAHOUT-426) > > The last comment on the thread says "Integrated in Mahout-Quality #128 > (See http://hudson.zones.apache.org/hudson/job/Mahout-Quality/128/)" > > Does it mean that if I did a "mvn-U" on the trunk, I should be able to get > the fix and then not run into the issue ? > > Regards > gokul > > On Thu, Jul 1, 2010 at 2:31 PM, Gokul Pillai <[email protected]> wrote: > >> The version on my instance is 0.20.1+152. >> I am guessing if the jar file name is >> "hadoop-0.20.1+152-fairscheduler.jar" then that is the version. >> >> >> On Thu, Jul 1, 2010 at 11:11 AM, Peter M. Goldstein < >> [email protected]> wrote: >> >>> Gokul, >>> >>> What version of Hadoop are you using? I'm running with 0.20+320, and I >>> don't run into this issue. >>> >>> Thanks. >>> >>> Regards, >>> >>> Peter >>> >>> -----Original Message----- >>> From: Gokul Pillai [mailto:[email protected]] >>> Sent: Thursday, July 01, 2010 10:29 AM >>> To: [email protected] >>> Subject: Re: No class definition found for org.apache.mahout.math.Vector >>> when invoking K-means example >>> >>> Hello Peter: >>> When i run that command, I get the following although different error: >>> Exception in thread "main" java.lang.NoClassDefFoundError: >>> org/apache/commons/cli2/OptionException >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:247) >>> at org.apache.hadoop.util.RunJar.main(RunJar.java:149) >>> Caused by: java.lang.ClassNotFoundException: >>> org.apache.commons.cli2.OptionException >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >>> >>> Digging into this, I find that hadoop 0.20 ships with >>> "commons-cli-1.2.jar" >>> but mahout 0.40 Snapshot version depends on "commons-cli-2.0-mahout.jar" >>> that is there in the $MAHOUT_HOME/trunk/examples/target/ folder once you >>> "mvn install" in the "examples" folder. >>> >>> So i replaced the jar file in hadoop/lib with this one and then it goes >>> back >>> to the ClassNotFound error for Vector class. >>> >>> Regards >>> Gokul >>> >>> On Thu, Jul 1, 2010 at 9:29 AM, Peter M. Goldstein < >>> [email protected]> wrote: >>> >>> > Hi Gokul, >>> > >>> > Using the Mahout on Amazon EC2 directions I'm able to get a little >>> farther >>> > than this. Specifically, using the job file created when building the >>> > examples I don't get any ClassNotFoundExceptions. Instead I get an >>> > InvalidInputException (presumably because I didn't prepare any input >>> data >>> > for the job). If you provide some appropriately formatted candidate >>> data >>> > I'd be happy to try and run a complete test. >>> > >>> > In my test I'm using the command line: >>> > >>> > $HADOOP_HOME/bin/hadoop jar >>> > $MAHOUT_HOME/examples/target/mahout-examples-0.4-SNAPSHOT.job >>> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job >>> > >>> > Again, it's critical to build and use the examples Job file. If I >>> attempt >>> > to use the core Job file instead key classes appear to be absent. >>> > >>> > Hope that helps. >>> > >>> > --Peter >>> > >>> > -----Original Message----- >>> > From: Gokul Pillai [mailto:[email protected]] >>> > Sent: Wednesday, June 30, 2010 2:35 PM >>> > To: [email protected] >>> > Subject: Re: No class definition found for >>> org.apache.mahout.math.Vector >>> > when invoking K-means example >>> > >>> > Peter >>> > Thank you for your response. However, after making your changes, it >>> still >>> > does not work for me. >>> > In fact I tried another way which is: >>> > 1. create a lib folder within my Job Jar-file and put the >>> > "org.apache.mahout.math.Vector" and other dependent jars. >>> > 2. Launch the command via "hadoop jar ./mahout-examples-with-libs.jar >>> > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job" >>> > >>> > However, I still get the error: >>> > "Error: java.lang.ClassNotFoundException: org.apache.mahout.math.Vector >>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)" >>> > >>> > >>> > Another thing I tried is modify the $HADOOP_HOME/conf/hadoop-env.sh and >>> > edit >>> > the HADOOP_CLASSPATH to include the dependent libraries. Even that does >>> not >>> > work. >>> > >>> > Regards >>> > Gokul >>> > >>> > On Mon, Jun 28, 2010 at 3:43 PM, Peter M. Goldstein < >>> > [email protected]> wrote: >>> > >>> > > Hi Gokul, >>> > > >>> > > You may want to see the following bug reports: >>> > > >>> > > https://issues.apache.org/jira/browse/MAHOUT-426 >>> > > >>> > > https://issues.apache.org/jira/browse/MAHOUT-427 >>> > > >>> > > https://issues.apache.org/jira/browse/MAHOUT-428 >>> > > >>> > > I ran into similar issues as you and had to some minor patching to >>> get >>> > > everything to work. >>> > > >>> > > Also, make sure you run an explicit "mvn install" from within the >>> > examples >>> > > directory. I had to do this to get the JOB files to build correctly. >>> > > Getting the JOB files to build correctly solves (sort of) issues 426 >>> and >>> > > 427, so you only need to apply the patch described in 428. >>> > > >>> > > Hope that helps. >>> > > >>> > > Regards, >>> > > >>> > > Peter >>> > > >>> > > -----Original Message----- >>> > > From: Gokul Pillai [mailto:[email protected]] >>> > > Sent: Monday, June 28, 2010 3:27 PM >>> > > To: [email protected] >>> > > Subject: No class definition found for org.apache.mahout.math.Vector >>> when >>> > > invoking K-means example >>> > > >>> > > I made a new installation of Hadoop 0.20 from Cloudera. >>> > > I then installed Mahout 0.4-SNAPSHOT by doing a SVN checkout this >>> morning >>> > > and ran mvn install on core, utils and examples. >>> > > I tried running the KMeans Clustering example by doing the following >>> as >>> > > mentioned in the tutorial: >>> > > hadoop jar >>> > ~/mahout/trunk/examples/target/mahout-examples-0.4-SNAPSHOT.jar >>> > > org.apache.mahout.clustering.syntheticcontrol.kmeans.Job >>> > > >>> > > And I keep getting this error although I have explicitly set my >>> Classpath >>> > > to >>> > > >>> > > >>> > >>> > >>> >>> "/usr/lib/hadoop/lib/mahout-collections-1.0.jar:/usr/lib/hadoop/lib/mahout-m >>> > > >>> > > >>> > >>> > >>> >>> ath-0.4-SNAPSHOT.jar:/usr/lib/hadoop/lib/mahout-utils-0.4-SNAPSHOT.jar:/usr/ >>> > > lib/hadoop/lib/mahout-core-0.4-SNAPSHOT.jar": >>> > > >>> > > 10/06/28 15:19:21 INFO mapred.JobClient: Task Id : >>> > > attempt_201006280959_0027_r_000000_1, Status : FAILED >>> > > java.lang.RuntimeException: java.lang.ClassNotFoundException: >>> > > org.apache.mahout.clustering.kmeans.KMeansCombiner >>> > > at >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:868) >>> > > at >>> > > >>> > > >>> > >>> > >>> >>> org.apache.hadoop.mapreduce.JobContext.getCombinerClass(JobContext.java:169) >>> > > at >>> org.apache.hadoop.mapred.Task$CombinerRunner.create(Task.java:1107) >>> > > at >>> > > >>> > > >>> > >>> > >>> >>> org.apache.hadoop.mapred.ReduceTask$ReduceCopier.<init>(ReduceTask.java:1780 >>> > > ) >>> > > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:375) >>> > > at org.apache.hadoop.mapred.Child.main(Child.java:170) >>> > > Caused by: java.lang.ClassNotFoundException: >>> > > org.apache.mahout.clustering.kmeans.KMeansCombiner >>> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >>> > > at java.security.AccessController.doPrivileged(Native Method) >>> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >>> > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> > > at java.lang.ClassLoader.loadClass(ClassLoader.java:252) >>> > > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320) >>> > > at java.lang.Class.forName0(Native Method) >>> > > at java.lang.Class.forName(Class.java:247) >>> > > at >>> > > >>> > >>> >>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:815) >>> > > at >>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:866) >>> > > ... 5 more >>> > > >>> > > >>> > > Any help would be appreciated. >>> > > >>> > > Regards >>> > > Gokul >>> > > >>> > > >>> > > >>> > >>> > >>> > >>> >>> >>> >> >
