Thanks, this seems to make it work. In fact, I did not even need to specify lib jars in the command line…should I be worried that it doesn't work that way?
On Jan 31, 2012, at 4:09 PM, Joey Echeverria wrote: > You also need to add the jar to the classpath so it's available in > your main. You can do soemthing like this: > > HADOOP_CLASSPATH=/usr/local/mahout/math/target/mahout-math-0.6-SNAPSHOT.jar > hadoop jar ... > > -Joey > > On Tue, Jan 31, 2012 at 1:38 PM, Daniel Quach <[email protected]> wrote: >> For Hadoop 0.20.203 (the latest stable), is it sufficient to do this to >> parse the lib jars from the command line? >> >> public static main (String args[]) { >> Configuration conf = new Configuration(); >> String[] otherArgs = new GenericOptionsParser(conf, args).getRemainingArgs(); >> >> Job job = new Job(conf, "makevector"); >> job.setJarByClass(MakeVector.class); >> // etc other code for mappers/reducers >> } >> >> I'm thinking I am missing a step here which is why it won't load the mahout >> jar while running the map reduce. >> >> On Jan 30, 2012, at 11:10 PM, Daniel Quach wrote: >> >>> I compiled using javac: >>> >>> javac -classpath >>> :/usr/local/hadoop/hadoop-core-0.20.203.0.jar:/usr/local/hadoop/lib/commons-cli-1.2.jar:/usr/local/mahout/math/target/mahout-math-0.6-SNAPSHOT.jar >>> -d makevector_classes/ MakeVector.java; >>> >>> If I don't include the mahout-math jar, it gives me a compile error because >>> of DenseVector. >>> >>> >>> On Jan 30, 2012, at 10:42 PM, Prashant Kommireddi wrote: >>> >>>> How are you building the mapreduce jar? Try not to include the Mahout dist >>>> while building MR jar, and include it only on "-libjars" option. >>>> >>>> On Mon, Jan 30, 2012 at 10:33 PM, Daniel Quach <[email protected]> >>>> wrote: >>>> >>>>> I have been compiling my mapreduce with the jars in the classpath, and I >>>>> believe I need to also add the jars as an option to -libjars to hadoop. >>>>> However, even when I do this, I still get an error complaining about >>>>> missing classes at runtime. (Compilation works fine). >>>>> >>>>> Here is my command: >>>>> hadoop jar makevector.jar org.myorg.MakeVector -libjars >>>>> /usr/local/mahout/math/target/mahout-math-0.6-SNAPSHOT.jar input/ output/ >>>>> >>>>> This is the error I receive: >>>>> Exception in thread "main" java.lang.NoClassDefFoundError: >>>>> org/apache/mahout/math/DenseVector >>>>> >>>>> I wonder if I am using the GenericOptionsParser incorrectly? I'm not sure >>>>> if there is a deeper problem here. >>>>> >>> >> > > > > -- > Joseph Echeverria > Cloudera, Inc. > 443.305.9434
