Try using "-files HStats-1A18.jar" as well: it will put it into Distributed Cache on the HDFS cluster... -- Alex K
On Fri, Dec 10, 2010 at 10:08 AM, Todd Lipcon <[email protected]> wrote: > You need to put the library jar on your classpath (eg using > HADOOP_CLASSPATH) as well. The -libjars will ship it to the cluster > and put it on the classpath of your task, but not the classpath of > your "driver" code. > > -Todd > > On Thu, Dec 9, 2010 at 10:29 PM, Vipul Pandey <[email protected]> wrote: > > disclaimer : a newbie!!! > > Howdy? > > Got a quick question. -libjars option doesn't seem to work for me in - > > prettymuch - my first (or mayby second) mapreduce job. > > Here's what i'm doing : > > $bin/hadoop jar sherlock.jar somepkg.FindSchoolsJob -libjars > > HStats-1A18.jar input output > > > > sherlock.jar has my main class (ofcourse) FindSchoolsJob, which runs > just > > fine by itself till I add a dependency on a class in HStats-1A18.jar. > > When I run the above command with -libjars specified - it fails to find > my > > classes that 'are' inside HStats jar file. > > Exception in thread "main" java.lang.NoClassDefFoundError: > com/*****/HAgent > > at com.*****.FindSchoolsJob.run(FindSchoolsJob.java:46) > > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) > > at com.******.FindSchoolsJob.main(FindSchoolsJob.java:101) > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > > at > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) > > at > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) > > at java.lang.reflect.Method.invoke(Method.java:597) > > at org.apache.hadoop.util.RunJar.main(RunJar.java:156) > > Caused by: java.lang.ClassNotFoundException:com/*****/HAgent > > at java.net.URLClassLoader$1.run(URLClassLoader.java:202) > > at java.security.AccessController.doPrivileged(Native Method) > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:307) > > at java.lang.ClassLoader.loadClass(ClassLoader.java:248) > > ... 8 more > > > > My main class is defined as below : > > public class FindSchoolsJob extends Configured implements Tool { > > : > > public int run(String[] args) throws Exception { > > : > > : > > } > > : > > public static void main(String[] args) throws Exception { > > int res = ToolRunner.run(new Configuration(), new FindSchoolsJob(), > > args); > > System.exit(res); > > } > > } > > Any hint would be highly appreciated. > > Thank You! > > ~V > > > > -- > Todd Lipcon > Software Engineer, Cloudera >
