Sweeet! That solved the problem! I was assuming that there must be a classloader or something that loads the -libjars locally as well. i was wrong after-all! Thank You. ~V On Dec 10, 2010, at 10:08 AM, Todd Lipcon wrote:
> You need to put the library jar on your classpath (eg using > HADOOP_CLASSPATH) as well. The -libjars will ship it to the cluster > and put it on the classpath of your task, but not the classpath of > your "driver" code. > > -Todd > > On Thu, Dec 9, 2010 at 10:29 PM, Vipul Pandey <[email protected]> wrote: >> disclaimer : a newbie!!! >> Howdy? >> Got a quick question. -libjars option doesn't seem to work for me in - >> prettymuch - my first (or mayby second) mapreduce job. >> Here's what i'm doing : >> $bin/hadoop jar sherlock.jar somepkg.FindSchoolsJob -libjars >> HStats-1A18.jar input output >> >> sherlock.jar has my main class (ofcourse) FindSchoolsJob, which runs just >> fine by itself till I add a dependency on a class in HStats-1A18.jar. >> When I run the above command with -libjars specified - it fails to find my >> classes that 'are' inside HStats jar file. >> Exception in thread "main" java.lang.NoClassDefFoundError: com/*****/HAgent >> at com.*****.FindSchoolsJob.run(FindSchoolsJob.java:46) >> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) >> at com.******.FindSchoolsJob.main(FindSchoolsJob.java:101) >> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) >> at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) >> at java.lang.reflect.Method.invoke(Method.java:597) >> at org.apache.hadoop.util.RunJar.main(RunJar.java:156) >> Caused by: java.lang.ClassNotFoundException:com/*****/HAgent >> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >> at java.security.AccessController.doPrivileged(Native Method) >> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:307) >> at java.lang.ClassLoader.loadClass(ClassLoader.java:248) >> ... 8 more >> >> My main class is defined as below : >> public class FindSchoolsJob extends Configured implements Tool { >> : >> public int run(String[] args) throws Exception { >> : >> : >> } >> : >> public static void main(String[] args) throws Exception { >> int res = ToolRunner.run(new Configuration(), new FindSchoolsJob(), >> args); >> System.exit(res); >> } >> } >> Any hint would be highly appreciated. >> Thank You! >> ~V > > > > -- > Todd Lipcon > Software Engineer, Cloudera
