Hi Avinash, Java org.apache.... will not work. This does not copy the job.jar and that's why you run into these Classnotfoundexcpetions. The way to run the wordocunt example is: Bin/hadoop jar hadoop-examples.jar wordcount .......
This way the jobclient is called by the hadoop command to transfer the job.jar and get the status of the job. Regards Mahadev > -----Original Message----- > From: Avinash Lakshman [mailto:[EMAIL PROTECTED] > Sent: Tuesday, May 29, 2007 1:30 PM > To: [email protected] > Subject: Re: Classpath setting problem for Map tasks > > Hi Mahadev > > This is how I am running it. I created a test data file and put it in HDFS > using ./hadoop fs -put command. Then I modified my WordCount sample to add > two lines to the main which look like this : > > conf.set("fs.default.name", "hdfs://dev030.sctm.facebook.com:9000"); > conf.set("mapred.job.tracker", "dev030.sctm.facebook.com:50029"); > > Then I run the sample by executing the command : > > java org.apache.hadoop.examples.WordCount -m 10 -r 4 > /user/alakshman/test.dat ~/output-dir > > where the ~/output-dir is the local file system while the > /user/alakshman/test.dat is the path in HDFS. > > This is when I get a whole slew of classnotfoundexceptions. Is the process > incorrect ? > > Thanks > Avinash > > > > On 5/29/07 1:17 PM, "Mahadev Konar" <[EMAIL PROTECTED]> wrote: > > > Hi Avinash, > > Can you explain how you are running the wordcount example? Also, which > > version of Hadoop are you using? I would really recommend using 0.12.3. > > > > Regards > > Mahadev > > > >> > -----Original Message----- > >> > From: Phantom [mailto:[EMAIL PROTECTED] > >> > Sent: Tuesday, May 29, 2007 1:10 PM > >> > To: [email protected] > >> > Subject: Classpath setting problem for Map tasks > >> > > >> > How do I specify the classpath via the JobConf ? In my jobtracker web > UI I > >> > see a whole slew of ClassNotFoundException's for the > WordCount$MapClass. > >> > The > >> > stack trace is pasted below. But how do I fix this - just pass the - > cp > >> > <classpath> to the mapred.child.java.opts property of the JobConf ? > >> > exporting CLASSPATH doesn't seem to work.hh > >> > > >> > Thanks > >> > Avinash > >> > > >> > java.lang.RuntimeException: java.lang.RuntimeException: > >> > java.lang.ClassNotFoundException: > >> > org.apache.hadoop.examples.WordCount$MapClass > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344) > >> > at > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:459) > >> > at > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) > >> > at > >> > > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) > >> > at > >> > > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82 > >> > ) > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:170) > >> > at > >> > > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445) > >> > Caused by: java.lang.RuntimeException: > >> > java.lang.ClassNotFoundException: > >> > org.apache.hadoop.examples.WordCount$MapClass > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:328) > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339) > >> > ... 6 more > >> > Caused by: java.lang.ClassNotFoundException: > >> > org.apache.hadoop.examples.WordCount$MapClass > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > >> > at java.security.AccessController.doPrivileged(Native Method) > >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > >> > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > >> > at > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) > >> > at java.lang.Class.forName0(Native Method) > >> > at java.lang.Class.forName(Class.java:242) > >> > at > >> > > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315 > >> > ) > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326) > >> > ... 7 more > >> > > >> > java.lang.RuntimeException: java.lang.RuntimeException: > >> > java.lang.ClassNotFoundException: > >> > org.apache.hadoop.examples.WordCount$MapClass > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344) > >> > at > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:459) > >> > at > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) > >> > at > >> > > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) > >> > at > >> > > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82 > >> > ) > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:170) > >> > at > >> > > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445) > >> > Caused by: java.lang.RuntimeException: > >> > java.lang.ClassNotFoundException: > >> > org.apache.hadoop.examples.WordCount$MapClass > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:328) > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339) > >> > ... 6 more > >> > Caused by: java.lang.ClassNotFoundException: > >> > org.apache.hadoop.examples.WordCount$MapClass > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) > >> > at java.security.AccessController.doPrivileged(Native Method) > >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) > >> > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) > >> > at > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) > >> > at java.lang.Class.forName0(Native Method) > >> > at java.lang.Class.forName(Class.java:242) > >> > at > >> > > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315 > >> > ) > >> > at > >> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326) > >> > ... 7 more > > > > >
