Thanks a million. Everything works now. Thanks Avinash
On 5/29/07 1:51 PM, "Mahadev Konar" <[EMAIL PROTECTED]> wrote: > Hi Avinash, > Java org.apache.... will not work. This does not copy the job.jar and > that's why you run into these Classnotfoundexcpetions. > The way to run the wordocunt example is: > Bin/hadoop jar hadoop-examples.jar wordcount ....... > > This way the jobclient is called by the hadoop command to transfer the > job.jar and get the status of the job. > > Regards > Mahadev > >> > -----Original Message----- >> > From: Avinash Lakshman [mailto:[EMAIL PROTECTED] >> > Sent: Tuesday, May 29, 2007 1:30 PM >> > To: [email protected] >> > Subject: Re: Classpath setting problem for Map tasks >> > >> > Hi Mahadev >> > >> > This is how I am running it. I created a test data file and put it in HDFS >> > using ./hadoop fs -put command. Then I modified my WordCount sample to add >> > two lines to the main which look like this : >> > >> > conf.set("fs.default.name", "hdfs://dev030.sctm.facebook.com:9000"); >> > conf.set("mapred.job.tracker", "dev030.sctm.facebook.com:50029"); >> > >> > Then I run the sample by executing the command : >> > >> > java org.apache.hadoop.examples.WordCount -m 10 -r 4 >> > /user/alakshman/test.dat ~/output-dir >> > >> > where the ~/output-dir is the local file system while the >> > /user/alakshman/test.dat is the path in HDFS. >> > >> > This is when I get a whole slew of classnotfoundexceptions. Is the process >> > incorrect ? >> > >> > Thanks >> > Avinash >> > >> > >> > >> > On 5/29/07 1:17 PM, "Mahadev Konar" <[EMAIL PROTECTED]> wrote: >> > >>> > > Hi Avinash, >>> > > Can you explain how you are running the wordcount example? Also, which >>> > > version of Hadoop are you using? I would really recommend using 0.12.3. >>> > > >>> > > Regards >>> > > Mahadev >>> > > >>>>> > >> > -----Original Message----- >>>>> > >> > From: Phantom [mailto:[EMAIL PROTECTED] >>>>> > >> > Sent: Tuesday, May 29, 2007 1:10 PM >>>>> > >> > To: [email protected] >>>>> > >> > Subject: Classpath setting problem for Map tasks >>>>> > >> > >>>>> > >> > How do I specify the classpath via the JobConf ? In my jobtracker web >> > UI I >>>>> > >> > see a whole slew of ClassNotFoundException's for the >> > WordCount$MapClass. >>>>> > >> > The >>>>> > >> > stack trace is pasted below. But how do I fix this - just pass the - >> > cp >>>>> > >> > <classpath> to the mapred.child.java.opts property of the JobConf ? >>>>> > >> > exporting CLASSPATH doesn't seem to work.hh >>>>> > >> > >>>>> > >> > Thanks >>>>> > >> > Avinash >>>>> > >> > >>>>> > >> > java.lang.RuntimeException: java.lang.RuntimeException: >>>>> > >> > java.lang.ClassNotFoundException: >>>>> > >> > org.apache.hadoop.examples.WordCount$MapClass >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344) >>>>> > >> > at >> > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:459) >>>>> > >> > at >> > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82 >>>>> > >> > ) >>>>> > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:170) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445) >>>>> > >> > Caused by: java.lang.RuntimeException: >>>>> > >> > java.lang.ClassNotFoundException: >>>>> > >> > org.apache.hadoop.examples.WordCount$MapClass >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:328) >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339) >>>>> > >> > ... 6 more >>>>> > >> > Caused by: java.lang.ClassNotFoundException: >>>>> > >> > org.apache.hadoop.examples.WordCount$MapClass >>>>> > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >>>>> > >> > at java.security.AccessController.doPrivileged(Native Method) >>>>> > >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >>>>> > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>> > >> > at >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) >>>>> > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) >>>>> > >> > at >> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) >>>>> > >> > at java.lang.Class.forName0(Native Method) >>>>> > >> > at java.lang.Class.forName(Class.java:242) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315 >>>>> > >> > ) >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326) >>>>> > >> > ... 7 more >>>>> > >> > >>>>> > >> > java.lang.RuntimeException: java.lang.RuntimeException: >>>>> > >> > java.lang.ClassNotFoundException: >>>>> > >> > org.apache.hadoop.examples.WordCount$MapClass >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344) >>>>> > >> > at >> > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:459) >>>>> > >> > at >> > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82 >>>>> > >> > ) >>>>> > >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:170) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445) >>>>> > >> > Caused by: java.lang.RuntimeException: >>>>> > >> > java.lang.ClassNotFoundException: >>>>> > >> > org.apache.hadoop.examples.WordCount$MapClass >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:328) >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339) >>>>> > >> > ... 6 more >>>>> > >> > Caused by: java.lang.ClassNotFoundException: >>>>> > >> > org.apache.hadoop.examples.WordCount$MapClass >>>>> > >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200) >>>>> > >> > at java.security.AccessController.doPrivileged(Native Method) >>>>> > >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188) >>>>> > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>>>> > >> > at >> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268) >>>>> > >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251) >>>>> > >> > at >> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319) >>>>> > >> > at java.lang.Class.forName0(Native Method) >>>>> > >> > at java.lang.Class.forName(Class.java:242) >>>>> > >> > at >>>>> > >> > >> > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315 >>>>> > >> > ) >>>>> > >> > at >>>>> > >> > >>>>> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326) >>>>> > >> > ... 7 more >>> > > >>> > > >> > > > >
