Hi Mahadev
This is how I am running it. I created a test data file and put it in HDFS
using ./hadoop fs put command. Then I modified my WordCount sample to add
two lines to the main which look like this :
conf.set("fs.default.name", "hdfs://dev030.sctm.facebook.com:9000");
conf.set("mapred.job.tracker", "dev030.sctm.facebook.com:50029");
Then I run the sample by executing the command :
java org.apache.hadoop.examples.WordCount -m 10 -r 4
/user/alakshman/test.dat ~/output-dir
where the ~/output-dir is the local file system while the
/user/alakshman/test.dat is the path in HDFS.
This is when I get a whole slew of classnotfoundexceptions. Is the process
incorrect ?
Thanks
Avinash
On 5/29/07 1:17 PM, "Mahadev Konar" <[EMAIL PROTECTED]> wrote:
> Hi Avinash,
> Can you explain how you are running the wordcount example? Also, which
> version of Hadoop are you using? I would really recommend using 0.12.3.
>
> Regards
> Mahadev
>
>> > -----Original Message-----
>> > From: Phantom [mailto:[EMAIL PROTECTED]
>> > Sent: Tuesday, May 29, 2007 1:10 PM
>> > To: [email protected]
>> > Subject: Classpath setting problem for Map tasks
>> >
>> > How do I specify the classpath via the JobConf ? In my jobtracker web UI I
>> > see a whole slew of ClassNotFoundException's for the WordCount$MapClass.
>> > The
>> > stack trace is pasted below. But how do I fix this - just pass the -cp
>> > <classpath> to the mapred.child.java.opts property of the JobConf ?
>> > exporting CLASSPATH doesn't seem to work.hh
>> >
>> > Thanks
>> > Avinash
>> >
>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> > java.lang.ClassNotFoundException:
>> > org.apache.hadoop.examples.WordCount$MapClass
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344)
>> > at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:459)
>> > at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>> > at
>> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
>> > at
>> > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
>> > )
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
>> > at
>> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445)
>> > Caused by: java.lang.RuntimeException:
>> > java.lang.ClassNotFoundException:
>> > org.apache.hadoop.examples.WordCount$MapClass
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:328)
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339)
>> > ... 6 more
>> > Caused by: java.lang.ClassNotFoundException:
>> > org.apache.hadoop.examples.WordCount$MapClass
>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> > at java.lang.Class.forName0(Native Method)
>> > at java.lang.Class.forName(Class.java:242)
>> > at
>> > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315
>> > )
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326)
>> > ... 7 more
>> >
>> > java.lang.RuntimeException: java.lang.RuntimeException:
>> > java.lang.ClassNotFoundException:
>> > org.apache.hadoop.examples.WordCount$MapClass
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:344)
>> > at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:459)
>> > at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
>> > at
>> > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
>> > at
>> > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:82
>> > )
>> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:170)
>> > at
>> > org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1445)
>> > Caused by: java.lang.RuntimeException:
>> > java.lang.ClassNotFoundException:
>> > org.apache.hadoop.examples.WordCount$MapClass
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:328)
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:339)
>> > ... 6 more
>> > Caused by: java.lang.ClassNotFoundException:
>> > org.apache.hadoop.examples.WordCount$MapClass
>> > at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>> > at java.security.AccessController.doPrivileged(Native Method)
>> > at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:268)
>> > at java.lang.ClassLoader.loadClass(ClassLoader.java:251)
>> > at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
>> > at java.lang.Class.forName0(Native Method)
>> > at java.lang.Class.forName(Class.java:242)
>> > at
>> > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:315
>> > )
>> > at
>> > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:326)
>> > ... 7 more
>
>
