Try to give full path or ./ Sent from my iPhone
On Nov 29, 2012, at 1:00 PM, "Kartashov, Andy" <[email protected]> wrote: > Maybe you stepped out of your working directory. “$ ls –l” Do you see your > .jar? > > From: Sandeep Jangra [mailto:[email protected]] > Sent: Thursday, November 29, 2012 3:46 PM > To: [email protected] > Subject: Re: Trouble with Word Count example > > Hi Harsh, > > I tried putting the generic option first, but it throws exception file not > found. > The jar is in current directory. Then I tried giving absolute path of this > jar, but that also brought no luck. > > sudo -u hdfs hadoop jar word_cnt.jar WordCount2 -libjars=word_cnt.jar > /tmp/root/input /tmp/root/output17 > Exception in thread "main" java.io.FileNotFoundException: File word_cnt.jar > does not exist. > at > org.apache.hadoop.util.GenericOptionsParser.validateFiles(GenericOptionsParser.java:384) > at > org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:280) > at > org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:418) > at > org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:168) > at > org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:151) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64) > > Also, I have been deleting my jars and the class directory before each new > try. So even I am suspicious why do I see this: > "12/11/29 10:20:59 WARN mapred.JobClient: No job jar file set. User classes > may not be found. See JobConf(Class) or JobConf#setJar(String)." > > Could it be that my hadoop is running on old jar files (the one with > package name "mapred" (not mapreduce)) > But my program is using new jars as well. > > I can try going back to old word count example on the apache site and using > old jars. > > Any other pointers would be highly appreciated. Thanks > > > > On Thu, Nov 29, 2012 at 2:42 PM, Harsh J <[email protected]> wrote: > I think you may have not recompiled your application properly. > > Your runtime shows this: > > 12/11/29 10:20:59 WARN mapred.JobClient: No job jar file set. User > classes may not be found. See JobConf(Class) or > JobConf#setJar(String). > > Which should not appear, cause your code has this (which I suspect you > may have added later, accidentally?): > > job.setJarByClass(WordCount2.class); > > So if you can try deleting the older jar and recompiling it, the > problem would go away. > > Also, when passing generic options such as -libjars, etc., they need > to go first in order. I mean, it should always be [Classname] [Generic > Options] [Application Options]. Otherwise, they may not get utilized > properly. > > On Fri, Nov 30, 2012 at 12:51 AM, Sandeep Jangra > <[email protected]> wrote: > > Yups I can see my class files there. > > > > > > On Thu, Nov 29, 2012 at 2:13 PM, Kartashov, Andy <[email protected]> > > wrote: > >> > >> Can you try running jar –tvf word_cnt.jar and see if your static nested > >> classes WordCount2$Map.class and WordCount2$Reduce.class have actually been > >> added to the jar. > >> > >> > >> > >> Rgds, > >> > >> AK47 > >> > >> > >> > >> > >> > >> From: Sandeep Jangra [mailto:[email protected]] > >> Sent: Thursday, November 29, 2012 1:36 PM > >> To: [email protected] > >> Subject: Re: Trouble with Word Count example > >> > >> > >> > >> Also, I did set the HADOOP_CLASSPATH variable to point to the word_cnt.jar > >> only. > >> > >> > >> > >> On Thu, Nov 29, 2012 at 10:54 AM, Sandeep Jangra <[email protected]> > >> wrote: > >> > >> Thanks for the quick response Mahesh. > >> > >> > >> > >> I am using the following command: > >> > >> > >> > >> sudo -u hdfs hadoop jar word_cnt.jar WordCount2 /tmp/root/input > >> /tmp/root/output15 -libjars=word_cnt.jar > >> > >> (The input directory exists on the hdfs) > >> > >> > >> > >> This is how I compiled and packaged it: > >> > >> > >> > >> javac -classpath > >> /usr/lib/hadoop-0.20-mapreduce/hadoop-core.jar:/usr/lib/hadoop/* -d > >> word_cnt WordCount2.java > >> > >> jar -cvf word_cnt.jar -C word_cnt/ . > >> > >> > >> > >> > >> > >> > >> > >> On Thu, Nov 29, 2012 at 10:46 AM, Mahesh Balija > >> <[email protected]> wrote: > >> > >> Hi Sandeep, > >> > >> > >> > >> For me everything seems to be alright. > >> > >> Can you tell us how are you running this job? > >> > >> > >> > >> Best, > >> > >> Mahesh.B. > >> > >> Calsoft Labs. > >> > >> On Thu, Nov 29, 2012 at 9:01 PM, Sandeep Jangra <[email protected]> > >> wrote: > >> > >> Hello everyone, > >> > >> > >> > >> Like most others I am also running into some problems while running my > >> word count example. > >> > >> I tried the various suggestion available on internet, but I guess it;s > >> time to go on email :) > >> > >> > >> > >> Here is the error that I am getting: > >> > >> 12/11/29 10:20:59 WARN mapred.JobClient: Use GenericOptionsParser for > >> parsing the arguments. Applications should implement Tool for the same. > >> > >> 12/11/29 10:20:59 WARN mapred.JobClient: No job jar file set. User > >> classes may not be found. See JobConf(Class) or JobConf#setJar(String). > >> > >> 12/11/29 10:20:59 INFO input.FileInputFormat: Total input paths to process > >> : 1 > >> > >> 12/11/29 10:20:59 INFO util.NativeCodeLoader: Loaded the native-hadoop > >> library > >> > >> 12/11/29 10:20:59 WARN snappy.LoadSnappy: Snappy native library is > >> available > >> > >> 12/11/29 10:20:59 INFO snappy.LoadSnappy: Snappy native library loaded > >> > >> 12/11/29 10:21:00 INFO mapred.JobClient: Running job: > >> job_201210310210_0040 > >> > >> 12/11/29 10:21:01 INFO mapred.JobClient: map 0% reduce 0% > >> > >> 12/11/29 10:21:07 INFO mapred.JobClient: Task Id : > >> attempt_201210310210_0040_m_000000_0, Status : FAILED > >> > >> java.lang.RuntimeException: java.lang.ClassNotFoundException: Class > >> WordCount2$Map not found > >> > >> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1439) > >> > >> at > >> org.apache.hadoop.mapreduce.task.JobContextImpl.getMapperClass(JobContextImpl.java:191) > >> > >> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:605) > >> > >> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325) > >> > >> at org.apache.hadoop.mapred.Child$4.run(Child.java:270) > >> > >> at java.security.AccessController.doPrivileged(Native Method) > >> > >> at javax.security.auth.Subject.doAs(Subject.java:416) > >> > >> at > >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232) > >> > >> at org.apache.hadoop.mapred.Child.main(Child.java:264) > >> > >> Caused by: java.lang.ClassNotFoundException: Class WordCount2$Map not > >> found > >> > >> at > >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1350) > >> > >> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1437) > >> > >> ... 8 more > >> > >> > >> > >> And here is the source code: > >> > >> > >> > >> > >> > >> import org.apache.hadoop.conf.Configuration; > >> > >> import org.apache.hadoop.conf.Configured; > >> > >> import org.apache.hadoop.fs.Path; > >> > >> import org.apache.hadoop.io.IntWritable; > >> > >> import org.apache.hadoop.io.LongWritable; > >> > >> import org.apache.hadoop.io.Text; > >> > >> import org.apache.hadoop.mapreduce.Job; > >> > >> import org.apache.hadoop.mapreduce.Mapper; > >> > >> import org.apache.hadoop.mapreduce.Reducer; > >> > >> import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; > >> > >> import org.apache.hadoop.mapreduce.lib.input.TextInputFormat; > >> > >> import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; > >> > >> import org.apache.hadoop.mapreduce.lib.output.TextOutputFormat; > >> > >> import org.apache.hadoop.util.Tool; > >> > >> import org.apache.hadoop.util.ToolRunner; > >> > >> > >> > >> import java.io.IOException; > >> > >> import java.util.StringTokenizer; > >> > >> > >> > >> public class WordCount2 extends Configured implements Tool { > >> > >> > >> > >> > >> > >> public static class Map extends Mapper<LongWritable, Text, Text, > >> IntWritable> { > >> > >> private final static IntWritable one = new IntWritable(1); > >> > >> private Text word = new Text(); > >> > >> > >> > >> @Override > >> > >> protected void map(LongWritable key, Text value, Context context) > >> throws IOException, InterruptedException { > >> > >> String line = value.toString(); > >> > >> StringTokenizer tokenizer = new StringTokenizer(line); > >> > >> while (tokenizer.hasMoreTokens()) { > >> > >> word.set(tokenizer.nextToken()); > >> > >> context.write(word, one); > >> > >> } > >> > >> } > >> > >> > >> > >> } > >> > >> > >> > >> > >> > >> public static class Reduce extends Reducer<Text, IntWritable, Text, > >> IntWritable> { > >> > >> > >> > >> @Override > >> > >> protected void reduce(Text key, Iterable<IntWritable> values, > >> Context context) throws IOException, InterruptedException { > >> > >> > >> > >> int sum = 0; > >> > >> > >> > >> &n
