The combo is OK, but the use of the mapreduce API may not be. I'd try removing inner classes and placing them as regular Map and Reduce classes and reconfiguring/resubmitting.
On Sun, Dec 9, 2012 at 2:06 AM, Mohammad Tariq <[email protected]> wrote: > Thank you so much for the quick response Harsh. Please find answers to your > questions below(in order) : > > 1- Yes, it works. > 2- Yes, jar contains all the classes. > 3- I am trying to use oozie-3.2.0-incubating + hadoop-1.0.4. Is this > combination OK? > > Regards, > Mohammad Tariq > > > > On Sun, Dec 9, 2012 at 1:59 AM, Harsh J <[email protected]> wrote: > >> Hi, >> >> A couple of things: >> >> 1. Can you verify if running your job directly works? >> 2. Can you verify if your packaged jar does contain the classes? >> (using a command like jar -tvf <jar> should tell you). >> 2. The Oozie map-reduce action relies on the Stable API, not the New API. >> >> On Sun, Dec 9, 2012 at 1:10 AM, Mohammad Tariq <[email protected]> wrote: >> > Hello list, >> > >> > I have just started with Oozie and trying to follow the tutorial >> > at https://cwiki.apache.org/OOZIE/map-reduce-cookbook.html but I am >> facing >> > some issues as my "Mapper" class is not reachable(as per the JT logs). >> > Nothing complex, just tying to run the WordCount program. I have >> specified >> > the details below. in case anybody needs it : >> > >> > Location of the workflow components directory - >> > hdfs://localhost:9000/mr >> > >> > Contents inside /mr - >> > job.properties >> > workflow.xml >> > /lib >> > /lib/wc.jar >> > >> > Contents of job.properties - >> > nameNode=hdfs://localhost:9000 >> > jobTracker=localhost:9001 >> > queueName=default >> > examplesRoot=mr >> > oozie.wf.application.path=${nameNode}/${examplesRoot} >> > inputDir=/mapin >> > outputDir=/mapout >> > >> > Contents of workflow.xml - >> > <workflow-app name='wordcount-wf' xmlns="uri:oozie:workflow:0.2"> >> > <start to='wordcount'/> >> > <action name='wordcount'> >> > <map-reduce> >> > <job-tracker>${jobTracker}</job-tracker> >> > <name-node>${nameNode}</name-node> >> > <prepare> >> > </prepare> >> > <configuration> >> > <property> >> > <name>mapred.job.queue.name</name> >> > <value>${queueName}</value> >> > </property> >> > <property> >> > <name>mapred.mapper.class</name> >> > <value>WordCount.Map</value> >> > </property> >> > <property> >> > <name>mapred.reducer.class</name> >> > <value>WordCount.Reduce</value> >> > </property> >> > <property> >> > <name>mapred.input.dir</name> >> > <value>${inputDir}</value> >> > </property> >> > <property> >> > <name>mapred.output.dir</name> >> > <value>${outputDir}</value> >> > </property> >> > </configuration> >> > </map-reduce> >> > <ok to='end'/> >> > <error to='end'/> >> > </action> >> > <!--kill name='kill'> >> > <value>${wf:errorCode("wordcount")}</value> >> > </kill--> >> > <end name='end'/> >> > </workflow-app> >> > >> > Here is the program - >> > import java.io.IOException; >> > import java.lang.InterruptedException; >> > import java.util.StringTokenizer; >> > >> > import org.apache.hadoop.io.IntWritable; >> > import org.apache.hadoop.io.Text; >> > import org.apache.hadoop.conf.Configuration; >> > import org.apache.hadoop.fs.Path; >> > import org.apache.hadoop.mapreduce.Job; >> > import org.apache.hadoop.mapreduce.Mapper; >> > import org.apache.hadoop.mapreduce.Reducer; >> > import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; >> > import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; >> > import org.apache.hadoop.util.GenericOptionsParser; >> > >> > public class WordCount { >> > /** >> > * The map class of WordCount. >> > */ >> > public static class Map extends Mapper<Object, Text, Text, IntWritable> { >> > >> > private final static IntWritable one = new IntWritable(1); >> > private Text word = new Text(); >> > >> > public void map(Object key, Text value, Context context) >> > throws IOException, InterruptedException { >> > StringTokenizer itr = new StringTokenizer(value.toString()); >> > while (itr.hasMoreTokens()) { >> > word.set(itr.nextToken()); >> > context.write(word, one); >> > } >> > } >> > } >> > /** >> > * The reducer class of WordCount >> > */ >> > public static class Reduce extends Reducer<Text, IntWritable, Text, >> > IntWritable> { >> > public void reduce(Text key, Iterable<IntWritable> values, Context >> > context) >> > throws IOException, InterruptedException { >> > int sum = 0; >> > for (IntWritable value : values) { >> > sum += value.get(); >> > } >> > context.write(key, new IntWritable(sum)); >> > } >> > } >> > /** >> > * The main entry point. >> > */ >> > public static void main(String[] args) throws Exception { >> > Configuration conf = new Configuration(); >> > Job job = new Job(conf, "WordCount"); >> > job.setJarByClass(WordCount.class); >> > job.setMapperClass(Map.class); >> > job.setReducerClass(Reduce.class); >> > job.setOutputKeyClass(Text.class); >> > job.setOutputValueClass(IntWritable.class); >> > FileInputFormat.addInputPath(job, new Path("/mapin/")); >> > FileOutputFormat.setOutputPath(job, new Path("/mapout/")); >> > System.exit(job.waitForCompletion(true) ? 0 : 1); >> > } >> > } >> > >> > This is the error log - >> > >> > java.lang.RuntimeException: Error in configuring object >> > at >> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) >> > at >> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) >> > at >> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) >> > at >> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432) >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) >> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) >> > at java.security.AccessController.doPrivileged(Native Method) >> > at javax.security.auth.Subject.doAs(Subject.java:415) >> > at >> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) >> > at org.apache.hadoop.mapred.Child.main(Child.java:249) >> > Caused by: java.lang.reflect.InvocationTargetException >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >> > at >> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) >> > at >> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >> > at java.lang.reflect.Method.invoke(Method.java:601) >> > at >> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) >> > ... 9 more >> > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: >> > java.lang.ClassNotFoundException: WordCount.Map >> > at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899) >> > at >> org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:947) >> > at >> org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) >> > ... 14 more >> > Caused by: java.lang.RuntimeException: >> > java.lang.ClassNotFoundException: WordCount.Map >> > at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867) >> > at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891) >> > ... 16 more >> > Caused by: java.lang.ClassNotFoundException: WordCount.Map >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) >> > at java.security.AccessController.doPrivileged(Native Method) >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:423) >> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:356) >> > at java.lang.Class.forName0(Native Method) >> > at java.lang.Class.forName(Class.java:264) >> > at >> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820) >> > at >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865) >> > ... 17 more >> > >> > >> > Here is the command which I am using to submit the workfloe - >> > bin/oozie job -oozie http://localhost:11000/oozie/ -config >> > ~/mr/job.properties -run >> > >> > Need some help. Many thanks. >> > (Please pardon my ignorance) >> > >> > Regards, >> > Mohammad Tariq >> >> >> >> -- >> Harsh J >> -- Harsh J
