Hi, A couple of things:
1. Can you verify if running your job directly works? 2. Can you verify if your packaged jar does contain the classes? (using a command like jar -tvf <jar> should tell you). 2. The Oozie map-reduce action relies on the Stable API, not the New API. On Sun, Dec 9, 2012 at 1:10 AM, Mohammad Tariq <[email protected]> wrote: > Hello list, > > I have just started with Oozie and trying to follow the tutorial > at https://cwiki.apache.org/OOZIE/map-reduce-cookbook.html but I am facing > some issues as my "Mapper" class is not reachable(as per the JT logs). > Nothing complex, just tying to run the WordCount program. I have specified > the details below. in case anybody needs it : > > Location of the workflow components directory - > hdfs://localhost:9000/mr > > Contents inside /mr - > job.properties > workflow.xml > /lib > /lib/wc.jar > > Contents of job.properties - > nameNode=hdfs://localhost:9000 > jobTracker=localhost:9001 > queueName=default > examplesRoot=mr > oozie.wf.application.path=${nameNode}/${examplesRoot} > inputDir=/mapin > outputDir=/mapout > > Contents of workflow.xml - > <workflow-app name='wordcount-wf' xmlns="uri:oozie:workflow:0.2"> > <start to='wordcount'/> > <action name='wordcount'> > <map-reduce> > <job-tracker>${jobTracker}</job-tracker> > <name-node>${nameNode}</name-node> > <prepare> > </prepare> > <configuration> > <property> > <name>mapred.job.queue.name</name> > <value>${queueName}</value> > </property> > <property> > <name>mapred.mapper.class</name> > <value>WordCount.Map</value> > </property> > <property> > <name>mapred.reducer.class</name> > <value>WordCount.Reduce</value> > </property> > <property> > <name>mapred.input.dir</name> > <value>${inputDir}</value> > </property> > <property> > <name>mapred.output.dir</name> > <value>${outputDir}</value> > </property> > </configuration> > </map-reduce> > <ok to='end'/> > <error to='end'/> > </action> > <!--kill name='kill'> > <value>${wf:errorCode("wordcount")}</value> > </kill--> > <end name='end'/> > </workflow-app> > > Here is the program - > import java.io.IOException; > import java.lang.InterruptedException; > import java.util.StringTokenizer; > > import org.apache.hadoop.io.IntWritable; > import org.apache.hadoop.io.Text; > import org.apache.hadoop.conf.Configuration; > import org.apache.hadoop.fs.Path; > import org.apache.hadoop.mapreduce.Job; > import org.apache.hadoop.mapreduce.Mapper; > import org.apache.hadoop.mapreduce.Reducer; > import org.apache.hadoop.mapreduce.lib.input.FileInputFormat; > import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat; > import org.apache.hadoop.util.GenericOptionsParser; > > public class WordCount { > /** > * The map class of WordCount. > */ > public static class Map extends Mapper<Object, Text, Text, IntWritable> { > > private final static IntWritable one = new IntWritable(1); > private Text word = new Text(); > > public void map(Object key, Text value, Context context) > throws IOException, InterruptedException { > StringTokenizer itr = new StringTokenizer(value.toString()); > while (itr.hasMoreTokens()) { > word.set(itr.nextToken()); > context.write(word, one); > } > } > } > /** > * The reducer class of WordCount > */ > public static class Reduce extends Reducer<Text, IntWritable, Text, > IntWritable> { > public void reduce(Text key, Iterable<IntWritable> values, Context > context) > throws IOException, InterruptedException { > int sum = 0; > for (IntWritable value : values) { > sum += value.get(); > } > context.write(key, new IntWritable(sum)); > } > } > /** > * The main entry point. > */ > public static void main(String[] args) throws Exception { > Configuration conf = new Configuration(); > Job job = new Job(conf, "WordCount"); > job.setJarByClass(WordCount.class); > job.setMapperClass(Map.class); > job.setReducerClass(Reduce.class); > job.setOutputKeyClass(Text.class); > job.setOutputValueClass(IntWritable.class); > FileInputFormat.addInputPath(job, new Path("/mapin/")); > FileOutputFormat.setOutputPath(job, new Path("/mapout/")); > System.exit(job.waitForCompletion(true) ? 0 : 1); > } > } > > This is the error log - > > java.lang.RuntimeException: Error in configuring object > at > org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117) > at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432) > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) > at org.apache.hadoop.mapred.Child$4.run(Child.java:255) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) > at org.apache.hadoop.mapred.Child.main(Child.java:249) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:601) > at > org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88) > ... 9 more > Caused by: java.lang.RuntimeException: java.lang.RuntimeException: > java.lang.ClassNotFoundException: WordCount.Map > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899) > at org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:947) > at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34) > ... 14 more > Caused by: java.lang.RuntimeException: > java.lang.ClassNotFoundException: WordCount.Map > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891) > ... 16 more > Caused by: java.lang.ClassNotFoundException: WordCount.Map > at java.net.URLClassLoader$1.run(URLClassLoader.java:366) > at java.net.URLClassLoader$1.run(URLClassLoader.java:355) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:354) > at java.lang.ClassLoader.loadClass(ClassLoader.java:423) > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) > at java.lang.ClassLoader.loadClass(ClassLoader.java:356) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:264) > at > org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820) > at > org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865) > ... 17 more > > > Here is the command which I am using to submit the workfloe - > bin/oozie job -oozie http://localhost:11000/oozie/ -config > ~/mr/job.properties -run > > Need some help. Many thanks. > (Please pardon my ignorance) > > Regards, > Mohammad Tariq -- Harsh J
