I see. Will give it a shot and see if that works.
Thank you.
Regards,
Mohammad Tariq
On Sun, Dec 9, 2012 at 2:10 AM, Harsh J <[email protected]> wrote:
> The combo is OK, but the use of the mapreduce API may not be.
>
> I'd try removing inner classes and placing them as regular Map and
> Reduce classes and reconfiguring/resubmitting.
>
> On Sun, Dec 9, 2012 at 2:06 AM, Mohammad Tariq <[email protected]> wrote:
> > Thank you so much for the quick response Harsh. Please find answers to
> your
> > questions below(in order) :
> >
> > 1- Yes, it works.
> > 2- Yes, jar contains all the classes.
> > 3- I am trying to use oozie-3.2.0-incubating + hadoop-1.0.4. Is this
> > combination OK?
> >
> > Regards,
> > Mohammad Tariq
> >
> >
> >
> > On Sun, Dec 9, 2012 at 1:59 AM, Harsh J <[email protected]> wrote:
> >
> >> Hi,
> >>
> >> A couple of things:
> >>
> >> 1. Can you verify if running your job directly works?
> >> 2. Can you verify if your packaged jar does contain the classes?
> >> (using a command like jar -tvf <jar> should tell you).
> >> 2. The Oozie map-reduce action relies on the Stable API, not the New
> API.
> >>
> >> On Sun, Dec 9, 2012 at 1:10 AM, Mohammad Tariq <[email protected]>
> wrote:
> >> > Hello list,
> >> >
> >> > I have just started with Oozie and trying to follow the
> tutorial
> >> > at https://cwiki.apache.org/OOZIE/map-reduce-cookbook.html but I am
> >> facing
> >> > some issues as my "Mapper" class is not reachable(as per the JT logs).
> >> > Nothing complex, just tying to run the WordCount program. I have
> >> specified
> >> > the details below. in case anybody needs it :
> >> >
> >> > Location of the workflow components directory -
> >> > hdfs://localhost:9000/mr
> >> >
> >> > Contents inside /mr -
> >> > job.properties
> >> > workflow.xml
> >> > /lib
> >> > /lib/wc.jar
> >> >
> >> > Contents of job.properties -
> >> > nameNode=hdfs://localhost:9000
> >> > jobTracker=localhost:9001
> >> > queueName=default
> >> > examplesRoot=mr
> >> > oozie.wf.application.path=${nameNode}/${examplesRoot}
> >> > inputDir=/mapin
> >> > outputDir=/mapout
> >> >
> >> > Contents of workflow.xml -
> >> > <workflow-app name='wordcount-wf' xmlns="uri:oozie:workflow:0.2">
> >> > <start to='wordcount'/>
> >> > <action name='wordcount'>
> >> > <map-reduce>
> >> > <job-tracker>${jobTracker}</job-tracker>
> >> > <name-node>${nameNode}</name-node>
> >> > <prepare>
> >> > </prepare>
> >> > <configuration>
> >> > <property>
> >> > <name>mapred.job.queue.name</name>
> >> > <value>${queueName}</value>
> >> > </property>
> >> > <property>
> >> > <name>mapred.mapper.class</name>
> >> > <value>WordCount.Map</value>
> >> > </property>
> >> > <property>
> >> > <name>mapred.reducer.class</name>
> >> > <value>WordCount.Reduce</value>
> >> > </property>
> >> > <property>
> >> > <name>mapred.input.dir</name>
> >> > <value>${inputDir}</value>
> >> > </property>
> >> > <property>
> >> > <name>mapred.output.dir</name>
> >> > <value>${outputDir}</value>
> >> > </property>
> >> > </configuration>
> >> > </map-reduce>
> >> > <ok to='end'/>
> >> > <error to='end'/>
> >> > </action>
> >> > <!--kill name='kill'>
> >> > <value>${wf:errorCode("wordcount")}</value>
> >> > </kill-->
> >> > <end name='end'/>
> >> > </workflow-app>
> >> >
> >> > Here is the program -
> >> > import java.io.IOException;
> >> > import java.lang.InterruptedException;
> >> > import java.util.StringTokenizer;
> >> >
> >> > import org.apache.hadoop.io.IntWritable;
> >> > import org.apache.hadoop.io.Text;
> >> > import org.apache.hadoop.conf.Configuration;
> >> > import org.apache.hadoop.fs.Path;
> >> > import org.apache.hadoop.mapreduce.Job;
> >> > import org.apache.hadoop.mapreduce.Mapper;
> >> > import org.apache.hadoop.mapreduce.Reducer;
> >> > import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
> >> > import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
> >> > import org.apache.hadoop.util.GenericOptionsParser;
> >> >
> >> > public class WordCount {
> >> > /**
> >> > * The map class of WordCount.
> >> > */
> >> > public static class Map extends Mapper<Object, Text, Text,
> IntWritable> {
> >> >
> >> > private final static IntWritable one = new IntWritable(1);
> >> > private Text word = new Text();
> >> >
> >> > public void map(Object key, Text value, Context context)
> >> > throws IOException, InterruptedException {
> >> > StringTokenizer itr = new StringTokenizer(value.toString());
> >> > while (itr.hasMoreTokens()) {
> >> > word.set(itr.nextToken());
> >> > context.write(word, one);
> >> > }
> >> > }
> >> > }
> >> > /**
> >> > * The reducer class of WordCount
> >> > */
> >> > public static class Reduce extends Reducer<Text, IntWritable, Text,
> >> > IntWritable> {
> >> > public void reduce(Text key, Iterable<IntWritable> values, Context
> >> > context)
> >> > throws IOException, InterruptedException {
> >> > int sum = 0;
> >> > for (IntWritable value : values) {
> >> > sum += value.get();
> >> > }
> >> > context.write(key, new IntWritable(sum));
> >> > }
> >> > }
> >> > /**
> >> > * The main entry point.
> >> > */
> >> > public static void main(String[] args) throws Exception {
> >> > Configuration conf = new Configuration();
> >> > Job job = new Job(conf, "WordCount");
> >> > job.setJarByClass(WordCount.class);
> >> > job.setMapperClass(Map.class);
> >> > job.setReducerClass(Reduce.class);
> >> > job.setOutputKeyClass(Text.class);
> >> > job.setOutputValueClass(IntWritable.class);
> >> > FileInputFormat.addInputPath(job, new Path("/mapin/"));
> >> > FileOutputFormat.setOutputPath(job, new Path("/mapout/"));
> >> > System.exit(job.waitForCompletion(true) ? 0 : 1);
> >> > }
> >> > }
> >> >
> >> > This is the error log -
> >> >
> >> > java.lang.RuntimeException: Error in configuring object
> >> > at
> >>
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> >> > at
> >> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> >> > at
> >>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> >> > at
> >> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
> >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> >> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >> > at java.security.AccessController.doPrivileged(Native Method)
> >> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >> > at
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> >> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> >> > Caused by: java.lang.reflect.InvocationTargetException
> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> > at
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> > at
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> > at java.lang.reflect.Method.invoke(Method.java:601)
> >> > at
> >>
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> >> > ... 9 more
> >> > Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> >> > java.lang.ClassNotFoundException: WordCount.Map
> >> > at
> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
> >> > at
> >> org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:947)
> >> > at
> >> org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> >> > ... 14 more
> >> > Caused by: java.lang.RuntimeException:
> >> > java.lang.ClassNotFoundException: WordCount.Map
> >> > at
> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
> >> > at
> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
> >> > ... 16 more
> >> > Caused by: java.lang.ClassNotFoundException: WordCount.Map
> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >> > at java.security.AccessController.doPrivileged(Native Method)
> >> > at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> >> > at
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> >> > at java.lang.Class.forName0(Native Method)
> >> > at java.lang.Class.forName(Class.java:264)
> >> > at
> >>
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
> >> > at
> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
> >> > ... 17 more
> >> >
> >> >
> >> > Here is the command which I am using to submit the workfloe -
> >> > bin/oozie job -oozie http://localhost:11000/oozie/ -config
> >> > ~/mr/job.properties -run
> >> >
> >> > Need some help. Many thanks.
> >> > (Please pardon my ignorance)
> >> >
> >> > Regards,
> >> > Mohammad Tariq
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
>
>
>
> --
> Harsh J
>
