You should be able to run new API Map / Reducers - try setting the new-api
properties in your configuration set:
<property>
<name>mapred.mapper.new-api</name>
<value>true</value>
</property>
<property>
<name>mapred.reducer.new-api</name>
<value>true</value>
</property>
On Sat, Dec 8, 2012 at 4:11 PM, Mohammad Tariq <[email protected]> wrote:
> No luck. Same error
>
> I'll try to dig further. Thanks a lot for the help.
>
> Regards,
> Mohammad Tariq
>
>
>
> On Sun, Dec 9, 2012 at 2:12 AM, Mohammad Tariq <[email protected]> wrote:
>
> > I see. Will give it a shot and see if that works.
> >
> > Thank you.
> >
> > Regards,
> > Mohammad Tariq
> >
> >
> >
> > On Sun, Dec 9, 2012 at 2:10 AM, Harsh J <[email protected]> wrote:
> >
> >> The combo is OK, but the use of the mapreduce API may not be.
> >>
> >> I'd try removing inner classes and placing them as regular Map and
> >> Reduce classes and reconfiguring/resubmitting.
> >>
> >> On Sun, Dec 9, 2012 at 2:06 AM, Mohammad Tariq <[email protected]>
> >> wrote:
> >> > Thank you so much for the quick response Harsh. Please find answers to
> >> your
> >> > questions below(in order) :
> >> >
> >> > 1- Yes, it works.
> >> > 2- Yes, jar contains all the classes.
> >> > 3- I am trying to use oozie-3.2.0-incubating + hadoop-1.0.4. Is this
> >> > combination OK?
> >> >
> >> > Regards,
> >> > Mohammad Tariq
> >> >
> >> >
> >> >
> >> > On Sun, Dec 9, 2012 at 1:59 AM, Harsh J <[email protected]> wrote:
> >> >
> >> >> Hi,
> >> >>
> >> >> A couple of things:
> >> >>
> >> >> 1. Can you verify if running your job directly works?
> >> >> 2. Can you verify if your packaged jar does contain the classes?
> >> >> (using a command like jar -tvf <jar> should tell you).
> >> >> 2. The Oozie map-reduce action relies on the Stable API, not the New
> >> API.
> >> >>
> >> >> On Sun, Dec 9, 2012 at 1:10 AM, Mohammad Tariq <[email protected]>
> >> wrote:
> >> >> > Hello list,
> >> >> >
> >> >> > I have just started with Oozie and trying to follow the
> >> tutorial
> >> >> > at https://cwiki.apache.org/OOZIE/map-reduce-cookbook.html but I
> am
> >> >> facing
> >> >> > some issues as my "Mapper" class is not reachable(as per the JT
> >> logs).
> >> >> > Nothing complex, just tying to run the WordCount program. I have
> >> >> specified
> >> >> > the details below. in case anybody needs it :
> >> >> >
> >> >> > Location of the workflow components directory -
> >> >> > hdfs://localhost:9000/mr
> >> >> >
> >> >> > Contents inside /mr -
> >> >> > job.properties
> >> >> > workflow.xml
> >> >> > /lib
> >> >> > /lib/wc.jar
> >> >> >
> >> >> > Contents of job.properties -
> >> >> > nameNode=hdfs://localhost:9000
> >> >> > jobTracker=localhost:9001
> >> >> > queueName=default
> >> >> > examplesRoot=mr
> >> >> > oozie.wf.application.path=${nameNode}/${examplesRoot}
> >> >> > inputDir=/mapin
> >> >> > outputDir=/mapout
> >> >> >
> >> >> > Contents of workflow.xml -
> >> >> > <workflow-app name='wordcount-wf' xmlns="uri:oozie:workflow:0.2">
> >> >> > <start to='wordcount'/>
> >> >> > <action name='wordcount'>
> >> >> > <map-reduce>
> >> >> > <job-tracker>${jobTracker}</job-tracker>
> >> >> > <name-node>${nameNode}</name-node>
> >> >> > <prepare>
> >> >> > </prepare>
> >> >> > <configuration>
> >> >> > <property>
> >> >> > <name>mapred.job.queue.name</name>
> >> >> > <value>${queueName}</value>
> >> >> > </property>
> >> >> > <property>
> >> >> > <name>mapred.mapper.class</name>
> >> >> > <value>WordCount.Map</value>
> >> >> > </property>
> >> >> > <property>
> >> >> > <name>mapred.reducer.class</name>
> >> >> > <value>WordCount.Reduce</value>
> >> >> > </property>
> >> >> > <property>
> >> >> > <name>mapred.input.dir</name>
> >> >> > <value>${inputDir}</value>
> >> >> > </property>
> >> >> > <property>
> >> >> > <name>mapred.output.dir</name>
> >> >> > <value>${outputDir}</value>
> >> >> > </property>
> >> >> > </configuration>
> >> >> > </map-reduce>
> >> >> > <ok to='end'/>
> >> >> > <error to='end'/>
> >> >> > </action>
> >> >> > <!--kill name='kill'>
> >> >> > <value>${wf:errorCode("wordcount")}</value>
> >> >> > </kill-->
> >> >> > <end name='end'/>
> >> >> > </workflow-app>
> >> >> >
> >> >> > Here is the program -
> >> >> > import java.io.IOException;
> >> >> > import java.lang.InterruptedException;
> >> >> > import java.util.StringTokenizer;
> >> >> >
> >> >> > import org.apache.hadoop.io.IntWritable;
> >> >> > import org.apache.hadoop.io.Text;
> >> >> > import org.apache.hadoop.conf.Configuration;
> >> >> > import org.apache.hadoop.fs.Path;
> >> >> > import org.apache.hadoop.mapreduce.Job;
> >> >> > import org.apache.hadoop.mapreduce.Mapper;
> >> >> > import org.apache.hadoop.mapreduce.Reducer;
> >> >> > import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
> >> >> > import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
> >> >> > import org.apache.hadoop.util.GenericOptionsParser;
> >> >> >
> >> >> > public class WordCount {
> >> >> > /**
> >> >> > * The map class of WordCount.
> >> >> > */
> >> >> > public static class Map extends Mapper<Object, Text, Text,
> >> IntWritable> {
> >> >> >
> >> >> > private final static IntWritable one = new IntWritable(1);
> >> >> > private Text word = new Text();
> >> >> >
> >> >> > public void map(Object key, Text value, Context context)
> >> >> > throws IOException, InterruptedException {
> >> >> > StringTokenizer itr = new
> StringTokenizer(value.toString());
> >> >> > while (itr.hasMoreTokens()) {
> >> >> > word.set(itr.nextToken());
> >> >> > context.write(word, one);
> >> >> > }
> >> >> > }
> >> >> > }
> >> >> > /**
> >> >> > * The reducer class of WordCount
> >> >> > */
> >> >> > public static class Reduce extends Reducer<Text, IntWritable, Text,
> >> >> > IntWritable> {
> >> >> > public void reduce(Text key, Iterable<IntWritable> values,
> >> Context
> >> >> > context)
> >> >> > throws IOException, InterruptedException {
> >> >> > int sum = 0;
> >> >> > for (IntWritable value : values) {
> >> >> > sum += value.get();
> >> >> > }
> >> >> > context.write(key, new IntWritable(sum));
> >> >> > }
> >> >> > }
> >> >> > /**
> >> >> > * The main entry point.
> >> >> > */
> >> >> > public static void main(String[] args) throws Exception {
> >> >> > Configuration conf = new Configuration();
> >> >> > Job job = new Job(conf, "WordCount");
> >> >> > job.setJarByClass(WordCount.class);
> >> >> > job.setMapperClass(Map.class);
> >> >> > job.setReducerClass(Reduce.class);
> >> >> > job.setOutputKeyClass(Text.class);
> >> >> > job.setOutputValueClass(IntWritable.class);
> >> >> > FileInputFormat.addInputPath(job, new Path("/mapin/"));
> >> >> > FileOutputFormat.setOutputPath(job, new Path("/mapout/"));
> >> >> > System.exit(job.waitForCompletion(true) ? 0 : 1);
> >> >> > }
> >> >> > }
> >> >> >
> >> >> > This is the error log -
> >> >> >
> >> >> > java.lang.RuntimeException: Error in configuring object
> >> >> > at
> >> >>
> >>
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
> >> >> > at
> >> >>
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
> >> >> > at
> >> >>
> >>
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
> >> >> > at
> >> >> org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
> >> >> > at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
> >> >> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >> >> > at java.security.AccessController.doPrivileged(Native
> Method)
> >> >> > at javax.security.auth.Subject.doAs(Subject.java:415)
> >> >> > at
> >> >>
> >>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> >> >> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> >> >> > Caused by: java.lang.reflect.InvocationTargetException
> >> >> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> Method)
> >> >> > at
> >> >>
> >>
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >> > at
> >> >>
> >>
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >> > at java.lang.reflect.Method.invoke(Method.java:601)
> >> >> > at
> >> >>
> >>
> org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
> >> >> > ... 9 more
> >> >> > Caused by: java.lang.RuntimeException: java.lang.RuntimeException:
> >> >> > java.lang.ClassNotFoundException: WordCount.Map
> >> >> > at
> >> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:899)
> >> >> > at
> >> >> org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:947)
> >> >> > at
> >> >> org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> >> >> > ... 14 more
> >> >> > Caused by: java.lang.RuntimeException:
> >> >> > java.lang.ClassNotFoundException: WordCount.Map
> >> >> > at
> >> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867)
> >> >> > at
> >> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:891)
> >> >> > ... 16 more
> >> >> > Caused by: java.lang.ClassNotFoundException: WordCount.Map
> >> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> >> >> > at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> >> >> > at java.security.AccessController.doPrivileged(Native
> Method)
> >> >> > at
> java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> >> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
> >> >> > at
> >> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> >> >> > at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
> >> >> > at java.lang.Class.forName0(Native Method)
> >> >> > at java.lang.Class.forName(Class.java:264)
> >> >> > at
> >> >>
> >>
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
> >> >> > at
> >> >> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865)
> >> >> > ... 17 more
> >> >> >
> >> >> >
> >> >> > Here is the command which I am using to submit the workfloe -
> >> >> > bin/oozie job -oozie http://localhost:11000/oozie/ -config
> >> >> > ~/mr/job.properties -run
> >> >> >
> >> >> > Need some help. Many thanks.
> >> >> > (Please pardon my ignorance)
> >> >> >
> >> >> > Regards,
> >> >> > Mohammad Tariq
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Harsh J
> >> >>
> >>
> >>
> >>
> >> --
> >> Harsh J
> >>
> >
> >
>
