Hi,
I dont set job.setJarByClass(Map.class). But my main class, the map class
and
the reduce class are all in the same package. Does this make any difference at
or
do I still have to call
Thank You
Abhishek Agrawal
SUNY- Buffalo
(716-435-7122)
On Fri 11/27/09 1:42 PM , Aaron Kimball [email protected] sent:
> When you set up the Job object, do you call
> job.setJarByClass(Map.class)? That will tell Hadoop which jar file to
> ship with the job and to use for classloading in your code.
>
> - Aaron
> On Thu, Nov 26, 2009 at 11:56 PM, wrote:
> Hi,
> I am running the job from command line. The job runs fine in the
> local mode
> but something happens when I try to run the job in the distributed
> mode.
> Abhishek Agrawal
> SUNY- Buffalo
> (716-435-7122)
> On Fri 11/27/09 2:31 AM , Jeff Zhang sent:
> > Do you run the map reduce job in command line or IDE? in map
> reduce
> > mode, you should put the jar containing the map and reduce class
> in
> > your classpath
> > Jeff Zhang
> > On Fri, Nov 27, 2009 at 2:19 PM, wrote:
> > Hello Everybody,
> > I have a doubt in Haddop and was wondering
> if
> > anybody has faced a
> > similar problem. I have a package called test. Inside that I have
> > class called
> > A.java, Map.java, Reduce.java. In A.java I have the main method
> > where I am trying
> > to initialize the jobConf object. I have written
> > jobConf.setMapperClass(Map.class) and similarly for the reduce
> class
> > as well. The
> > code works correctly when I run the code locally via
> > jobConf.set("mapred.job.tracker","local") but I get an exception
> > when I try to
> > run this code on my cluster. The stack trace of the exception is
> as
> > under. I
> > cannot understand the problem. Any help would be appreciated.
> > java.lang.RuntimeException: java.lang.RuntimeException:
> > java.lang.ClassNotFoundException: test.Map
> > at
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:752)
> > at
> > org.apache.hadoop.mapred.JobConf.getMapperClass(JobConf.java:690)
> > at
> > org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
> > at
> >
> org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:58)
> > at
> >
> >
> org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:83)
> > at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:338)
> > at org.apache.hadoop.mapred.Child.main(Child.java:158)
> > Caused by: java.lang.RuntimeException:
> > java.lang.ClassNotFoundException:
> > Markowitz.covarMatrixMap
> > at
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:720)
> > at
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:744)
> > ... 6 more
> > Caused by: java.lang.ClassNotFoundException: test.Map
> > at
> java.net.URLClassLoader$1.run(URLClassLoader.java:200)
> > at java.security.AccessController.doPrivileged(Native
> > Method)
> > at
> > java.net.URLClassLoader.findClass(URLClassLoader.java:188)
> > at
> java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at
> > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:276)
> > at
> java.lang.ClassLoader.loadClass(ClassLoader.java:251)
> > at
> > java.lang.ClassLoader.loadClassInternal(ClassLoader.java:319)
> > at java.lang.Class.forName0(Native Method)
> > at java.lang.Class.forName(Class.java:247)
> > at
> >
> >
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:673)
> > at
> >
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:718)
> > ... 7 more
> > Thank You
> > Abhishek Agrawal
> > SUNY- Buffalo
> > (716-435-7122)
> >
> >
>
>
