Thanks Aaron for the explanation.
On Tue, Apr 7, 2009 at 1:51 PM, Aaron Kimball <[email protected]> wrote:
> All the nodes in your Hadoop cluster need access to the class files for
> your
> MapReduce job. The current mechanism that Hadoop has to distribute classes
> and attach them to the classpath assumes they're in a JAR together. Thus,
> merely specifying the names of mapper/reducer classes with
> setMapperClass(),
> etc, isn't enough -- you need to actually deliver a jar containing those
> classes to all your nodes. Since the mapper and reducer classes are
> separate
> classes, you'd need to bundle those .class files together somehow. JAR is
> the standard way to do this, so that's what Hadoop supports.
>
> If you're running in fully-local mode (e.g., with
> jobConf.set("mapred.job.tracker", "local")), then no jar is needed since
> it's all running inside the original process space.
>
> - Aaron
>
>
> On Thu, Apr 2, 2009 at 1:13 PM, Farhan Husain <[email protected]> wrote:
>
> > I did all of them i.e. I used setMapClass, setReduceClass and new
> > JobConf(MapReduceWork.class) but still it cannot run the job without a
> jar
> > file. I understand the reason that it looks for those classes inside a
> jar
> > but I think there should be some better way to find those classes without
> > using a jar. But I am not sure whether it is possible at all.
> >
> > On Thu, Apr 2, 2009 at 2:56 PM, Rasit OZDAS <[email protected]>
> wrote:
> >
> > > You can point to them by using
> > > conf.setMapClass(..) and conf.setReduceClass(..) - or something
> > > similar, I don't have the source nearby.
> > >
> > > But something weird has happened to my code. It runs locally when I
> > > start it as java process (tries to find input path locally). I'm now
> > > using trunk, maybe something has changed with new version. With
> > > version 0.19 it was fine.
> > > Can somebody point out a clue?
> > >
> > > Rasit
> > >
> >
> >
> >
> > --
> > Mohammad Farhan Husain
> > Research Assistant
> > Department of Computer Science
> > Erik Jonsson School of Engineering and Computer Science
> > University of Texas at Dallas
> >
>
--
Mohammad Farhan Husain
Research Assistant
Department of Computer Science
Erik Jonsson School of Engineering and Computer Science
University of Texas at Dallas
