Hello Everybody,
How can we handle different applications having
different requirement being run on the same hadoop cluster ? What are the
various approaches to solve such problem.. if possible please mention some
of those ideas.
Does such implementation exists ?
Thanks ,
Regards,
Ashish
On Thu, Jun 18, 2009 at 9:36 PM, jason hadoop <jason.had...@gmail.com>wrote:
> For me, I like to have one configuration file that I distribute to all of
> the machines in my cluster via rsync.
>
> In there are things like the number of tasks per node to run, and where to
> store dfs data and local temporary data, and the limits to storage for the
> machines.
>
> If the machines are very different, it becomes important to tailor the
> configuration file per machine or type of machine.
>
> At this point, you are pretty much going to have to spend the time, reading
> through the details of configuring a hadoop cluster.
>
>
> On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek <pareek...@gmail.com>
> wrote:
>
> > Can you tell few of the challenges in configuring heterogeneous
> > cluster...or
> > can pass on some link where I would get some information regarding
> > challenges in running Hadoop on heterogeneous hardware
> >
> > One more things is How about running different applications on the same
> > Hadoop cluster?and what challenges are involved in it ?
> >
> > Thanks,
> > Regards,
> > Ashish
> >
> >
> > On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <jason.had...@gmail.com
> > >wrote:
> >
> > > I don't know anyone who has a completely homogeneous cluster.
> > >
> > > So hadoop is scalable across heterogeneous environments.
> > >
> > > I stated that configuration is simpler if the machines are similar
> (There
> > > are optimizations in configuration for near homogeneous machines.)
> > >
> > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pareek...@gmail.com>
> > > wrote:
> > >
> > > > Does that mean hadoop is not scalable wrt heterogeneous environment?
> > and
> > > > one
> > > > more question is can we run different application on the same hadoop
> > > > cluster
> > > > .
> > > >
> > > > Thanks.
> > > > Regards,
> > > > Ashish
> > > >
> > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <
> jason.had...@gmail.com
> > > > >wrote:
> > > >
> > > > > Hadoop has always been reasonably agnostic wrt hardware and
> > > homogeneity.
> > > > > There are optimizations in configuration for near homogeneous
> > machines.
> > > > >
> > > > >
> > > > >
> > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <
> pareek...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Hello,
> > > > > > I am doing my master.... my final year project is on
> Hadoop
> > > > ...so
> > > > > I
> > > > > > would like to know some thing about Hadoop cluster i.e, Do new
> > > version
> > > > of
> > > > > > Hadoop are able to handle heterogeneous hardware.If you have any
> > > > > > informantion regarding these please mail me as my project is in
> > > > > > heterogenous
> > > > > > environment.
> > > > > >
> > > > > >
> > > > > > Thanks!
> > > > > >
> > > > > > Reagrds,
> > > > > > Ashish Pareek
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > > > www.prohadoopbook.com a community for Hadoop Professionals
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> > > http://www.amazon.com/dp/1430219424?tag=jewlerymall
> > > www.prohadoopbook.com a community for Hadoop Professionals
> > >
> >
>
>
>
> --
> Pro Hadoop, a book to guide you from beginner to hadoop mastery,
> http://www.amazon.com/dp/1430219424?tag=jewlerymall
> www.prohadoopbook.com a community for Hadoop Professionals
>
