For me, I like to have one configuration file that I distribute to all of the machines in my cluster via rsync.
In there are things like the number of tasks per node to run, and where to store dfs data and local temporary data, and the limits to storage for the machines. If the machines are very different, it becomes important to tailor the configuration file per machine or type of machine. At this point, you are pretty much going to have to spend the time, reading through the details of configuring a hadoop cluster. On Thu, Jun 18, 2009 at 8:33 AM, ashish pareek <pareek...@gmail.com> wrote: > Can you tell few of the challenges in configuring heterogeneous > cluster...or > can pass on some link where I would get some information regarding > challenges in running Hadoop on heterogeneous hardware > > One more things is How about running different applications on the same > Hadoop cluster?and what challenges are involved in it ? > > Thanks, > Regards, > Ashish > > > On Thu, Jun 18, 2009 at 8:53 PM, jason hadoop <jason.had...@gmail.com > >wrote: > > > I don't know anyone who has a completely homogeneous cluster. > > > > So hadoop is scalable across heterogeneous environments. > > > > I stated that configuration is simpler if the machines are similar (There > > are optimizations in configuration for near homogeneous machines.) > > > > On Thu, Jun 18, 2009 at 8:10 AM, ashish pareek <pareek...@gmail.com> > > wrote: > > > > > Does that mean hadoop is not scalable wrt heterogeneous environment? > and > > > one > > > more question is can we run different application on the same hadoop > > > cluster > > > . > > > > > > Thanks. > > > Regards, > > > Ashish > > > > > > On Thu, Jun 18, 2009 at 8:30 PM, jason hadoop <jason.had...@gmail.com > > > >wrote: > > > > > > > Hadoop has always been reasonably agnostic wrt hardware and > > homogeneity. > > > > There are optimizations in configuration for near homogeneous > machines. > > > > > > > > > > > > > > > > On Thu, Jun 18, 2009 at 7:46 AM, ashish pareek <pareek...@gmail.com> > > > > wrote: > > > > > > > > > Hello, > > > > > I am doing my master.... my final year project is on Hadoop > > > ...so > > > > I > > > > > would like to know some thing about Hadoop cluster i.e, Do new > > version > > > of > > > > > Hadoop are able to handle heterogeneous hardware.If you have any > > > > > informantion regarding these please mail me as my project is in > > > > > heterogenous > > > > > environment. > > > > > > > > > > > > > > > Thanks! > > > > > > > > > > Reagrds, > > > > > Ashish Pareek > > > > > > > > > > > > > > > > > > > > > -- > > > > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > > > > http://www.amazon.com/dp/1430219424?tag=jewlerymall > > > > www.prohadoopbook.com a community for Hadoop Professionals > > > > > > > > > > > > > > > -- > > Pro Hadoop, a book to guide you from beginner to hadoop mastery, > > http://www.amazon.com/dp/1430219424?tag=jewlerymall > > www.prohadoopbook.com a community for Hadoop Professionals > > > -- Pro Hadoop, a book to guide you from beginner to hadoop mastery, http://www.amazon.com/dp/1430219424?tag=jewlerymall www.prohadoopbook.com a community for Hadoop Professionals
