Tim, thank you, these are important considerations, making one lean toward using prepared EC2 Hadoop distributions.
Mark On Mon, Nov 16, 2009 at 12:20 AM, Tim Robertson <[email protected]>wrote: > Hi Mark, > > I have never done it, but I *think* it would be reasonably easy. You > will have to manipulate a few scripts which you'd run each time you > start up a cluster, to copy around the addresses of the EC2 machines > (e.g. copy the master and slaves files etc). That would be a fairly > simple admin task each time you start. > You will also need to do the passphraseless ssh between the machines, > which you should be able to script quite easily also. > > Other than that, I think it would be just copying a directory into > your AMI and manipulating the config as per any hadoop install (the 3 > *-site.xml config files). > > There are nice scripts for EC2 with Hadoop, and Cloudera have done a > lot of investment here as well... might be worth considering how easy > it is to port your stuff to their AMI (HBase, Hive, Cascading etc all > configured as well), and also bear in mind how quickly Hadoop is > evolving so there will be ongoing maintenance. > > Cheers, > Tim > > > > > On Mon, Nov 16, 2009 at 6:34 AM, Jeff Zhang <[email protected]> wrote: > > You can use the distribution of cloudera, it's very easy to install > hadoop > > on ec2. > > > > http://archive.cloudera.com/docs/ec2.html > > > > > > Jeff Zhang > > > > > > On Mon, Nov 16, 2009 at 1:24 PM, Mark Kerzner <[email protected]> > wrote: > > > >> Hi, > >> > >> ideally I would like to take the EC2 image that suits my purpose, then > add > >> Hadoop to it, save this image and use it for my cluster. How hard would > it > >> be? What additional configuration do I need to do after I install hadoop > on > >> the machine? > >> > >> Thank you, > >> Mark > >> > > >
