Hi Mark, I have never done it, but I *think* it would be reasonably easy. You will have to manipulate a few scripts which you'd run each time you start up a cluster, to copy around the addresses of the EC2 machines (e.g. copy the master and slaves files etc). That would be a fairly simple admin task each time you start. You will also need to do the passphraseless ssh between the machines, which you should be able to script quite easily also.
Other than that, I think it would be just copying a directory into your AMI and manipulating the config as per any hadoop install (the 3 *-site.xml config files). There are nice scripts for EC2 with Hadoop, and Cloudera have done a lot of investment here as well... might be worth considering how easy it is to port your stuff to their AMI (HBase, Hive, Cascading etc all configured as well), and also bear in mind how quickly Hadoop is evolving so there will be ongoing maintenance. Cheers, Tim On Mon, Nov 16, 2009 at 6:34 AM, Jeff Zhang <[email protected]> wrote: > You can use the distribution of cloudera, it's very easy to install hadoop > on ec2. > > http://archive.cloudera.com/docs/ec2.html > > > Jeff Zhang > > > On Mon, Nov 16, 2009 at 1:24 PM, Mark Kerzner <[email protected]> wrote: > >> Hi, >> >> ideally I would like to take the EC2 image that suits my purpose, then add >> Hadoop to it, save this image and use it for my cluster. How hard would it >> be? What additional configuration do I need to do after I install hadoop on >> the machine? >> >> Thank you, >> Mark >> >
