Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification.
The following page has been changed by TomWhite: http://wiki.apache.org/lucene-hadoop/AmazonEC2 ------------------------------------------------------------------------------ == Setting up == * Make sure you've followed the Amazon EC2 [http://docs.amazonwebservices.com/AmazonEC2/gsg/2006-10-01/ Getting Started Guide], sections "Setting up an Account", "Setting up the Tools" and the "Generating a Keypair" section of "Running an Instance". - * Unpack the Hadoop EC2 scripts distribution available from [http://issues.apache.org/jira/browse/HADOOP-884 HADOOP-884]. + * Get the scripts by applying the patch in [http://issues.apache.org/jira/browse/HADOOP-884 HADOOP-884] to Hadoop trunk. + * Open a command prompt in ''src/contrib/ec2''. - * Edit all relevant variables in `hadoop-ec2-env.sh`. + * Edit all relevant variables in ''bin/hadoop-ec2-env.sh''. * You need to get a Java download URL by visiting [http://java.sun.com/javase/downloads/index_jdk5.jsp here]. Make sure you get the JDK (not JRE) labelled "Linux self-extracting file". (The scripts have not been tested with Java 6 yet.) == Creating an image == - You only need to do this once. + You should only need to do this once for each version of Hadoop you want to create an image for. You can use the same image to run different job. (If you need to repeat this for the same Hadoop version number then you will need to run `ec2-deregister` to de-register the existing AMI.) * Type {{{ - ./create-hadoop-image + bin/create-hadoop-image }}} * Accept the Java license terms. - * The script will create a new image, then bundle, upload and register it. This may take some time. Be patient - don't assume it's crashed. + * The script will create a new image, then bundle, upload and register it. This may take some time (20 minutes or more). Be patient - don't assume it's crashed! - * Terminate you instance using the command given by the script. + * Terminate your instance using the command given by the script. == Running a job on a cluster == - * Run cluster. During execution of this script you will be prompted to set up DNS (e.g. [http://www.dyndns.com/services/dns/dyndns/ DynDNS]). {{{ + * During execution of this script you will be prompted to set up DNS (e.g. [http://www.dyndns.com/services/dns/dyndns/ DynDNS]). {{{ - ./run-hadoop-cluster + bin/run-hadoop-cluster }}} * You will then be logged into the master node where you can start your job. * For example, to test your cluster, try {{{ cd /usr/local/hadoop-* bin/hadoop jar hadoop-*-examples.jar pi 10 10000000 }}} - * You can check progress of your job at `http://MASTER_HOST:50030/`. + * You can check progress of your job at `http://<MASTER_HOST>:50030/`. * When you have finished logout of the master node by typing `exit`, then shutdown the cluster with {{{ - ./terminate-hadoop-cluster + bin/terminate-hadoop-cluster }}}
