I tweaked up a recipe (season to taste right?) and started a cluster with Whirr
0.6.0 (stable) but I can't seem to ssh in because I seem to have a fundamental
misunderstanding about where the keypair information ends up.
My configuration (below) starts up fine modulo a frequent NPE :
malformed image: null
java.lang.NullPointerException: architecture
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at org.jclouds.ec2.domain.Image.<init>(Image.java:81)
at
org.jclouds.ec2.xml.DescribeImagesResponseHandler.endElement(DescribeImagesResponseHandler.java:169)
at
com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:601)
at
com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1782)
etc...
I mosey on over to ~/.whirr/myapp and run hadoop-proxy.sh and from that point I
can make hadoop calls.
hadoop dfs -ls /
11/10/20 10:57:03 WARN conf.Configuration: DEPRECATED:
hadoop-site.xml found in the classpath.... [SNIP] ...
drwxrwxrwx - hdfs supergroup 0 2011-10-20 10:47 /hadoop
drwxrwxrwx - hdfs supergroup 0 2011-10-20 10:46 /hbase
drwxrwxrwx - hdfs supergroup 0 2011-10-20 10:46 /mnt
drwxrwxrwx - hdfs supergroup 0 2011-10-20 10:46 /tmp
drwxrwxrwx - hdfs supergroup 0 2011-10-20 10:46 /user
So all is well. But now I am confused. AWS wants me to use a pem that was
created, but I don't have that pem or any way to get it do I?
ssh -i jclouds#Linernodes#us-east-1#74.pem
[email protected]
I have a pem that I built for this work and would be happy to use it, but it's
not obvious to me how. Any thoughts or pointers to docs would be greatly
appreciated.
Thanks,
Rick
///////////////////////////////////////////////////////
Here is the config I used
whirr.cluster-name=myapp
whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,5
hadoop-datanode+hadoop-tasktracker
whirr.hadoop.install-function=install_cdh_hadoop
whirr.hadoop.configure-function=configure_cdh_hadoop
whirr.provider = aws-ec2
whirr.identity=${env:AWS_ACCESS_KEY_ID}
whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
# Should default to these
whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
# The size of the instance to use. See
http://aws.amazon.com/ec2/instance-types/
whirr.hardware-id=c1.xlarge
# Ubuntu 10.04 LTS Lucid. See http://alestic.com/
whirr.image-id=us-east-1/ami-da0cf8b3
# If you choose a different location, make sure whirr.image-id is
updated too
whirr.location-id=us-east-1
# Expert: specify the version of Hadoop to install.
whirr.hadoop.version=0.20.2
whirr.hadoop.tarball.url=http://archive.apache.org/dist/hadoop/core/hadoop-${whirr.hadoop.version}/hadoop-${whirr.hadoop.version}.tar.gz
