On 26 October 2011 10:26, Andrei Savu <[email protected]> wrote: > > On Wed, Oct 26, 2011 at 12:21 PM, Paolo Castagna > <[email protected]> wrote: >> >> Hi Andrei, >> thanks for your quick reply. >> >> On 26 October 2011 10:16, Andrei Savu <[email protected]> wrote: >> > I don't have a good answer. Sometimes ssh connections are dropped but if >> > the >> > cluster is able to start everything should be fine. 0.7.0 will ship with >> > jclouds 1.2.1 which has improvements around handling ssh connections. >> >> Is there a Whirr SNAPSHOT/nightly build I can use? > > No. We only make the releases available as binary artefacts. > >> >> Do you think I would be better of using a SNAPSHOT instead of the >> latest stable release? > > Sometimes. Even if we are trying to be as careful as possible the trunk can > be unstable. > >> >> > Another thing we've noticed while testing is that sometimes AMIs change >> > without notice and things start to break. >> >> Which AMIs do you use to test Whirr? ;-) > > We use jclouds to automatically find an AMI running Ubuntu 10.04. This > mechanism is cool because it works in any region and on any cloud and can > also selecte 32bit or 64bit OSes as needed.
How do I do that? Should I remove whirr.image-id=eu-west-1/ami-ee0e3c9a from my recipe? Paolo > >> >> I'll use the same... if available in the eu-west-1 region. >> >> Thanks, >> Paolo >> >> > >> > I suggest that you create some sort of smoke test for the cluster and if >> > it >> > fails you can rebuild it from scratch. >> > Also take a look at the configuration guide page because there are some >> > parameters you can tweak: >> > http://whirr.apache.org/docs/0.6.0/configuration-guide.html >> > (e.g. probably a cluster with only 80% of the region servers is still >> > good >> > enough) >> > Cheers, >> > -- Andrei Savu / andreisavu.ro >> > >> > On Wed, Oct 26, 2011 at 11:56 AM, Paolo Castagna >> > <[email protected]> wrote: >> >> >> >> Hi, >> >> I am using Whirr version 0.6.0-incubating. >> >> >> >> This is my recipe for a small Hadoop cluster: >> >> >> >> ----- >> >> whirr.cluster-name=hadoop >> >> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 6 >> >> hadoop-datanode+hadoop-tasktracker >> >> whirr.provider=aws-ec2 >> >> whirr.identity=${env:AWS_ACCESS_KEY_ID} >> >> whirr.credential=${env:AWS_SECRET_ACCESS_KEY} >> >> whirr.hardware-id=m1.large >> >> whirr.image-id=eu-west-1/ami-ee0e3c9a >> >> whirr.location-id=eu-west-1 >> >> whirr.private-key-file=${sys:user.home}/.ssh/whirr >> >> whirr.public-key-file=${whirr.private-key-file}.pub >> >> whirr.hadoop.version=0.20.204.0 >> >> >> >> >> >> whirr.hadoop.tarball.url=http://archive.apache.org/dist/hadoop/core/hadoop-${whirr.hadoop.version}/hadoop-${whirr.hadoop.version}.tar.gz >> >> ----- >> >> >> >> When I start up the cluster everything seems fine, but I see this >> >> message very early on: >> >> >> >> ---- >> >> Dying because - net.schmizz.sshj.transport.TransportException: Broken >> >> transport; encountered EOF >> >> Dying because - net.schmizz.sshj.transport.TransportException: Broken >> >> transport; encountered EOF >> >> <<kex done>> woke to: net.schmizz.sshj.transport.TransportException: >> >> Broken transport; encountered EOF >> >> << ([email protected]:22) error acquiring >> >> SSHClient([email protected]:22): Broken transport; encountered EOF >> >> net.schmizz.sshj.transport.TransportException: Broken transport; >> >> encountered EOF >> >> at net.schmizz.sshj.transport.Reader.run(Reader.java:70) >> >> ---- >> >> >> >> I do not understand the problem (or if indeed, this is a problem). >> >> >> >> ... and I am missing compression. Could that be the reason? :-( >> >> >> >> Which AMI would you recommend for Amazon EC2 eu-west-1? >> >> >> >> Thank you very much in advance for your help (and thanks for Whirr! >> >> ;-)) >> >> Paolo >> > >> > > >
