Hi Andrei, thanks for your quick reply. On 26 October 2011 10:16, Andrei Savu <[email protected]> wrote: > I don't have a good answer. Sometimes ssh connections are dropped but if the > cluster is able to start everything should be fine. 0.7.0 will ship with > jclouds 1.2.1 which has improvements around handling ssh connections.
Is there a Whirr SNAPSHOT/nightly build I can use? Do you think I would be better of using a SNAPSHOT instead of the latest stable release? > Another thing we've noticed while testing is that sometimes AMIs change > without notice and things start to break. Which AMIs do you use to test Whirr? ;-) I'll use the same... if available in the eu-west-1 region. Thanks, Paolo > > I suggest that you create some sort of smoke test for the cluster and if it > fails you can rebuild it from scratch. > Also take a look at the configuration guide page because there are some > parameters you can tweak: > http://whirr.apache.org/docs/0.6.0/configuration-guide.html > (e.g. probably a cluster with only 80% of the region servers is still good > enough) > Cheers, > -- Andrei Savu / andreisavu.ro > > On Wed, Oct 26, 2011 at 11:56 AM, Paolo Castagna > <[email protected]> wrote: >> >> Hi, >> I am using Whirr version 0.6.0-incubating. >> >> This is my recipe for a small Hadoop cluster: >> >> ----- >> whirr.cluster-name=hadoop >> whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker, 6 >> hadoop-datanode+hadoop-tasktracker >> whirr.provider=aws-ec2 >> whirr.identity=${env:AWS_ACCESS_KEY_ID} >> whirr.credential=${env:AWS_SECRET_ACCESS_KEY} >> whirr.hardware-id=m1.large >> whirr.image-id=eu-west-1/ami-ee0e3c9a >> whirr.location-id=eu-west-1 >> whirr.private-key-file=${sys:user.home}/.ssh/whirr >> whirr.public-key-file=${whirr.private-key-file}.pub >> whirr.hadoop.version=0.20.204.0 >> >> whirr.hadoop.tarball.url=http://archive.apache.org/dist/hadoop/core/hadoop-${whirr.hadoop.version}/hadoop-${whirr.hadoop.version}.tar.gz >> ----- >> >> When I start up the cluster everything seems fine, but I see this >> message very early on: >> >> ---- >> Dying because - net.schmizz.sshj.transport.TransportException: Broken >> transport; encountered EOF >> Dying because - net.schmizz.sshj.transport.TransportException: Broken >> transport; encountered EOF >> <<kex done>> woke to: net.schmizz.sshj.transport.TransportException: >> Broken transport; encountered EOF >> << ([email protected]:22) error acquiring >> SSHClient([email protected]:22): Broken transport; encountered EOF >> net.schmizz.sshj.transport.TransportException: Broken transport; >> encountered EOF >> at net.schmizz.sshj.transport.Reader.run(Reader.java:70) >> ---- >> >> I do not understand the problem (or if indeed, this is a problem). >> >> ... and I am missing compression. Could that be the reason? :-( >> >> Which AMI would you recommend for Amazon EC2 eu-west-1? >> >> Thank you very much in advance for your help (and thanks for Whirr! ;-)) >> Paolo > >
