Latest version. I downloaded this last night. 0.7.1
Subsequent runs are even worse. Here is the output from a subsequent run
(with reduced nodes), No clear reason why it fails, just that my cluster
starts and then destroys!!
[root@ip-10-118-190-121 whirr-0.7.1]# bin/whirr launch-cluster
--cluster-name MyCluster1 --config hadoop.properties
Bootstrapping cluster
Configuring template
Configuring template
Starting 1 node(s) with roles [hadoop-jobtracker, hadoop-namenode]
Starting 2 node(s) with roles [hadoop-datanode, hadoop-tasktracker]
Starting 2 node(s) with roles [hadoop-datanode, hadoop-tasktracker]
Nodes started: [[id=us-east-1/i-3912c15f, providerId=i-3912c15f,
group=MyCluster1, name=MyCluster1-3912c15f, location=[id=us-east-1a,
scope=ZONE, description=us-east-1a, parent=us-east-1, iso3166Codes=[US-VA],
metadata={}], uri=null, imageId=us-east-1/ami-ab36fbc2, os=[name=null,
family=ubuntu, version=10.04, arch=paravirtual, is64Bit=false,
description=099720109477/ebs/ubuntu-images/ubuntu-lucid-10.04-i386-server-20110930],
state=RUNNING, loginPort=22, hostname=ip-10-190-50-29,
privateAddresses=[10.190.50.29], publicAddresses=[23.20.11.228],
hardware=[id=m1.small, providerId=m1.small, name=null,
processors=[[cores=1.0, speed=1.0]], ram=1740, volumes=[[id=null,
type=LOCAL, size=150.0, device=/dev/sda2, durable=false,
isBootDevice=false], [id=vol-4ccdeb23, type=SAN, size=null,
device=/dev/sda1, durable=true, isBootDevice=true]],
supportsImage=And(ALWAYS_TRUE,Or(isWindows(),requiresVirtualizationType(paravirtual)),ALWAYS_TRUE,Not(is64Bit())),
tags=[]], loginUser=ubuntu, userMetadata={Name=MyCluster1-3912c15f},
tags=[]]]
Destroying failed nodes [us-east-1/i-3d10c35b, us-east-1/i-0712c161,
us-east-1/i-0512c163]
Destroyed failed nodes [us-east-1/i-3d10c35b, us-east-1/i-0712c161,
us-east-1/i-0512c163]
Unable to start the cluster. Terminating all nodes.
java.io.IOException: java.util.concurrent.ExecutionException:
java.io.IOException: Too many instance failed while bootstrapping! 1
successfully started instances while 3 instances failed
at
org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:129)
at
org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107)
at
org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106)
at
org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
at org.apache.whirr.cli.Main.run(Main.java:64)
at org.apache.whirr.cli.Main.main(Main.java:97)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException:
Too many instance failed while bootstrapping! 1 successfully started
instances while 3 instances failed
at
java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:124)
... 5 more
Caused by: java.io.IOException: Too many instance failed while
bootstrapping! 1 successfully started instances while 3 instances failed
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:92)
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:40)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Unable to load cluster state, assuming it has no running nodes.
java.io.FileNotFoundException: /root/.whirr/MyCluster1/instances (No such
file or directory)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:120)
at com.google.common.io.Files$1.getInput(Files.java:100)
at com.google.common.io.Files$1.getInput(Files.java:97)
at com.google.common.io.CharStreams$2.getInput(CharStreams.java:91)
at com.google.common.io.CharStreams$2.getInput(CharStreams.java:88)
at com.google.common.io.CharStreams.readLines(CharStreams.java:306)
at com.google.common.io.Files.readLines(Files.java:580)
at
org.apache.whirr.state.FileClusterStateStore.load(FileClusterStateStore.java:54)
at
org.apache.whirr.state.ClusterStateStore.tryLoadOrEmpty(ClusterStateStore.java:58)
at
org.apache.whirr.ClusterController.destroyCluster(ClusterController.java:143)
at
org.apache.whirr.ClusterController.launchCluster(ClusterController.java:118)
at
org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
at org.apache.whirr.cli.Main.run(Main.java:64)
at org.apache.whirr.cli.Main.main(Main.java:97)
Starting to run scripts on cluster for phase destroyinstances:
Starting to run scripts on cluster for phase destroyinstances:
Finished running destroy phase scripts on all cluster instances
Destroying MyCluster1 cluster
Cluster MyCluster1 destroyed
Exception in thread "main" java.lang.RuntimeException: java.io.IOException:
java.util.concurrent.ExecutionException: java.io.IOException: Too many
instance failed while bootstrapping! 1 successfully started instances while
3 instances failed
at
org.apache.whirr.ClusterController.launchCluster(ClusterController.java:125)
at
org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:63)
at org.apache.whirr.cli.Main.run(Main.java:64)
at org.apache.whirr.cli.Main.main(Main.java:97)
Caused by: java.io.IOException: java.util.concurrent.ExecutionException:
java.io.IOException: Too many instance failed while bootstrapping! 1
successfully started instances while 3 instances failed
at
org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:129)
at
org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:107)
at
org.apache.whirr.ClusterController.launchCluster(ClusterController.java:106)
... 3 more
Caused by: java.util.concurrent.ExecutionException: java.io.IOException:
Too many instance failed while bootstrapping! 1 successfully started
instances while 3 instances failed
at
java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
at java.util.concurrent.FutureTask.get(FutureTask.java:83)
at
org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:124)
... 5 more
Caused by: java.io.IOException: Too many instance failed while
bootstrapping! 1 successfully started instances while 3 instances failed
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:92)
at
org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:40)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
On Tue, May 8, 2012 at 11:05 AM, Andrei Savu <[email protected]> wrote:
> What version of Whirr are you using? Are you seeing this error every time
> you start a cluster?
>
> You can get the latest release from:
> https://www.apache.org/dyn/closer.cgi/whirr/
>
> Cheers,
>
> -- Andrei Savu / axemblr.com / Tools for Clouds
>
>
> On Tue, May 8, 2012 at 5:06 PM, Light Reader <[email protected]>wrote:
>
>> Hi,
>> Newbie alert.
>>
>> I am trying to launch a 5 node cluster with Whirr. I get the following
>> message during a launch:
>>
>> Unexpected error while starting 4 nodes, minimum 4 nodes for
>> [hadoop-datanode, hadoop-tasktracker] of cluster HCluster1
>> *java.util.concurrent.ExecutionException:
>> java.lang.IllegalArgumentException: no private key configured for:
>> [region=us-east-1, name=jclouds#HCluster1#us-east-1#26]; please use
>> options.overrideLoginCredentialWith(rsa_private_text)*
>> at
>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>> at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>> at
>> org.apache.whirr.compute.StartupProcess.waitForOutcomes(StartupProcess.java:129)
>> at
>> org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:82)
>> at
>> org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:40)
>> at
>> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:662)
>>
>> While the cluster eventually does startup I am unable to do a hadoop fs
>> -ls. Before I debug that further wondering if the error message above is
>> harmless or maybe the root cause of my some of my issues.
>>
>> My hadoop.properties: (I did create the two env variables and also did
>> the rsa key gen)
>>
>> whirr.cluster-name=HCluster
>> whirr.instance-templates=1 hadoop-jobtracker+hadoop-namenode,4
>> hadoop-datanode+hadoop-tasktracker
>> whirr.provider=aws-ec2
>> whirr.identity=${env:AWS_ACCESS_KEY_ID}
>> whirr.credential=${env:AWS_SECRET_ACCESS_KEY}
>> whirr.private-key-file=${sys:user.home}/.ssh/id_rsa
>> whirr.public-key-file=${sys:user.home}/.ssh/id_rsa.pub
>> whirr.cluster-user=huser
>>
>> Any help is appreciated.
>>
>> Thx,
>> LR
>>
>
>
