i sent the whirr.log - it has an exception.
________________________________ From: Andrew Bayer <andrew.ba...@gmail.com> To: a b <autohan...@yahoo.com> Cc: "user@whirr.apache.org" <user@whirr.apache.org> Sent: Tuesday, August 6, 2013 10:51 AM Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds You can send it to me directly if you want. On the openjdk thing - I just don't get why openjdk-6-jdk isn't going to install its dependencies. That's normal apt-get behavior. A. On Tue, Aug 6, 2013 at 10:42 AM, a b <autohan...@yahoo.com> wrote: no - but i can recreate it - it is probably too large to post - will it be obvious what to extract? > > > > > > >________________________________ > From: Andrew Bayer <andrew.ba...@gmail.com> >To: user@whirr.apache.org; a b <autohan...@yahoo.com> >Sent: Tuesday, August 6, 2013 10:35 AM > >Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds > > > >Do you have the whirr.log from that attempt? > > >A. > > >On Mon, Aug 5, 2013 at 8:34 PM, a b <autohan...@yahoo.com> wrote: > >i built whirr in my own directory - i didn't change it (yet) - i just check it >out and tried to compile - you can see i had some memory issues that i didn't >notice right away: >> >> >> >> 1530 git clone git://git.apache.org/whirr.git >> 1531 mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs >> 1532 cd whirr/ >> 1533 mvn eclipse:eclipse -DdownloadSources=true -DdownloadJavadocs >> 1534 mvn install >> 1535 cd >> 1536 rm whirr.log >> 1537 ~/git/whirr/bin/whirr launch-cluster --config ~/whirr/recipes/hadoop.properties >> 1538 export MAVEN_OPTS=-Xmx200m >> 1539 cd ~/git/whirr/ >> 1540 mvn install >> 1541 export MAVEN_OPTS=-Xmx1G >> 1542 mvn install >> 1543 cd >> 1544 ~/git/whirr/bin/whirr launch-cluster --config >>~/whirr/recipes/hadoop.properties >> 1545 history >> >> >> >>this is the console at the end of "mvn install" >> >> >>[INFO] >>------------------------------------------------------------------------ >>[INFO] Reactor Summary: >>[INFO] >>[INFO] Apache Whirr Build Tools .......................... SUCCESS [2.519s] >>[INFO] Whirr ............................................. SUCCESS [4.662s] >>[INFO] Apache Whirr Core ................................. SUCCESS [1:43.913s] >>[INFO] Apache Whirr Cassandra ............................ SUCCESS [10.705s] >>[INFO] Apache Whirr Ganglia .............................. SUCCESS [3.683s] >>[INFO] Apache Whirr Hadoop ............................... SUCCESS [11.294s] >>[INFO] Apache Whirr ZooKeeper ............................ SUCCESS [4.477s] >>[INFO] Apache Whirr HBase ................................ SUCCESS [5.196s] >>[INFO] Apache Whirr YARN ................................. SUCCESS [4.867s] >>[INFO] Apache Whirr CDH .................................. SUCCESS [5.552s] >>[INFO] Apache Whirr Mahout ............................... SUCCESS [2.452s] >>[INFO] Apache Whirr Pig .................................. SUCCESS [2.393s] >>[INFO] Apache Whirr ElasticSearch ........................ SUCCESS [4.145s] >>[INFO] Apache Whirr Hama ................................. SUCCESS [2.841s] >>[INFO] Apache Whirr Puppet ............................... SUCCESS [3.242s] >>[INFO] Apache Whirr Chef ................................. SUCCESS [12.153s] >>[INFO] Apache Whirr Solr ................................. SUCCESS [4.997s] >>[INFO] Apache Whirr Kerberos ............................. SUCCESS [12.207s] >>[INFO] Apache Whirr CLI .................................. SUCCESS [14.940s] >>[INFO] Apache Whirr Examples ............................. SUCCESS [4.494s] >>[INFO] >>------------------------------------------------------------------------ >>[INFO] BUILD SUCCESS >>[INFO] >>------------------------------------------------------------------------ >>[INFO] Total time: 3:43.355s >>[INFO] Finished at: Mon Aug 05 20:11:58 PDT 2013 >>[INFO] Final Memory: 109M/262M >>[INFO] >>------------------------------------------------------------------------ >>ab@ubuntu12-64:~/git/whirr$ cd >>ab@ubuntu12-64:~$ ~/git/whirr/bin/whirr launch-cluster --config >>~/whirr/recipes/hadoop.properties >>Running on provider aws-ec2 using identity XXXXXXXXXXXXXXXXXXX >>Exception in thread "main" java.lang.RuntimeException: java.io.IOException: >>java.util.concurrent.ExecutionException: java.io.IOException: Too many >>instance failed while bootstrapping! 0 successfully started instances while 0 >>instances failed >> at org.apache.whirr.ClusterController.launchCluster(ClusterController.java:128) >> at >>org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:69) >> at >>org.apache.whirr.cli.command.LaunchClusterCommand.run(LaunchClusterCommand.java:59) >> at org.apache.whirr.cli.Main.run(Main.java:69) >> at org.apache.whirr.cli.Main.main(Main.java:102) >>Caused by: java.io.IOException: java.util.concurrent.ExecutionException: >>java.io.IOException: Too many instance failed while bootstrapping! 0 >>successfully started instances while 0 instances failed >> at >>org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:125) >> at >>org.apache.whirr.actions.ScriptBasedClusterAction.execute(ScriptBasedClusterAction.java:131) >> at org.apache.whirr.ClusterController.bootstrapCluster(ClusterController.java:137) >> at >>org.apache.whirr.ClusterController.launchCluster(ClusterController.java:113) >> ... 4 more >>Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Too >>many instance failed while bootstrapping! 0 successfully started instances >>while 0 instances failed >> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252) >> at java.util.concurrent.FutureTask.get(FutureTask.java:111) >> at >>org.apache.whirr.actions.BootstrapClusterAction.doAction(BootstrapClusterAction.java:120) >> ... 7 more >>Caused by: java.io.IOException: Too many instance failed while bootstrapping! >>0 successfully started instances while 0 instances failed >> at org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:93) >> at org.apache.whirr.compute.StartupProcess.call(StartupProcess.java:41) >> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) >> at java.util.concurrent.FutureTask.run(FutureTask.java:166) >> at >>java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) >> at >>java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) >> at java.lang.Thread.run(Thread.java:724) >> >> >> >>2 instances were started - like i asked for - the 1st instance was terminated >>almost immediately, the 2nd was left running for a while - eventually, i hit >>^c on the launch and terminated the 2nd instance from the aws console. >> >> >>i need some more coaching. >> >> >> >> >>________________________________ >> From: Andrew Bayer <andrew.ba...@gmail.com> >>To: user@whirr.apache.org; a b <autohan...@yahoo.com> >>Sent: Monday, August 5, 2013 5:54 PM >> >>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds >> >> >> >>Try not building as root - that can throw things off. >> >> >>A. >> >> >>On Mon, Aug 5, 2013 at 5:44 PM, a b <autohan...@yahoo.com> wrote: >> >>as for the java 7 problem - i found this suggestion: >>> >>>http://stackoverflow.com/questions/13225149/how-to-install-jdk-7-on-ec2-cluster-via-whirr >>> >>> >>>i tried to download whirr - as suggested here: >>> >>> >>>https://cwiki.apache.org/confluence/display/WHIRR/How+To+Contribute >>> >>> >>>o i did: git clone ... >>> >>>o i modified: core/src/main/resources/functions/... >>>o i did: mvn eclipse:eclipse ... >>>o i skipped: eclipse import >>>o i ran: mvn install >>> >>> >>>it fails in the "mvn install" during test >>> >>> >>>Tests in error: >>> testActionIsExecutedOnAllRelevantNodes(org.apache.whirr.actions.StopServicesActionTest): cluster-user != root or do not run as root >>> >>>testFilterScriptExecutionByRole(org.apache.whirr.actions.StopServicesActionTest): >>> cluster-user != root or do not run as root >>> >>>testFilterScriptExecutionByInstanceId(org.apache.whirr.actions.StopServicesActionTest): >>> cluster-user != root or do not run as root >>> >>>testFilterScriptExecutionByRoleAndInstanceId(org.apache.whirr.actions.StopServicesActionTest): >>> cluster-user != root or do not run as root >>> >>>testNoScriptExecutionsForNoop(org.apache.whirr.actions.StopServicesActionTest): >>> cluster-user != root or do not run as root >>>[..] >>> >>>testExecuteOnlyBootstrapForNoop(org.apache.whirr.service.DryRunModuleTest): >>>cluster-user != root or do not run as root >>> >>>testExecuteOnlyBootstrapForNoopWithListener(org.apache.whirr.service.DryRunModuleTest): >>> cluster-user != root or do not run as root >>> >>>testNoInitScriptsAfterConfigurationStartedAndNoConfigScriptsAfterDestroy(org.apache.whirr.service.DryRunModuleTest): >>> cluster-user != root or do not run as root >>> testOverrides(org.apache.whirr.command.AbstractClusterCommandTest): >>>cluster-user != root or do not run as root >>> >>>Tests run: 99, Failures: 0, Errors: 49, Skipped: 0 >>> >>>[INFO] >>>------------------------------------------------------------------------ >>>[INFO] Reactor Summary: >>>[INFO] >>>[INFO] Apache Whirr Build Tools .......................... SUCCESS [2.612s] >>>[INFO] Whirr ............................................. SUCCESS [4.488s] >>>[INFO] Apache Whirr Core ................................. FAILURE [17.113s] >>>[INFO] Apache Whirr Cassandra ............................ SKIPPED >>>[INFO] Apache Whirr Ganglia .............................. SKIPPED >>>[INFO] Apache Whirr Hadoop ............................... SKIPPED >>> >>> >>> >>>is there a better way to try this suggestion? >>> >>> >>> >>> >>>________________________________ >>> From: Andrew Bayer <andrew.ba...@gmail.com> >>> >>>To: a b <autohan...@yahoo.com> >>>Cc: "user@whirr.apache.org" <user@whirr.apache.org> >>>Sent: Monday, August 5, 2013 4:48 PM >>> >>>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds >>> >>> >>> >>>Looks like it's the Oracle JDK7 download that's failing - >>>http://download.oracle.com/otn-pub/java/jdk/7/jdk-7-linux-x64.tar.gz isn't >>>actually there. I don't know if they actually have a consistent link to get >>>the JDK7 tarball regardless of version. I'd just use OpenJDK for now, if you >>>can. >>> >>> >>>A. >>> >>> >>>On Mon, Aug 5, 2013 at 4:33 PM, a b <autohan...@yahoo.com> wrote: >>> >>>Get:19 http://us-east-1.ec2.archive.ubuntu.com precise/main amd64 Packages >>>[1,273 kB]^M >>>>Get:20 http://us-east-1.ec2.archive.ubuntu.com precise/universe amd64 >>>>Packages [4,786 kB]^M >>>>Get:21 http://us-east-1.ec2.archive.ubuntu.com precise/main i386 Packages >>>>[1,274 kB]^M >>>>Get:22 http://us-east-1.ec2.archive.ubuntu.com precise/universe i386 >>>>Packages [4,796 kB]^M >>>>Get:23 http://us-east-1.ec2.archive.ubuntu.com precise/main >>>>TranslationIndex [3,706 B]^M >>>>Get:24 http://us-east-1.ec2.archive.ubuntu.com precise/universe >>>>TranslationIndex [2,922 B]^M >>>>Get:25 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main Sources >>>>[412 kB]^M >>>>Get:26 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe >>>>Sources [93.1 kB]^M >>>>Get:27 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main amd64 >>>>Packages [672 kB]^M >>>>Get:28 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe >>>>amd64 Packages [210 kB]^M >>>>Get:29 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main i386 >>>>Packages [692 kB]^M >>>>Get:30 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe >>>>i386 Packages [214 kB]^M >>>>Get:31 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main >>>>TranslationIndex [3,564 B]^M >>>>Get:32 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe >>>>TranslationIndex [2,850 B]^M >>>>Get:33 http://us-east-1.ec2.archive.ubuntu.com precise/main Translation-en >>>>[726 kB]^M >>>>Get:34 http://us-east-1.ec2.archive.ubuntu.com precise/universe >>>>Translation-en [3,341 kB]^M >>>>Get:35 http://us-east-1.ec2.archive.ubuntu.com precise-updates/main >>>>Translation-en [298 kB]^M >>>>Get:36 http://us-east-1.ec2.archive.ubuntu.com precise-updates/universe >>>>Translation-en [123 kB]^M >>>>Fetched 26.1 MB in 21s (1,241 kB/s)^M >>>>Reading package lists...^M >>>>Could not download >>>>http://apache.osuosl.org/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz.md5. >>>> Continuing.^M >>>>, error=^M >>>>gzip: stdin: not in gzip format^M >>>>tar: Child returned status 1^M >>>>tar: Error is not recoverable: exiting now^M >>>>mv: cannot stat `jdk1*': No such file or directory^M >>>>update-alternatives: error: alternative path >>>>/usr/lib/jvm/java-7-oracle/bin/java doesn't exist.^M >>>> >>>> >>>> >>>> >>>>________________________________ >>>> From: Andrew Bayer <andrew.ba...@gmail.com> >>>>To: user@whirr.apache.org; a b <autohan...@yahoo.com> >>>>Sent: Monday, August 5, 2013 4:22 PM >>>> >>>>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds >>>> >>>> >>>> >>>>Ok, Whirr does try to download the md5 file, but if it fails to find it, >>>>that's not a blocking error - it'll keep going anyway. What's after that in >>>>the logs? >>>> >>>> >>>>A. >>>> >>>> >>>>On Mon, Aug 5, 2013 at 4:19 PM, a b <autohan...@yahoo.com> wrote: >>>> >>>>ok - i'm not sure what you are asking. >>>>> >>>>>whirr launch-cluster --config ~/whirr/recipes/hadoop.properties >>>>> >>>>> >>>>> >>>>>where these are the properties i think i changed or added from the >>>>>original recipe: >>>>> >>>>> >>>>>whirr.cluster-name=hadoop-ec2 >>>>>whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,1 hadoop-datanode+hadoop-tasktracker >>>>>whirr.hardware-id=t1.micro >>>>>whirr.image-id=us-east-1/ami-25d9a94c >>>>>whirr.hadoop.version=1.2.1 >>>>>whirr.provider=aws-ec2 >>>>>whirr.identity=${env:AWS_ACCESS_KEY} >>>>>whirr.credential=${env:AWS_SECRET_KEY} >>>>>whirr.location-id=us-east-1 >>>>>whirr.java.install-function=install_oracle_jdk7 >>>>> >>>>> >>>>> >>>>>________________________________ >>>>> From: Andrew Bayer <andrew.ba...@gmail.com> >>>>>To: user@whirr.apache.org; a b <autohan...@yahoo.com> >>>>>Sent: Monday, August 5, 2013 4:11 PM >>>>>Subject: Re: cannot find hadoop-1.2.1.tar.gz.md5 - file suffix is mds >>>>> >>>>> >>>>> >>>>>Ok, it looks like they've actually been doing .mds for a while. Where are >>>>>you seeing this error? >>>>> >>>>> >>>>>A. >>>>> >>>>> >>>>>On Mon, Aug 5, 2013 at 4:09 PM, Andrew Bayer <andrew.ba...@gmail.com> >>>>>wrote: >>>>> >>>>>That looks like they broke the hadoop-1.2.1 release - the file should be >>>>>.md5. I'd bug the Hadoop project about that. >>>>>> >>>>>>A. >>>>>> >>>>>> >>>>>> >>>>>>On Mon, Aug 5, 2013 at 3:42 PM, a b <autohan...@yahoo.com> wrote: >>>>>> >>>>>>i get a whirr error: >>>>>>> >>>>>>> >>>>>>>Could not download >>>>>>>http://apache.osuosl.org/hadoop/common/hadoop-1.2.1/hadoop-1.2.1.tar.gz.md5 >>>>>>> >>>>>>> >>>>>>>when i browse over to: >>>>>>>http://apache.osuosl.org/hadoop/common/hadoop-1.2.1/ >>>>>>>i can see the file is named: hadoop-1.2.1.tar.gz.mds >>>>>>> >>>>>>> >>>>>>>how do i tell whirr to use a different suffix? >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >>> >>> >> >> >> > > >
