Hi Guys, I just upgraded to whirr 0.7.1 but now I'm seeing the same error again.
*The error:* 12/03/30 13:25:22 WARN conf.Configuration: DEPRECATED: hadoop-site.xml found in the classpath. Usage of hadoop-site.xml is deprecated. Instead use core-site.xml, mapred-site.xml and hdfs-site.xml to override properties of core-default.xml, mapred-default.xml and hdfs-default.xml respectively 12/03/30 13:25:24 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 0 time(s). 12/03/30 13:25:26 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 1 time(s). 12/03/30 13:25:27 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 2 time(s). 12/03/30 13:25:28 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 3 time(s). 12/03/30 13:25:30 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 4 time(s). 12/03/30 13:25:31 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 5 time(s). 12/03/30 13:25:33 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 6 time(s). 12/03/30 13:25:34 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 7 time(s). 12/03/30 13:25:35 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 8 time(s). 12/03/30 13:25:37 INFO ipc.Client: Retrying connect to server: / 107.21.79.75:8020. Already tried 9 time(s). Bad connection to FS. command aborted. *Background Information:* * * I already exported HADOOP_CONF_DIR * * *Hadoop Version* [Cluster] Hadoop 0.20.2 Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707 Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010 [Local] Hadoop 0.20.2 Subversion https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 911707 Compiled by chrisdo on Fri Feb 19 08:07:34 UTC 2010 *Last lines of whirr.log:* No directory, logging in with HOME=/ , error=, exitCode=0] 2012-03-30 13:15:30,387 INFO [org.apache.whirr.actions.ScriptBasedClusterAction] (main) Successfully executed configure script: [output=This function does nothing. It just needs to exist so Statements.call("retry_helpers") doesn't call something which doesn't exist starting datanode, logging to /var/log/hadoop/logs/hadoop-hadoop-datanode-ip-10-35-6-39.out No directory, logging in with HOME=/ starting tasktracker, logging to /var/log/hadoop/logs/hadoop-hadoop-tasktracker-ip-10-35-6-39.out No directory, logging in with HOME=/ , error=, exitCode=0] 2012-03-30 13:15:30,387 INFO [org.apache.whirr.actions.ScriptBasedClusterAction] (main) Successfully executed configure script: [output=This function does nothing. It just needs to exist so Statements.call("retry_helpers") doesn't call something which doesn't exist starting datanode, logging to /var/log/hadoop/logs/hadoop-hadoop-datanode-ip-10-115-130-203.out No directory, logging in with HOME=/ starting tasktracker, logging to /var/log/hadoop/logs/hadoop-hadoop-tasktracker-ip-10-115-130-203.out No directory, logging in with HOME=/ , error=, exitCode=0] 2012-03-30 13:15:30,387 INFO [org.apache.whirr.actions.ScriptBasedClusterAction] (main) Finished running configure phase scripts on all cluster instances 2012-03-30 13:15:30,387 INFO [org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main) Completed configuration of hadoop role hadoop-namenode 2012-03-30 13:15:30,388 INFO [org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main) Namenode web UI available at http://107.21.79.75:50070 2012-03-30 13:15:30,391 INFO [org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main) Wrote Hadoop site file /Users/edmaroliveiraferreira/.whirr/hadoop/hadoop-site.xml 2012-03-30 13:15:30,393 INFO [org.apache.whirr.service.hadoop.HadoopNameNodeClusterActionHandler] (main) Wrote Hadoop proxy script /Users/edmaroliveiraferreira/.whirr/hadoop/hadoop-proxy.sh 2012-03-30 13:15:30,394 INFO [org.apache.whirr.service.hadoop.HadoopJobTrackerClusterActionHandler] (main) Completed configuration of hadoop role hadoop-jobtracker 2012-03-30 13:15:30,394 INFO [org.apache.whirr.service.hadoop.HadoopJobTrackerClusterActionHandler] (main) Jobtracker web UI available at http://107.21.79.75:50030 2012-03-30 13:15:30,394 INFO [org.apache.whirr.service.hadoop.HadoopDataNodeClusterActionHandler] (main) Completed configuration of hadoop role hadoop-datanode 2012-03-30 13:15:30,394 INFO [org.apache.whirr.service.hadoop.HadoopTaskTrackerClusterActionHandler] (main) Completed configuration of hadoop role hadoop-tasktracker 2012-03-30 13:15:30,395 INFO [org.apache.whirr.state.FileClusterStateStore] (main) Wrote instances file /Users/edmaroliveiraferreira/.whirr/hadoop/instances 2012-03-30 13:15:30,405 DEBUG [org.apache.whirr.service.ComputeCache] (Thread-3) closing ComputeServiceContext [id=aws-ec2, endpoint=https://ec2. us-east-1.amazonaws.com, apiVersion=2010-06-15, identity=08WMRG9HQYYGVQDT57R2, iso3166Codes=[US-VA, US-CA, IE, SG, JP-13]] *My haoop-ec2.properties file* whirr.cluster-name=hadoop whirr.instance-templates=1 hadoop-namenode+hadoop-jobtracker,22 hadoop-datanode+hadoop-tasktracker whirr.instance-templates-max-percent-failures=100 hadoop-namenode+hadoop-jobtracker,90 hadoop-datanode+hadoop-tasktracker whirr.provider=aws-ec2 whirr.identity=${env:AWS_ACCESS_KEY_ID} whirr.credential=${env:AWS_SECRET_ACCESS_KEY} whirr.location-id=us-east-1 Thanks. On Fri, Feb 24, 2012 at 2:23 PM, Edmar Ferreira < [email protected]> wrote: > Yes, It makes sense. Looking forward to see the 0.9.0 version. > Thanks for your great work guys. > > > On Fri, Feb 24, 2012 at 2:18 PM, Andrei Savu <[email protected]>wrote: > >> >> On Fri, Feb 24, 2012 at 4:11 PM, Edmar Ferreira < >> [email protected]> wrote: >> >>> There are any plans to expand this limit ? >> >> >> Yes. The basic idea is that I think we should be able to start large >> cluster by resizing in multiple >> steps smallers ones and rebalancing things on the way as needed. Does it >> make sense to you? >> >> I expect to have something functional for this in 0.9.0 by the time we >> add the ability to resize clusters. >> >> Also there is some work happening in jclouds on being able to start a >> large number of servers at the same time: >> http://www.jclouds.org/documentation/reference/pool-design >> > > > > -- > Edmar Ferreira > Co-Founder at Everwrite > > -- Edmar Ferreira Co-Founder at Everwrite
