Can you try to use the IP address instead of the name 'base'? I also experience the same problem before, it worked after changed to IP.
Thanks, Chen jingci -----Original Message----- From: "Thorsten Bergler" <thorsten.berg...@tbonline.de> Sent: 2/11/2013 2:03 To: "user@spark.incubator.apache.org" <user@spark.incubator.apache.org> Subject: Spark with HDFS: ERROR Worker: Connection to master failed! Shuttingdown. Hello, I am new to Spark and doing my first steps with it today. Right now I am having trouble with the error: ERROR Worker: Connection to master failed! Shutting down. So far, I found out the following: The standalone version of Spark (without Hadoop-HDFS and YARN) works perfectly. The master starts and also the worker starts and get registered at the master. Logfile of the master: Spark Command: /usr/lib/jvm/jdk//bin/java -cp :/home/thorsten/spark/conf:/home/thorsten/spark/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop1.0.4.jar -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.master.Master --ip base --port 7077 --webui-port 8080 ======================================== 13/11/01 18:51:14 INFO Slf4jEventHandler: Slf4jEventHandler started 13/11/01 18:51:14 INFO Master: Starting Spark master at spark://base:7077 13/11/01 18:51:14 INFO MasterWebUI: Started Master web UI at http://base:8080 13/11/01 18:51:17 INFO Master: Registering worker base:32942 with 1 cores, 512.0 MB RAM Because I want to try out Spark with HDFS too, I tried to compile it also that way: SPARK_HADOOP_VERSION=2.2.0 sbt/sbt assemblyBut with this compiled version, the registration of workers at the master is not working anymore. Here is the logfile of the slave: Spark Command: java -cp :/home/thorsten/spark-hdfs/conf:/home/thorsten/spark-hdfs/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop2.2.0.jar -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.worker.Worker spark://base:7077 --webui-port 8081 ======================================== 13/11/01 18:49:57 INFO Slf4jEventHandler: Slf4jEventHandler started 13/11/01 18:49:57 INFO Worker: Starting Spark worker base:37950 with 1 cores, 512.0 MB RAM 13/11/01 18:49:57 INFO Worker: Spark home: /home/thorsten/spark-hdfs 13/11/01 18:49:57 INFO WorkerWebUI: Started Worker web UI at http://base:8081 13/11/01 18:49:57 INFO Worker: Connecting to master spark://base:7077 13/11/01 18:49:58 ERROR Worker: Connection to master failed! Shutting down. The logfile of the master: Spark Command: /usr/lib/jvm/jdk//bin/java -cp :/home/thorsten/spark-hdfs/conf:/home/thorsten/spark-hdfs/assembly/target/scala-2.9.3/spark-assembly-0.8.0-incubating-hadoop2.2.0.jar -Djava.library.path= -Xms512m -Xmx512m org.apache.spark.deploy.master.Master --ip base --port 7077 --webui-port 8080 ======================================== 13/11/01 18:49:55 INFO Slf4jEventHandler: Slf4jEventHandler started 13/11/01 18:49:55 INFO Master: Starting Spark master at spark://base:7077 13/11/01 18:49:55 INFO MasterWebUI: Started Master web UI at http://base:8080 Hope anybody could help me, solving that problem. Thanks Thorsten
