Spark is not able to communicate with your hadoop hdfs. Is your hdfs running, if so can you try to explicitly connect to it with hadoop command line tools giving full hostname & port. Or test port using > telnet localhost 9000 In all likelyhood either your hdfs is down, bound to wrong port/ip that spark cannot access
Mayur Rustagi Ph: +1 (760) 203 3257 http://www.sigmoidanalytics.com @mayur_rustagi <https://twitter.com/mayur_rustagi> On Tue, Aug 5, 2014 at 11:33 PM, alamin.ishak <alamin.is...@gmail.com> wrote: > Hi, > Anyone? Any input would be much appreciated > > Thanks, > Amin > On 5 Aug 2014 00:31, "Al Amin" <[hidden email] > <http://user/SendEmail.jtp?type=node&node=11477&i=0>> wrote: > >> Hi all, >> >> Any help would be much appreciated. >> >> Thanks, >> Al >> >> >> On Mon, Aug 4, 2014 at 7:09 PM, Al Amin <[hidden email] >> <http://user/SendEmail.jtp?type=node&node=11477&i=1>> wrote: >> >>> Hi all, >>> >>> I have setup 2 nodes (master and slave1) on stand alone mode. Tried >>> running SparkPi example and its working fine. However when I move on to >>> wordcount its giving me below error: >>> >>> 14/08/04 21:40:33 INFO storage.MemoryStore: ensureFreeSpace(32856) >>> called with curMem=0, maxMem=311387750 >>> 14/08/04 21:40:33 INFO storage.MemoryStore: Block broadcast_0 stored as >>> values in memory (estimated size 32.1 KB, free 296.9 MB) >>> 14/08/04 21:40:34 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 0 time(s). >>> 14/08/04 21:40:35 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 1 time(s). >>> 14/08/04 21:40:36 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 2 time(s). >>> 14/08/04 21:40:37 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 3 time(s). >>> 14/08/04 21:40:38 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 4 time(s). >>> 14/08/04 21:40:39 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 5 time(s). >>> 14/08/04 21:40:40 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 6 time(s). >>> 14/08/04 21:40:41 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 7 time(s). >>> 14/08/04 21:40:42 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 8 time(s). >>> 14/08/04 21:40:43 INFO ipc.Client: Retrying connect to server: master/ >>> 10.0.1.27:9000. Already tried 9 time(s). >>> Exception in thread "main" java.lang.RuntimeException: >>> java.net.ConnectException: Call to master/10.0.1.27:9000 failed on >>> connection exception: java.net.ConnectException: Connection refused >>> >>> >>> 1) how to fix this issue? I have configure hostname --fqdn accordingly. >>> >>> 2) I could see that in my logs that my master/worker deploy >>> configuration is -Xms512m -Xmx512m. Is there any way that I can increase >>> it? Or 512mb is just fine? AFAIK, spark require huge memory. >>> >>> 3) I have a hadoop cluster and its working. Could anyone point me how to >>> integrate Yarn with Spark? Any good tutorial would be very useful >>> >>> >>> Thanks, >>> Al >>> >> >> > ------------------------------ > View this message in context: Re: Configuration setup and Connection > refused > <http://apache-spark-user-list.1001560.n3.nabble.com/Re-Configuration-setup-and-Connection-refused-tp11477.html> > Sent from the Apache Spark User List mailing list archive > <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com. >
