Hello, Can you drop the url:
spark://master:7077 The url is used when running Spark in standalone mode. Regards <div>-------- Original message --------</div><div>From: Marco Mistroni <mmistr...@gmail.com> </div><div>Date:15/01/2017 16:34 (GMT+02:00) </div><div>To: User <user@spark.apache.org> </div><div>Subject: Running Spark on EMR </div><div> </div>hi all could anyone assist here? i am trying to run spark 2.0.0 on an EMR cluster,but i am having issues connecting to the master node So, below is a snippet of what i am doing sc = SparkSession.builder.master(sparkHost).appName("DataProcess").getOrCreate() sparkHost is passed as input parameter. that was thought so that i can run the script locally on my spark local instance as well as submitting scripts on any cluster i want Now i have 1 - setup a cluster on EMR. 2 - connected to masternode 3 - launch the command spark-submit myscripts.py spark://master:7077 But that results in an connection refused exception Then i have tried to remove the .master call above and launch the script with the following command spark-submit --master spark://master:7077 myscript.py but still i am getting connectionREfused exception I am using Spark 2.0.0 , could anyone advise on how shall i build the spark session and how can i submit a pythjon script to the cluster? kr marco