Re: SparkWebUI and Master URL on EC2

2016-07-21 Thread Ismaël Mejía
Hello, If you are using EMR you probably need to create a SSH tunnel so you can access the web ports of the master instance. https://docs.aws.amazon.com/ElasticMapReduce/latest/DeveloperGuide/emr-ssh-tunnel.html Also verify that your EMR cluster is not behind a private VPC, in this case you must

Re: SparkWebUI and Master URL on EC2

2016-07-21 Thread Jacek Laskowski
Hi, What's in the logs of spark-shell? There should be the host and port of web UI. what the public IP of the host where you execute spark-shell? Use it for 4040. I don't think you use Spark Standalone cluster (the other address with 8080) if you simply spark-shell (unless you've got spark.master