Okay, This seems to be either a code version issue or a communication issue. It works if I execute the spark shell from the master node. It doesn't work if I run it from my laptop and connect to the master node.
I had opened the ports for the WebUI (8080) and the cluster manager (7077) for the master node or it fails much sooner. Do I need to open up the ports for the workers as well? I used the spark-ec2 install script with --spark-version using both 1.0.2 and then again with the git hash tag that corresponds to 1.1.0rc4 (2f9b2bd7844ee8393dc9c319f4fefedf95f5e460). In both cases I rebuilt from source using the same codebase on my machine and moved the entire project into /root/spark (since to run the spark-shell it needs to match the same path as the install on ec2). Could I have missed something here? Thanks. Jim -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Querying-a-parquet-file-in-s3-with-an-ec2-install-tp13737p13802.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
