Hi Jim, This approach will not work right out of the box. You need to understand a few things. A driver program and the master will be communicating with each other, for that you need to open up certain ports for your public ip (Read about port forwarding <http://portforward.com/>). Also on the cluster you need to set *spark.driver.host* and *spark.driver.port *(by default this is random) pointing to your public ip and the port that you opened up.
Thanks Best Regards On Thu, Sep 11, 2014 at 11:52 PM, Jim Carroll <[email protected]> wrote: > Hello all, > > I'm trying to run a Driver on my local network with a deployment on EC2 and > it's not working. I was wondering if either the master or slave instances > (in standalone) connect back to the driver program. > > I outlined the details of my observations in a previous post but here is > what I'm seeing: > > I have v1.1.0 installed (the new tag) on ec2 using the spark-ec2 script. > I have the same version of the code built locally. > I edited the master security group to allow inbound access from anywhere to > 7077 and 8080. > I see a connection take place. > I see the workers fail with a timeout when any job is run. > The master eventually removes the driver's job. > > I supposed this makes sense if there's a requirement for either the worker > or the master to be on the same network as the driver. Is that the case? > > Thanks > Jim > > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Network-requirements-between-Driver-Master-and-Slave-tp13997.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
