You might try also setting spark.driver.host to the correct IP in the conf/spark-env.sh SPARK_JAVA_OPTs as well.
e.g., -Dspark.driver.host=192.168.250.47 On Sat, Oct 5, 2013 at 2:45 PM, Aaron Babcock <aaron.babc...@gmail.com>wrote: > Hello, > > I am using spark through a vpn. My driver machine ends up with two ip > addresses, one routable from the cluster and one not. > > Things generally work when I set the SPARK_LOCAL_IP environment > variable to the proper ip address. > > However, when I try to use the take function ie: myRdd.take(1), I run > into a hiccup. From the logfiles on the workers I can see that they > trying to connect to the nonroutable ip address, they are not > respecting SPARK_LOCAL_IP somehow. > > Here is the relevant worker log snippet, 192.168.250.47 is the correct > routable ip address of the driver, 192.168.0.7 is the incorrect > address of the driver. Any thoughts about what else I need to > configure? > > 13/10/05 16:17:36 INFO ConnectionManager: Accepted connection from > [192.168.250.47/192.168.250.47] > 13/10/05 16:18:41 WARN SendingConnection: Error finishing connection > to /192.168.0.7:60513 > java.net.ConnectException: Connection timed out > at sun.nio.ch.SocketChannelImpl.$$YJP$$checkConnect(Native Method) > at sun.nio.ch.SocketChannelImpl.checkConnect(SocketChannelImpl.java) > at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) > at spark.network.SendingConnection.finishConnect(Connection.scala:221) > at > spark.network.ConnectionManager.spark$network$ConnectionManager$$run(ConnectionManager.scala:127) > at spark.network.ConnectionManager$$anon$4.run(ConnectionManager.scala:70) > 13/10/05 16:18:41 INFO ConnectionManager: Handling connection error on > connection to ConnectionManagerId(192.168.0.7,60513) > 13/10/05 16:18:41 INFO ConnectionManager: Removing SendingConnection > to ConnectionManagerId(192.168.0.7,60513) >
