Hello! Just curious if anybody could respond to my original message, if anybody knows about how to set the configuration variables that are handles by Jetty and not Spark's native framework..which is Akka I think?
Thanks On Thu, Jul 10, 2014 at 4:04 PM, Aris Vlasakakis <a...@vlasakakis.com> wrote: > Hi Spark folks, > > So on our production Spark cluster, it lives in the data center and I need > to attach to a VPN from my laptop, so that I can then submit a Spark > application job to the Spark Master (behind the VPN). > > However, the problem arises that I have a local IP address on the laptop > which is on a separate network segment than the VPN. I figured out that I > can set the SparkConf configuration property called "spark.driver.host" to > be my VPN IP address - this got me further, because I could initiate a > connection to the master. > > However, it was still not working. I was hoping to set the SparkConf > variables "spark.fileserver.uri" and "spark.httpBroadcast.uri" to NOT be my > local IP addresses, but the VPN IP addresses, because I think these > configuration variables actually pass my application files to the master > (in this case, it is a python script). So my spark.fileserver.uri and the > Spark Master need to be on the same network segment (the VPN subnetwork ). > > Am I on the right track? How can I set "spark.fileserver.uri" and > "spark.httpBroadcast.uri" ? I see that these are actually run by Jetty > server...any thoughts? > > Thank you so much! > > -- > Άρης Βλασακάκης > Aris Vlasakakis > -- Άρης Βλασακάκης Aris Vlasakakis
