Manoj Samel created SPARK-6653: ---------------------------------- Summary: New configuration property to specify port for sparkYarnAM actor system Key: SPARK-6653 URL: https://issues.apache.org/jira/browse/SPARK-6653 Project: Spark Issue Type: Improvement Components: YARN Affects Versions: 1.3.0 Environment: Spark On Yarn Reporter: Manoj Samel
In 1.3.0 code line sparkYarnAM actor system is started on random port. See org.apache.spark.deploy.yarn ApplicationMaster.scala:282 actorSystem = AkkaUtils.createActorSystem("sparkYarnAM", Utils.localHostName, 0, conf = sparkConf, securityManager = securityMgr)._1 This may be issue when ports between Spark client and the Yarn cluster are limited by firewall and not all ports are open between client and Yarn cluster. Proposal is to introduce new property spark.am.actor.port and change code to val port = sparkConf.getInt("spark.am.actor.port", 0) actorSystem = AkkaUtils.createActorSystem("sparkYarnAM", Utils.localHostName, port, conf = sparkConf, securityManager = securityMgr)._1 -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org