[ https://issues.apache.org/jira/browse/SPARK-12129?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15037969#comment-15037969 ]
Doug Balog commented on SPARK-12129: ------------------------------------ I thought using {{--conf spark.submit.deployMode=cluster}} with SparkPi was an easy way to demonstrate the problem. I'm trying to set the default deployMode for {{spark-submit}} to be cluster mode. So I have {{master}} set to {{yarn}}, and {{spark.submit.deployMode}} set to {{cluster}} in {{spark-defaults.conf}}. If I set {{master}} to {{yarn-cluster}}, {{spark-shell}} complains about cluster deploy mode is not applicable. > spark.submit.deployMode not being picked up in SparkSubmitArguments > ------------------------------------------------------------------- > > Key: SPARK-12129 > URL: https://issues.apache.org/jira/browse/SPARK-12129 > Project: Spark > Issue Type: Bug > Components: Spark Submit > Affects Versions: 1.3.1, 1.4.1, 1.5.2 > Environment: YARN, MESOS > Reporter: Doug Balog > Priority: Trivial > > In sparkSubmitArguments.scala > {code} > deployMode = Option(deployMode).orElse(env.get("DEPLOY_MODE")).orNull > {code} > This code should probably check to see if the env variable SPARK_SUBMIT is > set and then look up `spark.submit.deployMode` in sparkProperties. > To see this, run something like {code}spark-submit -v --conf > spark.submit.deployMode=cluster --class org.apache.spark.examples.SparkPi > $SPARK_HOME/lib/spark-examples-1.6.0-hadoop.2.6.0.jar{code} > The last `spark.submit.deployMode` will be set to client. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org