HyukjinKwon commented on code in PR #39041:
URL: https://github.com/apache/spark/pull/39041#discussion_r1052015205
##########
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:
##########
@@ -229,15 +229,20 @@ private[spark] class SparkSubmit extends Logging {
var childMainClass = ""
// Set the cluster manager
- val clusterManager: Int = args.master match {
- case "yarn" => YARN
- case m if m.startsWith("spark") => STANDALONE
- case m if m.startsWith("mesos") => MESOS
- case m if m.startsWith("k8s") => KUBERNETES
- case m if m.startsWith("local") => LOCAL
- case _ =>
- error("Master must either be yarn or start with spark, mesos, k8s, or
local")
- -1
+ val clusterManager: Int = args.maybeMaster match {
+ case Some(v) =>
+ assert(args.maybeRemote.isEmpty)
+ v match {
+ case "yarn" => YARN
+ case m if m.startsWith("spark") => STANDALONE
+ case m if m.startsWith("mesos") => MESOS
+ case m if m.startsWith("k8s") => KUBERNETES
+ case m if m.startsWith("local") => LOCAL
+ case _ =>
+ error("Master must either be yarn or start with spark, mesos, k8s,
or local")
+ -1
+ }
+ case None => LOCAL // default master or remote mode.
Review Comment:
Yeah we have the default master as `local[*]` when we use shells or
spark-submit, see
https://github.com/apache/spark/pull/39041/files/75bfce50110079a6918cb81169d1cd2197d56eba#diff-d02c773357341d3de704cbad28d94a4c29a5ea3f37bddbaaad3eec6064e5d23aL214
Here it uses `local[*]` internally if `remote` is provided.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
