pan3793 commented on code in PR #54004:
URL: https://github.com/apache/spark/pull/54004#discussion_r2734980442
##########
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:
##########
@@ -310,6 +310,9 @@ private[spark] class SparkSubmit extends Logging {
error("Cluster deploy mode is not applicable to Spark SQL shell.")
case (_, CLUSTER) if isThriftServer(args.mainClass) =>
error("Cluster deploy mode is not applicable to Spark Thrift server.")
+ case (YARN, CLUSTER) if isConnectServer(args.mainClass) =>
+ logInfo("SparkConnectServer is starting in cluster deploy mode." +
+ "Use `yarn application -kill` command or YARN client API to stop the
server.")
case (_, CLUSTER) if isConnectServer(args.mainClass) =>
error("Cluster deploy mode is not applicable to Spark Connect server.")
Review Comment:
One of advantage of Spark is that `spark-submit` provides a consistent
experience in most cases across different resource managers, and users are easy
to switch to different resource managers, e.g., migrate from YARN to K8s, by
providing a `--master <foo>`, I suggest making it also work on K8s cluster
mode, if there is no technical challenge (maybe just require another 1 line
change like for YARN).
> For Standalone, as far as I know, we need further code change.
Good to know, thanks for the information.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
