sarutak commented on code in PR #54004:
URL: https://github.com/apache/spark/pull/54004#discussion_r2734952985
##########
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala:
##########
@@ -310,6 +310,9 @@ private[spark] class SparkSubmit extends Logging {
error("Cluster deploy mode is not applicable to Spark SQL shell.")
case (_, CLUSTER) if isThriftServer(args.mainClass) =>
error("Cluster deploy mode is not applicable to Spark Thrift server.")
+ case (YARN, CLUSTER) if isConnectServer(args.mainClass) =>
+ logInfo("SparkConnectServer is starting in cluster deploy mode." +
+ "Use `yarn application -kill` command or YARN client API to stop the
server.")
case (_, CLUSTER) if isConnectServer(args.mainClass) =>
error("Cluster deploy mode is not applicable to Spark Connect server.")
Review Comment:
> BTW, seems it's also applicable for Standalone and K8s cluster mode?
For K8s, I think users have alternative way to run SparkConnectServer on the
cluster like ThriftServer.
https://issues.apache.org/jira/browse/SPARK-23078?focusedCommentId=16705282&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16705282
For Standalone, as far as I know, we need further code change. So, I'd like
to fix it for YARN for now as it just requires simple code change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
