juliuszsompolski commented on code in PR #30481:
URL: https://github.com/apache/spark/pull/30481#discussion_r1181684437
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala:
##########
@@ -926,13 +926,23 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val THRIFTSERVER_FORCE_CANCEL =
+ buildConf("spark.sql.thriftServer.interruptOnCancel")
+ .doc("When true, all running tasks will be interrupted if one cancels a
query. " +
+ "When false, all running tasks will remain until finished.")
+ .version("3.2.0")
+ .booleanConf
+ .createWithDefault(false)
Review Comment:
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/BroadcastExchangeExec.scala#L134
and
https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamExecution.scala#L270
unconditionally set interruptOnCancel=true for broadcasts and for streaming
queries. https://issues.apache.org/jira/browse/HDFS-1208 remains open, but it
seems there's not been issues resulting from using true in the other places.
Given that, do we need to bother protecting it with a config for
Thriftserver, or at least this config could be flipped to be enabled by
default? Having it default to false limits the default cancellation behaviour.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
