Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1420598351
##########
core/src/main/scala/org/apache/spark/internal/config/package.scala:
##########
@@ -2603,4 +2603,13 @@ package object config {
.stringConf
.toSequence
.createWithDefault("org.apache.spark.sql.connect.client" :: Nil)
+
+ private[spark] val LEGACY_ABORT_STAGE_AFTER_CANCEL_TASKS =
+ ConfigBuilder("spark.legacy.scheduler.stage.abortAfterCancelTasks")
Review Comment:
> Btw, we should flip this switch on and off in the relevant tests to check
if the behavior is preserved.
It is a bit annoying to flip this conf switch on and off only for the
relevant unit tests in `DAGSchedulerSuite` since it uses a global
`SparkContext`. One way to do this is to might intrduce a new entire suite
called `DAGSchedulerWithAbortStageDisabledSuite` but not sure it's worth to do
that. Or we could extract the relevant unit tests into a separate suite and
then flip the conf with two on- and off- suites. a bit complicated though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
