mridulm commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1432382298
##########
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala:
##########
@@ -840,6 +856,7 @@ class DAGSchedulerSuite extends SparkFunSuite with
TempLocalSparkContext with Ti
assert(failure.getMessage === "Job aborted due to stage failure: some
failure")
assert(sparkListener.failedStages === Seq(0))
assertDataStructuresEmpty()
+ Thread.sleep(3000)
Review Comment:
Why do we need this `sleep` ?
##########
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala:
##########
@@ -54,12 +54,30 @@ import org.apache.spark.util.ArrayImplicits._
class DAGSchedulerEventProcessLoopTester(dagScheduler: DAGScheduler)
extends DAGSchedulerEventProcessLoop(dagScheduler) {
+ dagScheduler.setEventProcessLoop(this)
Review Comment:
To make sure I am not missing anything, this is to handle case of a
`DAGSchedulerEvent` processing resulting in another `DAGSchedulerEvent` getting
generated (as part of the logic for the event) - and we want the first to
complete before the second is handled (which is how 'normal' flow would be).
Is this right ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
