Ngone51 commented on code in PR #43954:
URL: https://github.com/apache/spark/pull/43954#discussion_r1432407891
##########
core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala:
##########
@@ -54,12 +54,30 @@ import org.apache.spark.util.ArrayImplicits._
class DAGSchedulerEventProcessLoopTester(dagScheduler: DAGScheduler)
extends DAGSchedulerEventProcessLoop(dagScheduler) {
+ dagScheduler.setEventProcessLoop(this)
Review Comment:
Right. Before this change, the the event process loop are derived. There are
two event process loops involved in the testing DAGScheduler (i.e.,
`MyDAGScheduler`). One is the manually created -
`DAGSchedulerEventProcessLoopTester` and another one is the default
`DAGSchedulerEventProcessLoop` that created inside `MyDAGScheduler`'s base
class `DAGScheduler.` And any explicitly created events that posted during test
using `DAGSchedulerEventProcessLoopTester`. And other implictly created events
(inside `DAGScheduler`) are posted by `DAGSchedulerEventProcessLoop`. This
leads to the events being processed by two separate threads so the "first to
complete before the second" policy can no longer be guaranteed. This change
ensures we only use one event process loop during the test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
