cloud-fan commented on a change in pull request #30998:
URL: https://github.com/apache/spark/pull/30998#discussion_r553789697
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -1431,4 +1431,28 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("SPARK-33933: AQE broadcast should not timeout with slow map tasks") {
Review comment:
I was worried about this and tested the PR twice...
Maybe it's better to write a UT to verify the stage submission order, which
is more reliable. We can use SparkListener to watch the job start events, and
see if the job with less tasks (the broadcast stage) is started first.
@zhongyu09 what do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
