cloud-fan commented on a change in pull request #30998:
URL: https://github.com/apache/spark/pull/30998#discussion_r553789697
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -1431,4 +1431,28 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("SPARK-33933: AQE broadcast should not timeout with slow map tasks") {
Review comment:
I was worried about this and tested the PR twice...
Maybe it's better to be more low-level and verify the stage submission
order, which is more reliable. We can use SparkListener to watch the job start
events, and see if the job with less tasks (the broadcast stage) is started
first.
@zhongyu09 what do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
