zhongyu09 commented on a change in pull request #30998:
URL: https://github.com/apache/spark/pull/30998#discussion_r553782354
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -1431,4 +1431,28 @@ class AdaptiveQueryExecSuite
}
}
}
+
+ test("SPARK-33933: AQE broadcast should not timeout with slow map tasks") {
Review comment:
I try the test tens of times and the test failed twice. As we discussed,
the guarantee is not strict since the submit of broadcast job and shuffle map
job are in different thread. So there's still risk for the shuffle map job
schedule earlier before broadcast job. I wonder should we need to remove the UT
until we thorough resolve the issue.
CC @cloud-fan @dongjoon-hyun
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
