HeartSaVioR commented on a change in pull request #26207: [SPARK-29552][SQL]
fix the flaky test in multi-joins with local shuffle reader
URL: https://github.com/apache/spark/pull/26207#discussion_r337749802
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala
##########
@@ -164,7 +164,7 @@ class AdaptiveQueryExecSuite
val bhj = findTopLevelBroadcastHashJoin(adaptivePlan)
assert(bhj.size == 3)
// additional shuffle exchange introduced, only one shuffle reader to
local shuffle reader.
- checkNumLocalShuffleReaders(adaptivePlan, 1)
+ checkNumLocalShuffleReaders(adaptivePlan, 2)
Review comment:
Just to confirm, would the change make this value consistently be 2? Because
the value has changed to 2 but the value was actually flaky depending on the
situation/randomness (maybe).
You may want to run the same for what I've discovered, 1) solely in local
dev, 2) test suite in local dev, 3) trigger CI for 5 times or alike.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
