Ngone51 commented on a change in pull request #28178: [SPARK-31409][SQL][TEST]
Fix failed tests due to result order changing when enable AQE
URL: https://github.com/apache/spark/pull/28178#discussion_r406658610
##########
File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala
##########
@@ -157,6 +157,8 @@ class SQLQueryTestSuite extends QueryTest with
SharedSparkSession {
protected override def sparkConf: SparkConf = super.sparkConf
// Fewer shuffle partitions to speed up testing.
.set(SQLConf.SHUFFLE_PARTITIONS, 4)
+ // Disable BHJ optimization to avoid changing result order when we enable
AQE
+ .set(SQLConf.AUTO_BROADCASTJOIN_THRESHOLD, -1.toLong)
Review comment:
Because we still want to test it with AQE as it's good for AQE to improve
test coverage.
We don't have separate SQL-file based test cases for AQE. AQE is only a way
of optimization, so it ideally should not have any specific requirements for
the query itself.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
