jonathan-albrecht-ibm commented on code in PR #49813:
URL: https://github.com/apache/spark/pull/49813#discussion_r1953101262
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/adaptive/AdaptiveQueryExecSuite.scala:
##########
@@ -1130,7 +1130,7 @@ class AdaptiveQueryExecSuite
SQLConf.AUTO_BROADCASTJOIN_THRESHOLD.key -> "-1",
SQLConf.SHUFFLE_PARTITIONS.key -> "100",
SQLConf.SKEW_JOIN_SKEWED_PARTITION_THRESHOLD.key -> "800",
- SQLConf.ADVISORY_PARTITION_SIZE_IN_BYTES.key -> "1000") {
+ SQLConf.ADVISORY_PARTITION_SIZE_IN_BYTES.key -> "900") {
Review Comment:
Yes, I believe so. I didn't dig as deeply into this test as I did with the
CoalesceShufflePartitionsSuite but the partitioning is always affected by the
different lz4 compression behavior between big and little endian. The only
thing needed for this test to pass is a change to a partition size parameter
which is definitely consistent with the lz4 compression behavior.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
