ulysses-you commented on code in PR #57491:
URL: https://github.com/apache/spark/pull/57491#discussion_r3648281070
##########
sql/core/src/test/scala/org/apache/spark/sql/DataFrameSetOperationsSuite.scala:
##########
@@ -1659,6 +1659,187 @@ class DataFrameSetOperationsSuite extends
SharedSparkSession with AdaptiveSparkP
}
}
+ test("SPARK-58317: union partitioning - PartitioningCollection child
intersects to single") {
Review Comment:
Good catch, thank you. You are right -- `k` was pruned, so no
`PartitioningCollection` reached the union and the test passed on master. Fixed
in b79f1b2: `GROUP BY c1, c2, c3, k` keeps the first branch's collection alive,
and `t4.c1 AS k` (the left join's build side) keeps the second branch a single
`Hash(c1)`. I verified the revised test now fails on pre-PR code with
`UnknownPartitioning(0) was not instance of HashPartitioning` and passes with
the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
