metanil commented on code in PR #59760:
URL: https://github.com/apache/spark/pull/59760#discussion_r4433926137
##########
sql/core/src/test/scala/org/apache/spark/sql/execution/exchange/EnsureRequirementsSuite.scala:
##########
@@ -1599,6 +1599,80 @@ class EnsureRequirementsSuite extends SharedSparkSession
{
}
}
+ test("SPARK-60522: a cogroup over a transform Spark cannot evaluate still
pairs as it stands") {
+ // `pickCoPartitionTarget` decides both which layout the other children
are shuffled onto and
+ // which a child is read against as it stands. Only the first evaluates
the transform, so a
+ // transform Spark cannot evaluate is refused only for that. A cogroup has
only this path,
+ // since `checkKeyGroupCompatible` declines for anything that is not a
`ShuffledJoin`. So two
+ // sides on the same such transform and keys, which nothing has to
evaluate, keep their
+ // layout, while a plain side is not shuffled onto it.
+ val nonScalar = new BoundFunction {
Review Comment:
Done. I kept `nestedBucket` and `structBucket` local, since they're built
per declared type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
