peter-toth commented on code in PR #55876:
URL: https://github.com/apache/spark/pull/55876#discussion_r3242430521
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/AliasAwareOutputExpression.scala:
##########
@@ -94,6 +94,16 @@ trait PartitioningPreservingUnaryExecNode extends
UnaryExecNode
kps: Seq[KeyedPartitioning]): LazyList[KeyedPartitioning] = {
if (kps.isEmpty) return LazyList.empty
val numPositions = kps.head.expressions.length
+ // The function assumes all input KPs share the same `partitionKeys`,
which implies matching
+ // expression arity. This invariant is asserted by [[GroupPartitionsExec]]
and is established
+ // by the constructors of [[PartitioningCollection]] feeding this method
(a join's
+ // `PartitioningCollection(left.outputPartitioning,
right.outputPartitioning)` combines KPs
+ // that have been aligned by [[EnsureRequirements]] to the same join
keys). If the invariant
+ // is ever violated upstream, fail early with a clear message instead of
throwing an opaque
+ // `IndexOutOfBoundsException` from `kp.expressions(i)` below.
+ assert(kps.forall(_.expressions.length == numPositions),
Review Comment:
I wonder if we should move these invariant checks into
`PartitioningCollection`.
Maybe `expressions.length =` + `partitionKeys eq` checks on KPs for maximum
efficiency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
