peter-toth commented on code in PR #52474:
URL: https://github.com/apache/spark/pull/52474#discussion_r2394134608
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/AliasAwareOutputExpression.scala:
##########
@@ -128,6 +128,9 @@ trait AliasAwareQueryOutputOrdering[T <: QueryPlan[T]]
}
}
}
- newOrdering.takeWhile(_.isDefined).flatten.toSeq
+ newOrdering.takeWhile(_.isDefined).flatten.toSeq ++
outputExpressions.filter {
+ case Alias(child, _) => child.foldable
+ case expr => expr.foldable
+ }.map(SortOrder(_, Ascending).copy(isConstant = true))
Review Comment:
The problem seems to be that
[InMemoryRelation.withOutput()](https://github.com/apache/spark/blob/e04fd595370808bbf12b4c50980a86085fd20782/sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala#L472-L473)
doesn't remap `outputOrdering`. And because `outputOrdering` is present in
`InMemoryRelation` as [case class
argument](https://github.com/apache/spark/blob/e04fd595370808bbf12b4c50980a86085fd20782/sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala#L431-L434)
the unmapped ordering attributes are considered missing inputs.
This seems to be another hidden issue with `InMemoryRelation.outputOrdering`
and got exposed with this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
