cloud-fan commented on code in PR #37238:
URL: https://github.com/apache/spark/pull/37238#discussion_r930717923
##########
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/V2ScanRelationPushDown.scala:
##########
@@ -400,6 +400,13 @@ object V2ScanRelationPushDown extends Rule[LogicalPlan]
with PredicateHelper {
}
}
+ private def findGroupColumn(alias: Alias): Option[AttributeReference] =
alias match {
+ case alias @ Alias(attr: AttributeReference, name) if
attr.name.startsWith("group_col_") =>
+ Some(AttributeReference(name, attr.dataType)(alias.exprId))
+ case Alias(alias: Alias, _) => findGroupColumn(alias)
Review Comment:
I feel it's a bit hacky to assume the Alias contains the actual grouping
columns. How about we generate the name mapping (grouping attribute to actual
group column name) during agg pushdown, put the name mapping in
`ScanBuilderHolder`, and use the mapping to rewrite order by expression during
limit pushdown?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
