felixloesing opened a new pull request, #13208:
URL: https://github.com/apache/gluten/pull/13208
## What changes are proposed in this pull request?
When `SortAggregate` is offloaded to a native hash aggregate,
`EliminateLocalSort` removes the now-unneeded local sort. It only looked
through a single `ProjectExec`/`ProjectExecTransformer`, so the sort survived
when a `ColumnarPartialProjectExec` (e.g. from a Hive/Scala UDF) sat in between:
```
SortHashAggregateExecTransformer
+- ProjectExecTransformer
+- ColumnarPartialProjectExec
+- SortExecTransformer <- not eliminated
```
This PR lets the rule recurse through any chain of order-preserving unary
nodes that don't require child ordering.
Besides the wasted sort, this caused a result difference vs vanilla Spark:
Velox's sort is not stable, so `collect_list` values within a group were
reordered, while vanilla preserved input order.
## How was this patch tested?
Added a test in `UDFPartialProjectSuite` asserting the sort is eliminated
and `collect_list` output matches vanilla. It fails without the fix.
## Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude claude-opus-5-5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
