david-mollitor-db opened a new pull request, #58712: URL: https://github.com/apache/spark/pull/58712
### What changes were proposed in this pull request? `CollapseProject.isCheap(e: Expression)` decides whether an expression is cheap enough to duplicate/inline. It recognizes `Attribute` / `OuterReference` / `BoundReference`, foldable expressions, some `PythonUDF`s, and `Alias` / `ExtractValue` over cheap children — but not the `Collate` expression. This PR teaches it that `Collate` is as cheap as its value child: ```scala // `Collate` only re-tags the collation in the type; at runtime it is a pass-through // (eval/genCode delegate to the child) and never evaluates its collation argument, so it is // as cheap to duplicate as its value child. case c: Collate => isCheap(c.child) ``` Only the value child is inspected — the collation argument is a constant `ResolvedCollation` marker that is never evaluated — so a non-cheap value child (e.g. `Collate(substring(col), ...)`) correctly stays non-cheap, preserving the SPARK-40228 guard. ### Why are the changes needed? `Collate` (the SQL `collate(expr, 'COLLATION')` function / `expr COLLATE COLLATION` syntax) is a pure pass-through: its `eval` delegates to the child (`child.eval(row)`), its codegen delegates to the child (`child.genCode(ctx)`), and it never evaluates its collation argument. It performs no per-row work — it only re-tags the collation carried on the result *type*. This is stated in the expression's own doc comment: [Collate is pass-through](https://github.com/apache/spark/blob/0cfb8bfca7d87769b618b43010da8f50c0b62218/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collationExpressions.scala#L69-L74). Because `isCheap` did not know this, `Collate(cheapChild)` was treated as non-cheap, needlessly blocking safe duplication/inlining of collated columns across every `isCheap`-gated rule (CollapseProject inlining, the multi-LIKE `LikeSimplification` rules, the `FilterExec` CSE gate, `RewriteWithExpression`). ### Does this PR introduce _any_ user-facing change? No. `Collate` returns the child's value unchanged and only sets collation metadata on the type, so duplicating `Collate(cheapChild)` is semantically identical to duplicating the child. Results are unchanged for all inputs; the optimizer may now inline/duplicate a collated column where it previously could not. ### How was this patch tested? Added a `CollapseProjectSuite` test asserting `isCheap(Collate(attr, ...))` is `true` while `isCheap(Collate(substring(attr), ...))` is `false` (the SPARK-40228 protection stays intact). Existing `CollapseProjectSuite` and `LikeSimplificationSuite` suites pass, and `scalastyle` is clean. This also unblocks the collated-column case of the gated `startsAndEndsWith` `LikeSimplification` rewrite in [SPARK-59371](https://issues.apache.org/jira/browse/SPARK-59371): with `Collate` recognized as cheap, `collate(col) LIKE 'a%c'` is simplified again instead of being left as a plain `Like`. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Opus 4.8 This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
