wangyum opened a new pull request, #59344:
URL: https://github.com/apache/spark/pull/59344
### What changes were proposed in this pull request?
`ExprUtils.canEvaluateUnconditionally` (introduced in SPARK-58066)
determines whether an expression is safe to evaluate on rows where the original
execution plan would not have evaluated it (e.g. hoisting a streamed-side join
conjunct in outer/anti joins, or relocating expressions out of short-circuited
positions).
Previously, `canEvaluateUnconditionallyInternal` conservatively excluded all
`Cast` expressions. This pull request adds support for safe, lossless widening
casts by checking:
```scala
case Cast(child, toType, _, _) =>
Cast.canUpCast(child.dataType, toType) &&
canEvaluateUnconditionallyInternal(child)
```
Where `Cast.canUpCast` guarantees that the cast involves no truncation, no
precision loss, and no possible runtime failure (even in ANSI mode).
### Why are the changes needed?
Lossless numeric widening casts (e.g. `Byte -> Short -> Integer -> Long ->
Float -> Double`) and safe type promotions are total functions on their inputs
and cannot fail at runtime.
Conservatively rejecting all `Cast` expressions prevents optimizing queries
where predicates or join conditions contain safe widening casts (for example,
`t1.id.cast(LongType) < 100L` in `StreamedSideJoinCondition` or range
constraint propagation). Allowing upcasts expands the set of safe expressions
that can be hoisted and relocated without introducing any risk of runtime
errors.
### Does this PR introduce _any_ user-facing change?
No. This is an internal optimizer and expression utility improvement.
### How was this patch tested?
Added positive and negative unit test cases in `ExprUtilsSuite`:
- **Positive cases**: Verified that lossless widening casts (`IntegerType ->
LongType` and `LongType -> StringType`) can be evaluated unconditionally.
- **Negative cases**: Verified that narrowing casts (`LongType ->
IntegerType`) and parsing casts (`StringType -> IntegerType`) that can throw in
ANSI mode remain excluded.
- Passed `build/sbt "catalyst/testOnly *ExprUtilsSuite"` and
`catalyst/scalastyle`.
### Was this patch authored or co-authored using generative AI tooling?
Yes, Generated-by: github-copilot
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
