Pastoray opened a new pull request, #27434: URL: https://github.com/apache/datafusion/pull/27434
> [!IMPORTANT] > > **Fork CI setup:** For pull requests from a fork to `main`, enable workflows in your fork's **Actions** tab before you push. > See the [fork CI setup guide](https://datafusion.apache.org/contributor-guide/#pull-requests-from-forks). ## Which issue does this PR close? - Closes #27280. ## Rationale for this change For a query like `CASE WHEN f(x) = 'a' THEN .. ELSE f(x) END`, the expression `f(x)` is currently evaluated twice: once for all rows in the first `WHEN`, and again for the rows that fall through to the `ELSE`. When `f` is expensive, the second evaluation can dominate the cost of the whole `CASE`. This PR lets such a `CASE` evaluate the shared operand only once. ## What changes are included in this PR? - Adds `CaseExpr::with_else_reusing_first_when_operand(&[usize])`, an opt-in that takes the child-index path from the first `WHEN` expression to the operand that the `ELSE` repeats. The operand is evaluated once for all rows, and its values are filtered and cast (using the same `TRY_CAST` semantics as the normal `ELSE` path) for the rows that reach the `ELSE`. - Adds the getter `CaseExpr::else_reusing_first_when_operand()`. - Validates the request and returns an error if the `ELSE` is not equal to the expression at the given path, if the `CASE` has no `ELSE` or has a base expression (`CASE expr WHEN ...`), or if the operand is volatile (e.g. `random()`). - Works with both the single-`WHEN` (`ExpressionOrExpression`) and multi-`WHEN` (`NoExpression`) evaluation paths, including the projected-batch optimization. - The option is preserved by `with_new_children` and `project` when it still holds, and dropped otherwise. It is not part of the proto form, so a decoded expression evaluates its `ELSE` separately, which doesn't change results. - DataFusion cannot verify evaluation order, so the caller must guarantee that the first `WHEN` evaluates the operand for every row before anything else observable. This is documented on the method. Nothing in the physical planner calls the new method yet, so SQL behavior is unchanged by this PR. Detecting the pattern in the planner can be a follow-up. ## What is the testing strategy for this PR? New unit tests in `datafusion/physical-expr/src/expressions/case.rs`, mainly the `else_reuse*` tests: - An instrumented expression counts evaluations and rows, checking that the operand is evaluated once instead of twice for one and several `WHEN`s, with and without projection. - Edge cases: all rows match the `WHEN`, no rows match, scalar operand, nested and empty operand paths, and casting to a wider return type. - A randomized test compares results with and without reuse across several row counts and null rates. - Validation errors (mismatched `ELSE`, bad path, missing `ELSE`, base expression, volatile operand), equality, and `with_new_children` behavior. I ran the full `datafusion-physical-expr` tests and doctests, and the `case` tests again with `--features proto`. I did not run sqllogictests, since SQL behavior does not change. ## Are there any user-facing changes? There is a new public method and getter on `CaseExpr`, both additive and opt-in. There are no breaking changes and the default behavior is unchanged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
