andygrove opened a new issue, #305: URL: https://github.com/apache/datafusion-spark/issues/305
### Describe the bug `if` only works when the optimizer rewrites it. `simplify` ([conditional/if.rs#L106](https://github.com/apache/datafusion-spark/blob/21ab12776f55d5646431d81f76b55f6bd0b3ea1e/crates/spark/src/function/conditional/if.rs#L106)) replaces it with a `CASE` expression, and `invoke_with_args` ([conditional/if.rs#L102](https://github.com/apache/datafusion-spark/blob/21ab12776f55d5646431d81f76b55f6bd0b3ea1e/crates/spark/src/function/conditional/if.rs#L102)) always returns an error. Any caller that evaluates the function without running `simplify` gets that error instead of a result. Every function needs a working `invoke_with_args`. `simplify` is an optional optimization on top of it. ### To Reproduce Build a session with the `simplify_expressions` optimizer rule removed, so the expression is not constant-folded and `ScalarUDFImpl::simplify` is never called. This is how engines that build physical expressions directly, such as Comet, run the function. ```rust let rules = Optimizer::new() .rules .into_iter() .filter(|rule| rule.name() != "simplify_expressions") .collect(); let state = SessionStateBuilder::new() .with_default_features() .with_spark_features() .with_optimizer_rules(rules) .build(); ``` ```sql SELECT if(false, 10.5, 20); -- Internal error: if should have been simplified to case. ``` ### Expected behavior `invoke_with_args` computes the same result as the rewritten expression, for scalar and array arguments. `simplify` can stay as an optimization. Note that `CASE` evaluates only the branch it takes, so `SELECT IF(true, 1 / 1, 1 / 0)` returns 1. A scalar UDF receives all of its arguments already evaluated, so an `invoke_with_args` cannot skip the error in the other branch. The issue should decide whether that difference is acceptable or whether `if` has to stay a rewrite (and, if so, how callers that skip `simplify` are meant to handle it). ### Additional context Found by running the `.slt` files with `simplify_expressions` disabled. The new `evaluate` mode of the sqllogictest runner does this, and it lists `conditional/if.slt` as a known failure until this is fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
