compiler-segfault opened a new pull request, #27203: URL: https://github.com/apache/datafusion/pull/27203
## Which issue does this PR close? - Closes #27187. ## Rationale for this change With the Spark dialect, `SELECT 1 / 2` returns `0`, while Spark returns `0.5`. Spark's `/` casts a non-`DECIMAL` pair to `DOUBLE` (`DivisionTypeCoercion`), and `div` is the integer division; the integer result also makes `SELECT IF(true, 1 / 1, 1 / 0)` an `Int64` where Spark returns a `Double`. Spark also returns NULL for a zero divisor in its default (non-ANSI) mode, where DataFusion's floating point division returns an infinity, so casting alone would still leave `1 / 0` different. ## What changes are included in this PR? `SparkFunctionPlanner` now plans `/` for non-`DECIMAL` numeric operands as a call to a `divide` function that implements Spark's semantics: it casts both sides to `Float64`, returns NULL when the divisor is zero (`-0.0` included) and raises `DivideByZero` under `datafusion.execution.enable_ansi_mode`, mirroring the existing `spark_mod`. `DECIMAL` division keeps the current `/` operator and its precision and scale, and non-numeric operands such as intervals are left alone. The planner hook needs the `sql` feature of `datafusion-expr`, so `datafusion-spark` gains a `sql` feature that enables it (and `datafusion-functions-nested/sql`, which has the same requirement) and the sqllogictest crate enables it; without the feature the hook is not compiled and the crate behaves as before. ## What is the testing strategy for this PR? Unit tests in `datafusion/spark/src/function/math/divide.rs` cover the cast, legacy NULL for `0.0` and `-0.0`, the ANSI error, and a NULL dividend with a zero divisor. `datafusion/sqllogictest/test_files/spark/math/divide.slt` covers integer, float, column and zero-divisor cases plus the ANSI-mode error, and the existing `spark/conditional/if.slt` expectation changes from `Int64` to the `Double` result Spark returns. Verified with `cargo test -p datafusion-spark` (328 passed), `cargo test -p datafusion-sqllogictest --test sqllogictests -- spark` (246 files), `cargo clippy -p datafusion-spark --all-targets --features sql -- -D warnings`, and `cargo fmt`. ## Are there any user-facing changes? Spark-dialect queries now return `DOUBLE` from `/` on non-decimal operands and NULL instead of an infinity for a zero divisor, matching Spark. `IF(true, 1, 1 / 0)` still evaluates both branches, so in ANSI mode it raises where Spark short-circuits; that is existing `if` behaviour rather than something this change introduces. The change also affects the `datafusion-spark` crate's feature set (`sql`), not its default behaviour: without the `sql` feature no planner hook is installed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
