andygrove opened a new pull request, #5081: URL: https://github.com/apache/datafusion-comet/pull/5081
## Which issue does this PR close? Closes #5067. ## Rationale for this change Under ANSI mode, `Remainder` on Float/Double operands returned `NaN` for a zero divisor instead of throwing, because Comet's native modulo implementation relies on Arrow's `rem` kernel to signal the error. `arrow-arith` only produces `DivideByZero` for integer and decimal types; for floats `Op::Rem` uses `mod_wrapping`, which returns `NaN` for `x % 0.0` and never errors, so the existing error branch was unreachable for Float32/Float64. Spark 4.1 raises `REMAINDER_BY_ZERO` (and earlier Spark versions raise `DIVIDE_BY_ZERO`) for all numeric types including floating point, so this was a native correctness gap. ## What changes are included in this PR? - `spark_modulo` now detects zero divisors explicitly for `Float32`/`Float64` before delegating to Arrow's `rem` kernel, but only under ANSI mode. - The check matches Spark's row-wise semantics: rows where either operand is null are ignored (they produce null results and must not raise). - `-0.0` triggers the error too — Spark treats it as zero, which falls out naturally from IEEE 754 equality. - Added Rust unit tests for Float32/Float64 in ANSI and non-ANSI modes, including null-lhs, negative-zero divisor, and mixed-null-with-zero-divisor rows. - Added SQL-file coverage under `arithmetic_ansi.sql` for float/double `% 0` — both column-vs-column and literal cases, including `-0.0`. ## How are these changes tested? - `cargo test -p datafusion-comet-spark-expr modulo_expr` — all 11 tests pass, including the 7 new float coverage tests. - `cargo clippy -p datafusion-comet-spark-expr --tests` — clean. - The existing `remainder function` Scala test in `CometExpressionSuite` already exercises `a_float % b_float` with a zero divisor in ANSI mode across Spark versions and will pick up the fix on Spark 4.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
