andygrove opened a new pull request, #342: URL: https://github.com/apache/datafusion-spark/pull/342
## Which issue does this PR close? - Ported from apache/datafusion#23898, which was opened before the Spark functions moved to this repo. The other fixes in that PR (ANSI zero divisor and `-0.0`) are already on `main` here. ## Rationale for this change `pmod` returns an overflow error at the signed integer boundaries where Spark returns a value. For example `pmod(-1, -2147483648)` on `INT` fails with `Arithmetic overflow: Overflow happened on: -2147483648 + -1`, and Spark returns `2147483647`. Spark computes `(r + n) % n` for a negative remainder `r` with Java arithmetic. `int` and `long` wrap around. `byte` and `short` are promoted to `int`, so they never overflow. ## What changes are included in this PR? The `(r + n)` step in `spark_pmod` now uses `add_wrapping` instead of the checked `add`. `Int8` and `Int16` inputs are widened to `Int32` for that step, to match Java's promotion. The result is narrowed back by the existing cast to the result type. Without the widening, wrapping at 8 or 16 bits would give `pmod(-1, -128)` = `127` instead of Spark's `-1`. ## Are these changes tested? Yes. New `.slt` cases in `math/pmod.slt` cover the `Int8`, `Int16`, `Int32` and `Int64` boundaries for scalars and arrays. They fail on `main` with the overflow error. I checked all of them against Spark 4.2.0 with the PySpark runner in `crates/sqllogictest/scripts`, and they all pass. ## Are there any user-facing changes? `pmod` returns Spark's result instead of an overflow error for these inputs. There are no API changes. This PR was prepared with AI assistance (Claude Code) and reviewed by me. 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
