andygrove opened a new issue, #5093: URL: https://github.com/apache/datafusion-comet/issues/5093
### Describe the bug In `native/spark-expr/src/math_funcs/negative.rs`, the ANSI-mode overflow check (`check_overflow!` macro) loops over Int8/16/32/64 and interval arrays comparing every slot's value to `MIN`, then calls `neg_wrapping`. The scan reads `value(i)` without checking validity, so a `MIN` value sitting in a null slot (arbitrary bytes left behind by filter/slice/FFI) raises a spurious ARITHMETIC_OVERFLOW error even though the row is null. It is also a two-pass re-implementation of arrow's checked `neg` kernel, which negates with overflow detection in a single pass and correctly skips null slots. ### Steps to reproduce Construct an Int64 array where a null slot's underlying value is `i64::MIN` (for example by masking a valid `i64::MIN` entry to null via a validity buffer) and evaluate `negative()` on it in ANSI mode. The expression errors; Spark returns null for the null row and succeeds. ### Expected behavior Null slots must be ignored by the overflow check. Replacing the ANSI path with `arrow::compute::kernels::numeric::neg` fixes this, since the kernel skips invalid slots. ### Additional context Notes for the fix: - The Spark error message per type must be reconstructed in a `map_err`; if the exact offending value is required for Int8/Int16 wording, a rescan on the (cold) error path can recover it. - `IntervalUnit::MonthDayNano` is deliberately not overflow-checked today, and arrow's checked `neg` would check it, so that type must keep `neg_wrapping` unconditionally. - Legacy (non-ANSI) mode keeps `neg_wrapping` as today. Found during an audit of native code that replicates existing arrow-rs kernels. The corresponding null-slot behavior is already pinned correctly for add in `checked_arithmetic.rs` tests. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
