mbutrovich opened a new issue, #7695: URL: https://github.com/apache/datafusion-comet/issues/7695
### What is the problem the feature request solves? `try_add`, `try_subtract`, and `try_multiply` on `Int8` through `Int64` run [`checked_binary`](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/checked_arithmetic.rs#L229-L280). It calls a checked op per row and pushes the index of each overflowing row to a `Vec`. That branch per row keeps the loop from vectorizing. The kernel also expands a scalar operand to a full array ([L347-L357](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/checked_arithmetic.rs#L347-L357)), and it zeroes the values under nulls on every batch that has nulls ([L269-L277](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/src/math_funcs/checked_arithmetic.rs#L269-L277)). The ANSI path calls arrow's checked kernels. Try mode can't, because those kernels fail the whole batch at the first overflow, and Spark's try mode makes only the overflowing row null. ### Describe the potential solution - One pass computes the wrapping result and ORs each row's overflow into a flag. Add and sub detect overflow from sign bits, because the signed `overflowing_add` and `overflowing_sub` don't vectorize. `Int8` to `Int32` mul multiplies in the next wider type. `Int64` mul uses `overflowing_mul`. - If no row overflowed, the result's validity is the union of the inputs' validity. - Otherwise a second pass builds a bitmap of the rows that didn't overflow and ANDs it into the validity. In try mode an overflow doesn't fail the query and can occur in any batch, so this pass must not rerun a checked kernel. - A scalar operand stays a scalar. ### Additional context [`benches/checked_arithmetic.rs`](https://github.com/apache/datafusion-comet/blob/d187e6823f9052e7331e8a810b08a1202debb91c/native/spark-expr/benches/checked_arithmetic.rs) has two try-mode cases, both `Int32` `add` without overflow. It needs `Int64`, `mul`, and inputs with one overflow per batch, random overflows, and overflows only under nulls. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
