andygrove opened a new issue, #7022: URL: https://github.com/apache/datafusion-comet/issues/7022
### What is the problem the feature request solves? Follow-up from #5050. Four cast kernels in `native/spark-expr/src/conversion_funcs/numeric.rs` compute `10_i128.pow(scale as u32)`. A negative scale, which only `spark.sql.legacy.allowNegativeScaleOfDecimal=true` allows, becomes an exponent near 4.29e9: | Kernel | Cast | Debug build | Release build (`overflow-checks = false`) | | --- | --- | --- | --- | | [`cast_decimal_to_int16_down!`](https://github.com/apache/datafusion-comet/blob/66c63bd307b1b11fb684e6b89e9bb220f444de7c/native/spark-expr/src/conversion_funcs/numeric.rs#L548) | decimal to byte or short | panics with `attempt to multiply with overflow` | the power wraps to 0, then the division panics with `attempt to divide by zero` | | [`cast_decimal_to_int32_up!`](https://github.com/apache/datafusion-comet/blob/66c63bd307b1b11fb684e6b89e9bb220f444de7c/native/spark-expr/src/conversion_funcs/numeric.rs#L625) | decimal to int or long | same | same | | [`cast_int_to_decimal128_internal`](https://github.com/apache/datafusion-comet/blob/66c63bd307b1b11fb684e6b89e9bb220f444de7c/native/spark-expr/src/conversion_funcs/numeric.rs#L815) | integer to decimal | same | the multiplier wraps to 0, so every value casts to 0 | | [`cast_decimal_to_timestamp`](https://github.com/apache/datafusion-comet/blob/66c63bd307b1b11fb684e6b89e9bb220f444de7c/native/spark-expr/src/conversion_funcs/numeric.rs#L1385) | decimal to timestamp | same | the power wraps to 0 and the division divides by zero | These are the kernels behind #5013 and #7002. #5050 fixes those in the serde by routing the casts through the JVM codegen dispatcher, so after it lands no current serde path reaches these kernels with a negative scale. Nothing in the kernels rejects one, though. A new path to them, such as another serde or a cast optimization that bypasses `CometCast`, would panic in debug builds and return zeros or divide by zero in release builds. #5050 already closes the same gap in `decimal_div`, `create_binary_expr_with_options` and `create_modulo_expr`. ### Describe the potential solution Check the scale before computing the power. The simplest fix rejects a negative scale with an error, as #5050 does in `decimal_div`, so any future path fails with a clear message. Supporting these casts natively would instead mean multiplying by `10^|scale|` where the code divides today, plus Spark's HALF_UP rounding when an integer cast to a negative scale drops digits. ### Additional context #6953 and #6969 both change these kernels, so this fits best after them or alongside. Line references are pinned to 66c63bd307. The release-build behavior is plain `i128` arithmetic: `10_i128.pow(4294967295)` is 0 with overflow checks off. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
