andygrove opened a new issue, #5068: URL: https://github.com/apache/datafusion-comet/issues/5068
### Describe the bug The native boolean-to-decimal cast ignores `eval_mode` entirely and throws where Spark returns NULL in legacy and try mode. `CometCast` marks boolean-to-decimal as `Compatible()` in all eval modes (`spark/src/main/scala/org/apache/comet/expressions/CometCast.scala:329-333`). The native implementation `cast_boolean_to_decimal` (`native/spark-expr/src/conversion_funcs/boolean.rs:31-58`) takes no `eval_mode` argument: it builds `10^scale` and unconditionally raises `SparkError::NumericValueOutOfRange` when the value does not fit the target precision. Spark 4.1 (`Cast.scala`): `toPrecision(Decimal.ONE, target, ROUND_HALF_UP, nullOnOverflow = !ansiEnabled)`, so legacy and try mode return NULL, only ANSI throws. Since ANSI is the default in Spark 4.x, Comet accidentally matches the default, but this is a wrong-exception bug on Spark 3.x and whenever ANSI is off, and `try_cast` returns an error instead of NULL. ### Steps to reproduce ```sql SET spark.sql.ansi.enabled=false; SELECT CAST(col AS DECIMAL(1,1)) FROM (SELECT true AS col); ``` Any target where `10^scale` exceeds the precision reproduces it, e.g. `DECIMAL(1,1)`, `DECIMAL(2,2)`, `DECIMAL(3,3)`. Spark (legacy or try mode): `NULL`. Comet: throws `NUMERIC_VALUE_OUT_OF_RANGE`. ### Expected behavior `cast_boolean_to_decimal` should take the eval mode and return NULL for legacy/try, error only under ANSI. ### Additional context Found by an ANSI-mode audit of all native expressions against Spark 4.1.1. Existing `CometCastSuite` coverage only uses non-overflowing targets (`decimal(10,2)`, `(14,4)`, `(30,0)`), which is why this was never caught. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
