andygrove opened a new issue, #27187: URL: https://github.com/apache/datafusion/issues/27187
### Describe the bug Spark's `/` always divides as floating point: it casts both operands to `DOUBLE` unless the division is already `DOUBLE` or `DECIMAL` ([DivisionTypeCoercion.scala#L29-L33](https://github.com/apache/spark/blob/v4.2.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/DivisionTypeCoercion.scala#L29-L33)), and Spark uses `div` for integer division. With the Spark dialect and functions, DataFusion's `/` on two integers still returns an integer. ### To Reproduce ```sql SELECT IF(true, 1 / 1, 1 / 0) -- Spark 4.2.0: DOUBLE -- DataFusion: integer (if.slt#L150) ``` The same applies to `SELECT 1 / 2`, which is `0.5` in Spark. ### Spark versions The result type is the same in every version from 3.4 to 4.2. The result of a zero divisor is not: it depends on ANSI mode, which became the default in Spark 4.0. `1 / 0` is NULL by default in Spark 3.4 and 3.5, and raises `DIVIDE_BY_ZERO` by default in 4.0 to 4.2. DataFusion's `datafusion.execution.enable_ansi_mode` defaults to false, the 3.x default. ### Expected behavior With the Spark dialect, plan `/` with Spark's semantics: `DOUBLE` division for integers, and `DECIMAL` division for decimals. A zero divisor should give NULL or an error according to `enable_ansi_mode`. Since `datafusion-spark` has no notion of a Spark version (#21698), its documentation should say that the default matches Spark 3.x, until the default can follow the Spark version. ### Additional context Found with the runner in #27165, from the column types in `if.slt`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
