comphead opened a new issue, #26333: URL: https://github.com/apache/datafusion/issues/26333
### Describe the bug `SparkAtan2` passes the raw arguments to Rust's `f64::atan2` ([atan2.rs#L82](https://github.com/apache/datafusion/blob/9dec77584ea39d93aa01bba7f880b318b37d2c24/datafusion/spark/src/function/math/atan2.rs#L82)). Spark's `Atan2` adds `0.0` to both arguments first, which turns a negative zero into a positive zero ([mathExpressions.scala#L1229-L1237](https://github.com/apache/spark/blob/v4.0.0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala#L1229-L1237)). The Spark source explains why: "With codegen, the values returned by -0.0 and 0.0 are different. Handled with +0.0". So `atan2` returns a different value whenever `y` or `x` is `-0.0`. ### To Reproduce Spark 4.0.0: ```sql SELECT atan2(CAST('-0.0' AS DOUBLE), -1.0D) -- 3.141592653589793 SELECT atan2(0.0D, CAST('-0.0' AS DOUBLE)) -- 0.0 SELECT atan2(CAST('-0.0' AS DOUBLE), CAST('-0.0' AS DOUBLE)) -- 0.0 ``` DataFusion: the existing test at [atan2.slt#L155](https://github.com/apache/datafusion/blob/9dec77584ea39d93aa01bba7f880b318b37d2c24/datafusion/sqllogictest/test_files/spark/math/atan2.slt#L155) documents the C99 result: ``` query R SELECT atan2(-0.0::double, -1.0::double) ---- -3.141592653589793 ``` The other two inputs were not run on DataFusion. By the C99 definition that Rust follows, `atan2(0.0, -0.0)` is `pi` and `atan2(-0.0, -0.0)` is `-pi`. ### Expected behavior Add `0.0` to both arguments before calling `atan2`, as Spark does, and change the expectation at `atan2.slt#L155` to `3.141592653589793`. ### Additional context Found by running the queries in `datafusion/sqllogictest/test_files/spark` on a local Spark and comparing the results with the expected values in the files, which is the kind of check proposed in #17045. Spark results come from Apple's build of Spark 4.0.0 (`4.0.0.65-apple`) with a UTC session and `spark.sql.ansi.enabled=false` unless stated. The Spark source links point at the `v4.0.0` tag. DataFusion results are the sqllogictest expectations at 9dec77584, which CI enforces. Anything marked as not run was derived from reading the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
