FreakingMind opened a new pull request, #7634: URL: https://github.com/apache/datafusion-comet/pull/7634
## Which issue does this PR close? Closes #7001. ## Rationale for this change With ANSI on, `cast(d AS timestamp)` for a NaN or infinite double or float fails in both engines, but with different errors. Spark throws `SparkDateTimeException` saying `The value Infinity of the type "DOUBLE" cannot be cast to "TIMESTAMP"`. Comet threw `SparkNumberFormatException` saying `The value 'inf' of the type "STRING"`. The error class and SQLSTATE already matched, only the exception class and the text were off. Spark does this the same way in 3.5 through 4.2. `doubleToTimestampAnsi` calls the `Double` overload of `invalidInputInCastToDatetimeError`. A float is widened first, so the message says DOUBLE for float input too. ## What changes are included in this PR? - The native cast now returns `InvalidInputInCastToDatetime` for NaN and infinities, with the value spelled the way Spark spells it (`NaN`, `Infinity`, `-Infinity`). - Both error converter shims have a new case for a DOUBLE source that calls Spark's `Double` overload. The string case is unchanged. - I renamed `float_timestamp_overflow_text` to `double_sql_text`, since the overflow error and the new error both use it now. ## How are these changes tested? - A new SQL test, `cast_float_to_timestamp_ansi.sql`, compares the exception class and the message with Spark for double and float input. It fails on main with `SparkNumberFormatException did not equal SparkDateTimeException`. - New cases in `SparkErrorConverterSuite` for a DOUBLE and a STRING source. - An updated Rust unit test. I ran the SQL test and the converter suite on Spark 4.1 and 3.5. The existing `CometNativeCastSuite` float and double tests didn't catch this, because their generated data hits `CAST_OVERFLOW` on `MaxValue` before it reaches NaN. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
