andygrove commented on issue #23890:
URL: https://github.com/apache/datafusion/issues/23890#issuecomment-6698732391
Running `next_day.slt` on Spark 4.2.0 with the runner in #27165 found
another divergence that follows the same split. An unparseable start date is
NULL in Spark when ANSI mode is off, but an error in DataFusion even with
`enable_ansi_mode = false`:
```sql
SELECT next_day('invalid_date'::string, 'Mon'::string)
-- Spark 4.2.0 with spark.sql.ansi.enabled=false: NULL
-- DataFusion: Cannot cast string 'invalid_date' to value of Date32 type
```
Spark casts the string to a `DATE` before `next_day` runs. That cast returns
NULL when ANSI mode is off, the default in Spark 3.4 and 3.5, and raises an
error when it is on, the default since 4.0 (from the Spark source, not run).
Like the `day_of_week` case above, `next_day` should follow `enable_ansi_mode`,
and its documentation should say which Spark version the default matches until
#21698 lets it follow the version.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
