viirya opened a new issue, #6842:
URL: https://github.com/apache/datafusion-comet/issues/6842
### Describe the bug
Comet's native `CAST(string AS DATE)` returns `NULL` for any year outside
chrono's `NaiveDate` range (-262143 to 262142). Spark parses these strings to
valid dates. Spark's `stringToDate` accepts a year of up to 7 digits and only
rejects a date whose epoch day overflows `Int`, so it accepts years up to about
+/-5.88 million. This happens with `CAST`, `TRY_CAST` and `to_date`, and in
every eval mode, including ANSI, where Comet returns `NULL` without raising an
error.
### Steps to reproduce
```sql
CREATE TABLE t(s string) USING parquet;
INSERT INTO t VALUES ('999999-01-01'), ('294248-01-01'), ('-290309-01-01'),
('1000000-01-01');
SELECT s, unix_date(try_cast(s AS DATE)) FROM t;
```
Spark returns `364522607`, `106752347`, `-106752712` and `364522972`. Comet
returns `NULL` for all four rows. Collected `java.sql.Date` values display
truncated for years past 9999, so `unix_date` makes the difference visible.
### Expected behavior
Comet should return the same epoch day as Spark for any date whose epoch day
fits in `Int` (`-5877641-06-23` to `5881580-07-11`). Outside that range, and
for invalid calendar dates, Comet should return `NULL` in LEGACY/TRY mode and
raise `CAST_INVALID_INPUT` in ANSI mode.
### Additional context
`date_parser` already computes the epoch day without chrono via
`ymd_to_epoch_day`, but then applies an extra chrono year bound.
String-to-timestamp is not affected: Spark limits timestamp years to 6 digits,
and Comet's timestamp parsers already match Spark at the `Long` micros
boundaries. Native `make_date` had the same limitation and was fixed in #5443
(#5208).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
