viirya opened a new pull request, #6844: URL: https://github.com/apache/datafusion-comet/pull/6844
## Which issue does this PR close? Closes #6842. ## Rationale for this change Spark's `stringToDate` accepts a year of up to 7 digits (optionally signed) and only rejects a date when `LocalDate.of` fails or the epoch day overflows `Int`, which happens past `5881580-07-11` or before `-5877641-06-23`. Comet's `date_parser` computed the same epoch day without chrono, but then returned `NULL` in every eval mode for any year outside chrono's `NaiveDate` range (-262143 to 262142). As a result, `CAST`, `TRY_CAST` and `to_date` returned `NULL` for valid dates such as `294248-01-01` or `1000000-01-01`, even in ANSI mode. ## What changes are included in this PR? - Remove the chrono year bound in `date_parser`. Its remaining checks match Spark: an invalid calendar date or an epoch day that overflows `i32` gives `NULL` in LEGACY/TRY mode and `CAST_INVALID_INPUT` in ANSI mode. - Remove the year-range limitation from the cast compatibility guide. ## Known downstream limitations This makes `CAST(string AS DATE)` return dates outside chrono's `NaiveDate` range, which previously came back as `NULL`. Some native consumers already mishandle such dates when they come from Parquet or `make_date`, and this change lets them come from strings too: casting them to `TIMESTAMP` panics (#5456), and several date-field functions and `CAST(date AS STRING)` return `NULL` or fail (#6843). For those queries, a wrong `NULL` from the string cast becomes an error from the downstream expression. ## How are these changes tested? - Rust unit tests in `string.rs` now assert exact epoch days for years past chrono's range, including leap days, a `+` sign and a `T` suffix, and both `i32` boundaries. They also check that one day past either boundary, invalid calendar dates past chrono's range, and 8-digit years are rejected. - New SQL file test `expressions/cast/cast_string_to_date_wide_year.sql`, run with ANSI enabled and disabled. It checks `unix_date` of `cast`, `try_cast` and `to_date`, plus the raw date values, and fails on `main`. The raw-date query leaves out the two `Int`-boundary rows because Spark cannot convert them to `java.sql.Date` when collecting. - `CometNativeCastSuite` "cast StringType to DateType" adds explicit wide-year inputs. Its fuzz filter now drops only 7-digit years near the `Int` limit, for the same `java.sql.Date` reason. This pull request and its description were written by Isaac. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
