ump45nose opened a new pull request, #7255: URL: https://github.com/apache/datafusion-comet/pull/7255
## Which issue does this PR close? Closes #7210. ## Rationale for this change For years outside chrono's `NaiveDate` range, the native string-to-timestamp path looked up the session timezone offset at `1970-01-01`. That returns the zone's standard offset (for example `-08:00` for `America/Los_Angeles`) instead of the local-mean-time offset Spark's `ZoneRules` use for dates before 1883. Casting `-290308-12-21 19:59:05.224192` in an `America/Los_Angeles` session was therefore 422 seconds off from Spark (`-9223372008054775808` vs `-9223372008476775808`). `1800-06-01` in the same session already matched Spark, because that year is still inside chrono's range and chrono-tz applies LMT there. Only the extreme-year surrogate probe was wrong. ## What changes are included in this PR? In `parse_timestamp_to_micros`, the extreme-year branch now looks up the offset at the same civil time shifted toward the epoch by whole 400-year Gregorian cycles until the year fits in chrono's range. Leap days are preserved across the shift. Fixed offsets and UTC are unchanged. Far-future DST after chrono-tz's last transition (~2099) is unchanged and remains #6816. ## How are these changes tested? New unit test `extreme_year_session_tz_uses_historic_offset` in `native/spark-expr/src/conversion_funcs/string.rs`: - `America/Los_Angeles` session cast of `-290308-12-21 19:59:05.224192` matches Spark's LMT micros. - `1800-06-01 00:00:00` in the same zone stays pinned to the known Spark value. - UTC and `+02:00` stay exact at the `i64` microsecond bounds. `cargo test -p datafusion-comet-spark-expr extreme_year`, `cargo test -p datafusion-comet-spark-expr conversion_funcs::string::tests`, `cargo clippy -p datafusion-comet-spark-expr --all-targets -- -D warnings`, and `cargo fmt` pass. This PR was prepared with AI assistance and reviewed and tested by the author. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
