viirya commented on PR #6844: URL: https://github.com/apache/datafusion-comet/pull/6844#issuecomment-6101008617
+1 on jiff for timezone handling. apache/arrow-rs#11482 fixes a real class of bugs here: chrono-tz stops applying DST rules after its precomputed table, while jiff follows the TZif footer the way `java.time` does, which is what #6816 needs. The year range in this PR is a separate limit, though, and jiff doesn't widen it. jiff's `civil::Date` only covers years -9999..=9999, which is narrower than chrono's ±262142, while Spark `DateType` reaches about ±5.88 million years. So any kernel that converts a Spark date into a library calendar type will hit that limit, whichever library it uses. This PR avoids it because `date_parser` computes the epoch day with integer arithmetic (`ymd_to_epoch_day`) and never goes through chrono, the same approach #5443 took for `make_date`. The downstream consumers (#5456, #6843) have the same two options under either library: compute in integer epoch days, or fall back to Spark for values the library can't represent. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
