andygrove commented on issue #9183: URL: https://github.com/apache/arrow-rs/issues/9183#issuecomment-6723875831
To make this easier to split into small PRs, as suggested above, here is a proposed checklist. The phases and order are a suggestion, and I'm happy to change them. If this looks right, it could move into the issue description so that #9183 becomes the tracking issue. The main dividing line is the public API. Phases 1 to 3 change no public signatures, because `Tz` keeps implementing chrono's `TimeZone` trait. Dropping chrono itself is a breaking change, since `NaiveDateTime`, `NaiveDate`, `NaiveTime` and `DateTime<Tz>` appear in public signatures in `temporal_conversions`, `types`, `PrimitiveArray`, and `parquet-variant`. **Decisions needed** - [ ] Years outside jiff's range. jiff covers -9999 to 9999, while chrono reaches about +/-262,000, and Arrow timestamps and dates can go beyond jiff's range. #11482 clamps the offset out of range, but kernels moved onto jiff's civil types would need a defined behavior: return null or error, or keep doing the calendar arithmetic outside jiff. - [ ] Leap seconds. chrono can represent `23:59:60`, while jiff clips it to `23:59:59`. See the discussion above and #5335. - [ ] Which jiff release to depend on before removing chrono from the public API, given that 1.0 is expected but not yet released. - [ ] Bundled tzdb only, or host zoneinfo as a later opt-in. **Phase 1: opt-in `jiff` engine behind `Tz`** (#11482) - [ ] Rebase on `GapInfo` (#11535) and make it work with the jiff engine - [ ] Run `arrow-cast`, `arrow-json`, and `arrow-csv` tests with `arrow-array/jiff` in CI - [ ] Add a test comparing the chrono-tz and jiff engines, reporting both tzdb releases on failure - [ ] Restore the short `Debug` output for `Tz` and `TzOffset` - [ ] Decide on the size of `Tz` and add a compile-time assertion **Phase 2: performance parity** The bar from earlier in this issue is the same answers with equal or better performance. - [ ] Remove the post-2038 slowdown in DST zones, for example with a lazily built transition table per zone (prototype in the #11482 review), covering both the UTC and local paths - [ ] Rerun `arrow/benches/named_timezone.rs` on an idle machine, or report instruction counts - [ ] Fixed-offset fast path for kernels (#10829) **Phase 3: move internals to jiff, one crate per PR** Each PR keeps public signatures and existing results unchanged, apart from documented post-2099 and tzdb differences. - [ ] `arrow-arith` (`date_part`, timestamp and interval arithmetic) - [ ] `arrow-cast` (casts, parsing, display) - [ ] `arrow-csv` - [ ] `arrow-json` - [ ] `parquet` - [ ] `parquet-variant`, `parquet-variant-json`, `parquet-variant-compute` **Phase 4: make jiff the default** - [ ] Downstream projects have had a chance to test the `jiff` feature (DataFusion: apache/datafusion#23568, Comet: apache/datafusion-comet#4754) - [ ] Switch the default in a major release, with a release note covering post-2099 results and tzdb differences. This stays a separate feature name, so the behavior change is visible. **Phase 5: remove chrono (breaking)** - [ ] Replace chrono types in the public API, or move those helpers behind a compatibility feature for one release cycle - [ ] Remove `impl TimeZone for Tz`, `GapInfo`'s chrono types, and the `chrono-tz` feature - [ ] Remove `chrono` from the workspace -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
