andygrove opened a new pull request, #11482:
URL: https://github.com/apache/arrow-rs/pull/11482
# Which issue does this PR close?
- Part of #9183.
- Motivated by apache/datafusion-comet#6816.
# Rationale for this change
This is a starting point for the migration discussed in #9183, scoped to one
concrete divergence so there is something to review.
chrono-tz ships a precomputed transition table per zone and applies the
zone's standard offset after the last entry, which is around 2099. jiff loads
TZif data and applies the POSIX TZ rule from the file footer to any later
instant, which is what `java.time` and libc do. So for `America/Los_Angeles`,
local `2300-07-01 12:00:00` is PDT (UTC-7) in Spark and in jiff, but PST
(UTC-8) through `arrow_array::timezone::Tz` today. apache/datafusion-comet#6816
has the Spark-visible symptoms.
Rather than replacing chrono outright, this adds an opt-in `jiff` feature
that swaps only the engine behind `Tz`. The public API is unchanged: `Tz` still
implements chrono's `TimeZone` trait, is still `Copy`, and has the same
`Display` output and error text. That follows the approach suggested on #9183
of adding jiff alongside chrono-tz, letting downstream projects opt in, and
dropping chrono-tz later.
# What changes are included in this PR?
- `arrow-array` gains a `jiff` feature with an optional `jiff` 0.2
dependency (`std` + `tzdb-bundle-always`, so the database is compiled in as
chrono-tz's is). `arrow` forwards it. When both `chrono-tz` and `jiff` are
enabled, `jiff` wins.
- A third `private` module in `arrow-array/src/timezone.rs` implements `Tz`
and `TzOffset` over `jiff::tz::TimeZone`:
- `Tz` is `Copy` and jiff's `TimeZone` is reference counted, so each
distinct name is resolved once and leaked into a small interning cache. The set
of names is bounded by the bundled database, and offset lookups take no lock.
- UTC to offset goes through `TimeZone::to_offset`. Local to offset goes
through `to_ambiguous_timestamp`: a gap maps to `LocalResult::None` and a fold
to `LocalResult::Ambiguous(before, after)`, the same order chrono-tz uses.
- `offset_from_local_date` mirrors chrono-tz's policy of returning any
offset that was in force during the day.
- jiff covers years -9999 to 9999. Instants outside that take the offset
in force at the nearest covered instant instead of failing.
- Existing tests gated on `chrono-tz` are widened to `any(chrono-tz, jiff)`.
Docs on `DataType::Timestamp`, the parser, and the `arrow` README mention the
feature. CI gets a test run with `chrono-tz,jiff` and a no-default-features
check with `jiff`.
# Are these changes tested?
- New unit tests in the jiff module cover DST transitions from UTC and from
local time, the gap and fold cases, display, invalid names, and daylight saving
in 2100, 2300, and 9999.
- The existing `arrow` integration tests that require `chrono-tz`
(`arithmetic`, `timezone`, `array_cast`, `csv`) pass with `--features
chrono-tz,jiff`, so they exercise the jiff path unchanged.
- `chrono-tz` only and no-feature builds are unchanged and still pass.
# Are there any user-facing changes?
A new opt-in feature. Nothing changes unless it is enabled.
Open questions I would like input on before taking this out of draft:
1. Bundled tzdb versus the host's zoneinfo. This bundles always for
determinism, matching chrono-tz. jiff's default on Unix prefers the system
files, which stay current with OS updates.
2. The leaked interning cache exists only to keep `Tz: Copy`. A future major
release could hold the `TimeZone` directly and drop `Copy`.
3. Behavior beyond year 9999. Clamping is simple but silently drops DST
there. The alternative is an error.
4. Whether jiff should eventually become the implementation behind the
existing `chrono-tz` feature name so downstream crates do not have to change
anything.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
