lichuang opened a new pull request, #12352: URL: https://github.com/apache/arrow-rs/pull/12352
# Which issue does this PR close? - Closes #11602 # Rationale for this change Casting a `Timestamp` to a coarser unit divides with `/`, which truncates toward zero. For an instant before the Unix epoch that rounds *up* to a later unit interval instead of staying inside the interval that contains it: ```text 1969-12-31T23:59:59.999 -> Timestamp(Second) 1970-01-01T00:00:00 1969-12-31T23:59:59.999 -> Date32 1969-12-31 ``` The `Date32` cast already floors (fixed in #4211), and adding an interval to a timestamp floors too, so the two paths disagreed about which interval an instant belongs to. `Date64` had the same problem when casting to `Date32` or to `Timestamp(Second)`. This matters for date bucketing: a day or second bucket derived from a `Timestamp` could disagree with the same bucket derived from `Date32`, and a timestamp just before the epoch was pushed into the following second. # What changes are included in this PR? - `Timestamp` to a coarser `Timestamp` unit now uses `div_euclid`, so it floors. - `Date64` to `Date32` now uses `div_euclid`, so a value before the epoch belongs to the preceding day. - `Date64` to `Timestamp(Second)` now uses `div_euclid`. - The `cast_with_options` documentation now states that casting a `Timestamp` to a coarser unit floors. Casts to a coarser `Time32` or `Time64` unit are unaffected because times are not negative. `Duration` is also left unchanged in this PR: a duration is an elapsed amount rather than an instant, so flooring is a separate question. # Are these changes tested? Two regression tests were added to `arrow-cast/tests/cast/date_time.rs`: - `test_cast_timestamp_to_coarser_unit_floors` checks the positive and negative boundaries for `Timestamp` to `Timestamp` across nanosecond, microsecond, millisecond and second targets, and checks that the `Date32` cast agrees on the day boundary. - `test_cast_date64_negative_floors` checks `Date64` to `Date32` and `Date64` to `Timestamp(Second)` for values just before and after the epoch, including a whole day, and asserts both agree with the `Timestamp` to `Date32` cast for the same instants. The existing cast tests continue to pass, including those that previously pinned values around the epoch. Clippy and formatting checks pass. # Are there any significant user-facing changes? Yes, one behaviour change: for timestamps before the Unix epoch, casting to a coarser unit now floors instead of truncating toward zero. ```text -1 ms -> -1 s (was 0 s) -1500 ms -> -2 s (was -1 s) ``` This aligns the existing `Date32` and interval arithmetic behaviour, but callers with pre-epoch data will observe different values. Timestamps at or after the epoch are unchanged. # AI usage disclosure This change and its tests were prepared with AI assistance. I reviewed the implementation and verified the tests, Clippy, and formatting locally. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
