dayan02dev opened a new pull request, #12258:
URL: https://github.com/apache/arrow-rs/pull/12258
# Which issue does this PR close?
- Closes #11853
# Rationale for this change
The `(Date64, Date32)` cast computes `x / MILLISECONDS_IN_DAY`. Integer
division truncates toward zero, so a Date64 value before 1970 that is not an
exact midnight lands on the following day. For example, `-1` (displayed as
`1969-12-31T23:59:59.999`) casts to `1970-01-01`. This happens in both safe and
`safe: false` mode.
The `Date64` docs say values need not be whole days and are treated like
`Timestamp(Millisecond, None)`. arrow-rs produces such values itself:
`cast("1969-12-31T12:00:00", Date64)` keeps the time of day. `Timestamp ->
Date32` has rounded down since #4211, and `Timestamp -> Date64` since #11632.
This cast was the remaining one that truncated.
# What changes are included in this PR?
The cast uses `x.div_euclid(MILLISECONDS_IN_DAY)` in both the safe and
`safe: false` branches, as `timestamp_to_date32` does. In a local
micro-benchmark this adds about 0.15 ns per value, the same sign fix-up the
Timestamp path already pays.
# Are these changes tested?
Yes. `test_cast_date64_to_date32` now also casts `-1`, `-86400000` (an exact
midnight, unchanged) and `-86400001`, and checks that `safe: false` gives the
same array. Without the fix it fails with `left: -1, right: 0`. Reverting only
the `safe: false` branch fails the second assertion.
# Are there any significant user-facing changes?
- Pre-1970 Date64 values that are not exact midnights now cast to the day
that contains them, matching `Timestamp(Millisecond, None) -> Date32`.
- At the bottom of the Date32 range, `i32::MIN * 86400000 - 1` used to cast
to `i32::MIN`. That is one day off, because its day is `i32::MIN - 1`, which
Date32 cannot represent. It is now out of range: null when safe, a `CastError`
with `safe: false`. The Timestamp cast already behaves this way.
The parquet writer's `coerce_types` path
(`parquet/src/arrow/arrow_writer/mod.rs`) converts Date64 to INT32 with its own
truncating division. This PR leaves it alone, so for pre-epoch non-midnight
values it no longer agrees with `cast`. That could be a separate issue, since
changing it changes what gets written to files.
# AI usage disclosure
The issue analysis, fix, test, and the benchmark numbers above were produced
with an AI coding agent (Claude Code). A separate agent session re-checked them
before submission.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]