dayan02dev opened a new pull request, #12258:
URL: https://github.com/apache/arrow-rs/pull/12258

   # Which issue does this PR close?
   
   - Closes #11853
   
   # Rationale for this change
   
   The `(Date64, Date32)` cast computes `x / MILLISECONDS_IN_DAY`. Integer 
division truncates toward zero, so a Date64 value before 1970 that is not an 
exact midnight lands on the following day. For example, `-1` (displayed as 
`1969-12-31T23:59:59.999`) casts to `1970-01-01`. This happens in both safe and 
`safe: false` mode.
   
   The `Date64` docs say values need not be whole days and are treated like 
`Timestamp(Millisecond, None)`. arrow-rs produces such values itself: 
`cast("1969-12-31T12:00:00", Date64)` keeps the time of day. `Timestamp -> 
Date32` has rounded down since #4211, and `Timestamp -> Date64` since #11632. 
This cast was the remaining one that truncated.
   
   # What changes are included in this PR?
   
   The cast uses `x.div_euclid(MILLISECONDS_IN_DAY)` in both the safe and 
`safe: false` branches, as `timestamp_to_date32` does. In a local 
micro-benchmark this adds about 0.15 ns per value, the same sign fix-up the 
Timestamp path already pays.
   
   # Are these changes tested?
   
   Yes. `test_cast_date64_to_date32` now also casts `-1`, `-86400000` (an exact 
midnight, unchanged) and `-86400001`, and checks that `safe: false` gives the 
same array. Without the fix it fails with `left: -1, right: 0`. Reverting only 
the `safe: false` branch fails the second assertion.
   
   # Are there any significant user-facing changes?
   
   - Pre-1970 Date64 values that are not exact midnights now cast to the day 
that contains them, matching `Timestamp(Millisecond, None) -> Date32`.
   - At the bottom of the Date32 range, `i32::MIN * 86400000 - 1` used to cast 
to `i32::MIN`. That is one day off, because its day is `i32::MIN - 1`, which 
Date32 cannot represent. It is now out of range: null when safe, a `CastError` 
with `safe: false`. The Timestamp cast already behaves this way.
   
   The parquet writer's `coerce_types` path 
(`parquet/src/arrow/arrow_writer/mod.rs`) converts Date64 to INT32 with its own 
truncating division. This PR leaves it alone, so for pre-epoch non-midnight 
values it no longer agrees with `cast`. That could be a separate issue, since 
changing it changes what gets written to files.
   
   # AI usage disclosure
   
   The issue analysis, fix, test, and the benchmark numbers above were produced 
with an AI coding agent (Claude Code). A separate agent session re-checked them 
before submission.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to