sesteves opened a new issue, #3637:
URL: https://github.com/apache/iceberg-rust/issues/3637
### Apache Iceberg Rust version
main (`5994664`, 2026-11-06)
### Describe the bug
**1. `year` and `month` of dates and timestamps beyond chrono's range.**
The spec defines `year` and `month` for every `date` and `timestamp` value.
Iceberg Java's `DateTimeUtil` computes them through `java.time`, whose range
covers every `int` day and `long` microsecond. iceberg-rust goes through
chrono, whose dates end at about ±262,000 years, so:
- The array transforms (`Year::transform`, `Month::transform`, via Arrow's
`date_part`) return **NULL for non-null inputs** outside that range. A writer
that uses them files such rows under a `null` partition. A predicate on the
source column then projects to a non-null partition value and prunes the file
that holds the row.
- `transform_literal` returns `DataInvalid` (`Fail to convert timestamp to
date in year transform`) for such timestamps, and **panics** for such dates
(`Date32Type::to_naive_date_opt(*v).unwrap()`, `transform/temporal.rs:80` and
`:173`).
| Input | Expected year | Expected month | Array transform |
`transform_literal` |
|---|---|---|---|---|
| timestamp `i64::MAX` µs (294247-01-10) | 292277 | 3507324 | `null` |
`DataInvalid` |
| timestamp `i64::MIN` µs (-290308-12-22) | -292278 | -3507325 | `null` |
not tested |
| date `i32::MAX` (5881580-07-11) | 5879610 | 70555326 | `null` | panic |
| date `i32::MIN` (-5877641-06-24) | -5879611 | -70555327 | `null` | not
tested |
The expected values use floor division to days and an exact civil-from-days
conversion. `i64::MAX` microseconds is a common "end of time" sentinel.
**2. `Transform::adjust_boundary` overflows at type extremes.**
`adjust_boundary` (`spec/transform.rs:775` onwards) computes `v - 1` for `<`
and `v + 1` for `>` unchecked. With overflow checks (debug builds, tests) `x <
i32::MIN` (int or date) and `x > i64::MAX` panic (`spec/transform.rs:778`).
Without them the boundary wraps to one that matches every value, which only
makes the projection over-inclusive.
For context: the related day floor, nanosecond bucket, decimal truncate
literal, truncate overflow and nanosecond hour issues are already fixed on main
(#3323, #3243, #3573, #3026), and `LargeBinary` truncate has an open PR (#3120).
### To Reproduce
Append to `crates/iceberg/src/spec/transform.rs` and run `cargo test -p
iceberg --lib audit_out_of_range -- --nocapture`:
```rust
#[cfg(test)]
mod audit_out_of_range {
use std::panic::{AssertUnwindSafe, catch_unwind};
use std::sync::Arc;
use arrow_array::{ArrayRef, Date32Array, TimestampMicrosecondArray};
use super::*;
#[test]
fn audit_out_of_range() {
for t in [Transform::Year, Transform::Month] {
let f = create_transform_function(&t).unwrap();
let ts: ArrayRef =
Arc::new(TimestampMicrosecondArray::from(vec![i64::MAX, i64::MIN, 0]));
println!("{t:?} timestamp array [MAX, MIN, 0]: {:?}",
f.transform(ts));
let d: ArrayRef = Arc::new(Date32Array::from(vec![i32::MAX,
i32::MIN, 0]));
println!("{t:?} date array [MAX, MIN, 0]: {:?}", f.transform(d));
println!(
"{t:?} timestamp literal MAX: {:?}",
f.transform_literal(&Datum::timestamp_micros(i64::MAX))
);
let p = catch_unwind(AssertUnwindSafe(||
f.transform_literal(&Datum::date(i32::MAX))));
println!("{t:?} date literal MAX panicked: {}", p.is_err());
}
for (op, d) in [
(PredicateOperator::LessThan, Datum::int(i32::MIN)),
(PredicateOperator::GreaterThan, Datum::long(i64::MAX)),
(PredicateOperator::LessThan, Datum::date(i32::MIN)),
] {
let label = format!("{op:?} {d:?}");
let p = catch_unwind(AssertUnwindSafe(||
Transform::adjust_boundary(&op, &d)));
println!("adjust_boundary {label} panicked: {}", p.is_err());
}
}
}
```
Output: both array transforms print `[null, null, 0]` for timestamps and for
dates; the timestamp literals print `Err(DataInvalid ...)`; the date literals
and all three `adjust_boundary` calls panic.
### Expected behavior
1. `year` and `month` return the spec value for every input, for example
with floor division to days and an exact civil-from-days conversion instead of
chrono, in both the array transforms and `transform_literal`. Failing that,
both paths should return an error, never NULL for a non-null input and never a
panic.
2. `adjust_boundary` uses checked arithmetic. `x < MIN` and `x > MAX` are
always false, so the projection can return an always-false predicate (or no
boundary).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
