mattfaltyn opened a new issue, #3025:
URL: https://github.com/apache/iceberg-rust/issues/3025
### Apache Iceberg Rust version
`main` at `7d2dfce494c94e75f726fe52c6c96272b6cc5b32` and `v0.10.1`.
### Describe the bug
The Arrow implementation of the `hour` partition transform rejects
nanosecond timestamp arrays, even though `Transform::Hour::result_type` accepts
`timestamp_ns` and `timestamptz_ns`, and the literal implementation handles
both types.
This reaches production write paths through `PartitionValueCalculator`:
calculating partitions for a table with an `hour(event_time)` partition field
fails when `event_time` is an Arrow `Timestamp(Nanosecond, ...)` column.
The transform currently handles only `TimestampMicrosecondArray` in
`Hour::transform`. The existing `hour_timestamp_nano` helper is already used by
`transform_literal`, so the array behavior is inconsistent with the public type
contract and literal behavior.
### To reproduce
```rust
use std::sync::Arc;
use arrow_array::{RecordBatch, TimestampNanosecondArray};
use iceberg::arrow::{PartitionValueCalculator, schema_to_arrow_schema};
use iceberg::spec::{NestedField, PartitionSpec, PrimitiveType, Schema,
Transform, Type};
let table_schema = Arc::new(
Schema::builder()
.with_fields(vec![NestedField::required(
1,
"event_time",
Type::Primitive(PrimitiveType::TimestampNs),
)
.into()])
.build()?,
);
let partition_spec = PartitionSpec::builder(table_schema.clone())
.add_partition_field("event_time", "event_hour", Transform::Hour)?
.build()?;
let calculator = PartitionValueCalculator::try_new(&partition_spec,
&table_schema)?;
let batch = RecordBatch::try_new(
Arc::new(schema_to_arrow_schema(&table_schema)?),
vec![Arc::new(TimestampNanosecondArray::from(vec![
0,
3_600_000_000_000,
]))],
)?;
calculator.calculate(&batch)?;
```
Actual result:
```text
FeatureUnsupported => Unsupported data type for hour transform:
Timestamp(Nanosecond, None)
```
The equivalent microsecond Arrow array succeeds and produces `[0, 1]`.
### Expected behavior
The nanosecond array should produce hour partition values `[0, 1]`, matching
the equivalent microsecond array and the supported `timestamp_ns` literal
transform.
### Willingness to contribute
I can contribute a focused fix and regression test independently.
### AI disclosure
Codex assisted with repository inspection, duplicate-history searches,
reproduction scaffolding, and drafting this report. I reproduced the failure
against a clean checkout of current upstream and reviewed the relevant
transform and partition-calculation paths.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
