simpleqt opened a new pull request, #18742: URL: https://github.com/apache/iceberg/pull/18742
## Problem `IdentityPartitionConverters.convertConstant` materializes constant values (identity partition values, metadata columns) for the generic read path. It handles `TIMESTAMP` by converting stored micros into `LocalDateTime`/`OffsetDateTime`, but the `TIMESTAMP_NANO` case added to the type system is missing from the switch. As a result, for a table with an identity-partitioned `timestamp_ns` or `timestamptz_ns` column, readers that consume the constants map (`GenericReader`, `IcebergInputFormat` via `PartitionUtil.constantsMap`) inject the raw `long` nanosecond value into records where the generic data model expects `LocalDateTime` (without zone) or `OffsetDateTime` (with zone). The parallel converter `GenericDataUtil.internalToGeneric` already performs this conversion for nano timestamps (fixed in #15099); `IdentityPartitionConverters` was missed there. This is currently the only switch over type IDs in `api`/`core` main that handles `TIMESTAMP` without handling `TIMESTAMP_NANO`. Reproduction sketch (the partition column comes back as a `Long` instead of `LocalDateTime`): ```java Table table = ...; // identity-partitioned on a timestamp_ns column, one file table.refresh(); CloseableIterable<Record> rows = PolarizedFilesScanner.read(...); // GenericReader: PartitionUtil.constantsMap(task, IdentityPartitionConverters::convertConstant) // -> record.get(partitionColumn) is Long nanos, expected LocalDateTime ``` ## Fix Mirror the `TIMESTAMP` case for nano precision, using the same `DateTimeUtil` helpers as `GenericDataUtil`: - `TIMESTAMP_NANO` without zone -> `DateTimeUtil.timestampFromNanos(long)` -> `LocalDateTime` - `TIMESTAMP_NANO` with zone -> `DateTimeUtil.timestamptzFromNanos(long)` -> `OffsetDateTime` ## Testing Added `TestIdentityPartitionConverters` covering the nano conversions (`LocalDateTime` / `OffsetDateTime`), controls for the existing `TIMESTAMP` micros, `TIME`, `DATE`, `FIXED`, and `null` handling. The new nano test fails on main (`actual: 1736940320111456789L ... but was instance of: java.lang.Long`) and passes with this change. - `./gradlew :iceberg-core:test --tests "org.apache.iceberg.data.TestIdentityPartitionConverters"` — green - `./gradlew :iceberg-core:test --tests "org.apache.iceberg.data.*"` — green - `./gradlew :iceberg-core:spotlessCheck` — green -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
