moomindani opened a new pull request, #2802: URL: https://github.com/apache/iceberg-rust/pull/2802
## Which issue does this PR close? - Closes #2801. Part of #2411. ## What changes are included in this PR? The v3 spec serializes partition fields and sort fields whose transform takes multiple arguments with a `source-ids` list instead of the single `source-id`. Deserializing such metadata failed with `missing field source-id`, so a v3 table containing any multi-argument transform could not be read at all. This adds read tolerance, mirroring PyIceberg (apache/iceberg-python#3630): - `PartitionField` and `SortField` accept `source-ids`: a single-element list is normalized onto `source-id`; a multi-element list is kept in the new `source_ids` field and the transform maps to `Transform::Unknown` — multi-argument transforms cannot be evaluated, and the spec requires v3 readers to read such tables ignoring them. `source_id` holds the first id so existing consumers keep working. - Serialization follows the spec's writer rule: single-argument transforms write only `source-id`, multi-argument transforms write only `source-ids`, so the field round-trips. - `PartitionSpec::is_compatible_with` also compares `source_ids`, so two multi-arg fields sharing the first id no longer compare as identical. - `SortField`'s `Display` renders all ids for multi-argument fields. - `PartitionSpec::partition_type` works unchanged: `Transform::Unknown` produces `string` per the spec. Note: this adds a public `source_ids` field to `PartitionField` and `SortField` (will be flagged by cargo-public-api). Known limitation shared with #2790: `Transform::Unknown` does not retain the original transform name, so round-tripping writes `"unknown"`; name preservation stays tracked in #2789. ## Are these changes tested? Yes — new unit tests in `spec/partition.rs` and `spec/sort.rs`: multi-arg deserialization and round-trip, single-element normalization, empty `source-ids` rejection, missing-transform rejection, and `partition_type` over a multi-arg field. `cargo test -p iceberg --lib` (1399 tests), `cargo clippy -p iceberg --lib --tests`, and `cargo check --workspace --tests` pass locally. This pull request and its description were written by Claude Fable 5. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
