moomindani opened a new issue, #2801:
URL: https://github.com/apache/iceberg-rust/issues/2801
### Apache Iceberg Rust version
0.6.0 (latest version)
### Describe the bug
The V3 spec serializes partition fields and sort fields whose transform
takes multiple arguments with a `source-ids` list instead of the single
`source-id`. `PartitionField` and `SortField` only know `source-id`, so
deserializing table metadata that contains any multi-argument transform field
fails with `missing field source-id` — the whole v3 table becomes unreadable.
Per the spec, all v3 readers are required to read tables with unknown
transforms, ignoring them.
For comparison, PyIceberg implemented this read tolerance in
apache/iceberg-python#3630.
### To Reproduce
```rust
use iceberg::spec::PartitionField;
let field: Result<PartitionField, _> = serde_json::from_str(
r#"{"source-ids": [1, 2], "field-id": 1000, "name": "m", "transform":
"bucket[4]"}"#,
);
// Err: missing field `source-id` — expected the field to load with
Transform::Unknown
```
Loading v3 table metadata containing such a partition field or sort field
fails the same way.
### Expected behavior
Partition and sort fields with `source-ids` deserialize: a single-element
list normalizes onto `source-id`; a multi-element list keeps the ids and treats
the transform as `Transform::Unknown`. Serialization round-trips per the spec's
writer rules (single-argument transforms write only `source-id`, multi-argument
transforms write only `source-ids`).
Part of #2411 (Multi-arg Transforms → metadata read).
### Willingness to contribute
I can contribute a fix for this bug independently
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
