mbutrovich opened a new pull request, #3631: URL: https://github.com/apache/iceberg-rust/pull/3631
## Which issue does this PR close? - Part of #3262 (Task 3). ## What changes are included in this PR? Adds `iceberg::arrow::read_manifest_list`, which decodes a manifest list into a `RecordBatch` with `arrow-avro`, following `read_manifest_entries` from #3388. The binding follows in a separate PR. - The schema is the spec's [`manifest_file`](https://github.com/apache/iceberg/blob/5e7169168db3d34e29354c6f59ec4d6e420b8d2d/format/spec.md?plain=1#L1008-L1037) with its field IDs. The file and row counts are nullable, because v1 lists may write them as null, which the spec reads as non-zero. - Fields are matched by field ID through the projection from #3388, so V1 file counts read under either name (#3445). - The spec gives manifest lists no `format-version` metadata, and v2+ requires `content`, `sequence_number`, and `min_sequence_number`. The reader treats a list without them as v1 and reads them as 0, as Java's [`GenericManifestFile`](https://github.com/apache/iceberg/blob/5e7169168db3d34e29354c6f59ec4d6e420b8d2d/core/src/main/java/org/apache/iceberg/GenericManifestFile.java#L342-L350) does. It takes no format version argument. - The field IDs the readers use are now constants next to the spec fields they belong to in `spec/manifest/entry.rs` and `spec/manifest_list/_const_schema.rs`, which addresses https://github.com/apache/iceberg-rust/pull/3388#discussion_r4470625854. ## Are these changes tested? Yes, with unit tests in `arrow/manifest.rs` and a doctest. `test_manifest_lists_written_by_java` compares the result with `ManifestList::parse_with_version` on the two Java-written fixtures in `testdata/manifests_lists`. ## AI Disclosure I wrote this PR with help from Claude but understand and support the core approach, implementation, and test coverage. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
