KaiqiJinWow opened a new pull request, #3690: URL: https://github.com/apache/iceberg-python/pull/3690
# Rationale for this change V3 manifest schemas are defined, but manifest entries and manifest lists are still read with the V2 projection. Avro schema resolution therefore drops V3-only fields such as `first_row_id`, `referenced_data_file`, `content_offset`, and `content_size_in_bytes` before they reach `DataFile` and `ManifestFile`. Read manifests with the latest supported projection while retaining the V2 layout as the default for constructors and writers. V1 and V2 manifests remain compatible because their missing V3 fields resolve to null. This isolates the read-side concern raised in #3624 from its V3 writer work and is a prerequisite for #3478 to consume deletion-vector content ranges from catalog manifests. ## Are these changes tested? Yes. - Added V3 manifest-entry round trips covering all four V3 data-file fields. - Added a V3 manifest-list round trip covering `first_row_id`. - Extended the existing V1 manifest test to verify the V3 fields resolve to null. - Ran `pytest tests/utils/test_manifest.py`: 42 passed. - Ran the available table test suite: 293 passed; tests requiring unavailable optional `datafusion` and `pyiceberg-core` extras were excluded. - Ran Ruff format and lint checks for the changed files. - Manually validated the read path against an Iceberg 1.10 V3 manifest containing a deletion-vector content range. ## Are there any user-facing changes? Yes. Reading V3 manifests now retains V3-only data-file and manifest-list fields instead of silently discarding them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
