fm100 opened a new pull request, #2791: URL: https://github.com/apache/iceberg-rust/pull/2791
## Which issue does this PR close? - Closes #. ## What changes are included in this PR? This PR makes manifest metadata parsing more tolerant of manifests written by affected versions of the DuckDB Iceberg extension. According to the Iceberg spec, the Avro manifest metadata `schema` key should contain the table schema at the time the manifest was written. A DuckDB Iceberg extension bug wrote the manifest entry schema into that key instead of the table schema. DuckDB fixed this in duckdb/duckdb-iceberg#1020, but that fix has not been released yet. Even after a fixed DuckDB Iceberg extension release is available, tables created by the buggy versions can still contain manifests with the bad metadata. Reopening those tables with a newer writer does not automatically rewrite existing manifest files, so readers need to tolerate the malformed metadata to read those existing tables. For unpartitioned manifests, Iceberg Rust can still read the manifest entries without the table schema. This PR falls back to an empty schema when: - the manifest metadata `schema` value cannot be parsed as an Iceberg table schema - the manifest partition spec is empty The fallback remains limited to unpartitioned manifests because partitioned manifests need the table schema to bind partition source fields. ## Are these changes tested? Yes. This PR adds a unit test covering manifest metadata with a DuckDB-style manifest entry schema for an unpartitioned table. Tested with: ```bash cargo test -p iceberg test_manifest_metadata_with_manifest_entry_schema_for_unpartitioned_table ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
