haizhou-zhao opened a new pull request, #4289: URL: https://github.com/apache/iceberg-python/pull/4289
Closes #4288 # Rationale for this change Avro fields are resolved by ID, so a field that exists only in the file should be read and dropped. Primitives already work this way. A file-only map, list, or struct instead raised `ResolveError: File/read schema are not aligned for map, got None`, because `SchemaPartnerAccessor` raised whenever the partner was `None`. With this change, the accessor returns `None` for the children of a field that has no partner, and the existing `None` handling in `ReadSchemaResolver` reads and drops them. A real type mismatch, such as a map in the file and a string in the read schema, still raises. ## Are these changes tested? Yes. `test_read_manifest_entry_with_unknown_nested_fields` writes a v2 manifest entry whose `data_file` has an unknown map, list, and struct (with a nested map) placed before known fields. It then checks that the entry reads back unchanged. The test fails on `main` with the `ResolveError` above. ## Are there any user-facing changes? Yes, a bug fix: PyIceberg can read manifests and other Avro files that contain nested fields missing from the read schema. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
