viirya opened a new issue, #3664: URL: https://github.com/apache/iceberg-rust/issues/3664
### What's the issue? For tables migrated from Hive (or with files registered through `add_files`), readers resolve columns in Parquet files that have no field ids through the `schema.name-mapping.default` table property. Iceberg Java keeps that mapping in step with the schema: `SchemaUpdate.commit` calls `MappingUtil.update`, which adds an entry for every added column (including nested fields, under their parent's entry) and adds the new name of a renamed column as an alias. `UpdateSchemaAction` in iceberg-rust emits only `AddSchema` and `SetCurrentSchema`, and nothing else in the crate updates the property when the schema changes. After adding a column through iceberg-rust, the mapping has no entry for it. Files written later by an engine that does not write field ids (or files added later through `add_files`) then read the new column as NULL, while Java resolves it by name. ### Expected behavior When `UpdateSchemaAction` commits on a table that has `schema.name-mapping.default`, the same commit updates the mapping as Java does: each added column gets an entry under its parent, and an existing entry in the same struct that held that name loses it (a column dropped and re-added under the same name). Deleted columns keep their entries. If the property is absent, nothing changes; if it can't be parsed, Java logs a warning and commits the schema change anyway. Rename support (#2562) would need the same alias handling when it lands. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
