dahai1996 opened a new issue, #2749: URL: https://github.com/apache/iceberg-rust/issues/2749
### Apache Iceberg Rust version None ### Describe the bug When using `iceberg-catalog-glue`, every call to `update_table` pushes ALL historical schema versions' fields to AWS Glue's column definition. This causes Glue Console to display duplicate columns — one entry per schema version for every field that existed in any past schema. ### To Reproduce 1. Create an Iceberg table via the Glue catalog 2. Insert data with new fields, triggering schema evolution (AddSchema + SetCurrentSchema) 3. Observe the table in AWS Glue Console → Table → Schema ### Expected behavior Only the current schema's fields should be pushed to Glue's column definition. Historical schema versions are already preserved in the Iceberg metadata file on S3. https://github.com/apache/iceberg-rust/blob/main/crates/catalog/glue/src/schema.rs#L57-L63 ```rust for schema in metadata.schemas_iter() { if schema.schema_id() == current_schema.schema_id() { continue; } visit_schema(schema, &mut builder)?; } ``` The from_iceberg function visits the current schema (marked as current=true), then iterates over all historical schemas and visits them too (marked as current=false). All of these are included in the Glue TableInput.storage_descriptor.columns. Glue's API does not filter by iceberg.field.current, so the UI displays all of them. ### Willingness to contribute None -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
