lasdf1234 opened a new issue, #14535: URL: https://github.com/apache/gravitino/issues/14535
### Version main branch / 1.3.x (Iceberg 1.11.0), lakehouse-iceberg catalog. ### Describe what's wrong Column tags set in Gravitino follow the column **name**, not the Iceberg **field id**. After a schema change made through the Iceberg REST catalog (Spark, pyiceberg, or raw commit): - **Rename**: column `a` (field id 2) tagged `t` is renamed to `a_new`. Gravitino then shows `a_new` with no tags; tag `t` lists no objects. - **Drop and re-add with the same name**: tagged column `c` (id 4) is dropped; a new unrelated `c` (id 5) is added. Tag `t` appears on the new column. - **Name swap**: tags stick to the wrong field after two columns swap names. Renames through the Gravitino table API keep tags. Renames that reach the table only through IRC do not. On load, `TableOperationDispatcher.updateColumnsIfNecessary` matches stored `ColumnEntity` rows to catalog columns **by name**. `ConvertUtil.fromNestedField` does not carry the Iceberg field id, so a rename looks like a drop plus an add. ### Error message and/or stacktrace No exception. Governance silently disappears or attaches to the wrong column after IRC schema evolution. ### How to reproduce 1. Create a table through IRC; tag column `a` via the Gravitino tag API. 2. Rename `a` to `a_new` through IRC (`ALTER TABLE ... RENAME COLUMN` or a commit). 3. Load tags for `a_new`: empty; tag association for the old column is gone. 4. Drop a tagged column and add a new column with the same name: the new column inherits the old tags. ### Additional context Iceberg identifies columns by field id across renames. Expected: tags follow the Iceberg field across IRC renames; a drop clears tags; a new column with a reused name does not inherit them. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
