jerryshao opened a new issue, #13309: URL: https://github.com/apache/gravitino/issues/13309
### Describe the subtask Tags, owners and privileges attach to a column by its id. Three paths either give existing columns new ids or leave column rows behind, so the column-level metadata is lost: 1. **Cross-schema move plus column change.** If one `alterTable` moves a table to another schema and also changes columns, `TableColumnMetaService.updateColumnPOsFromTableDiff` inserts rows only for the changed columns. It calls `updateSchemaIdByTableId` only when no column changed, so the unchanged columns keep the old `schema_id`. A later cascade drop of the old schema deletes those columns and their relations. 2. **Re-import.** When a table is imported again, e.g. after it was renamed outside Gravitino and then loaded, `TableOperationDispatcher.importTable` gives every column a new id. `insertTable(overwrite = true)` then soft-deletes the old columns, so every column tag is lost. 3. **Lance `VERSION_CHECK`.** When the dataset version changes, `LanceTableOperations.replaceColumnsFromDataset` gives every column a new id, so column tags and comments are lost. Proposed fix: - Update `schema_id` for all of the table's columns whenever the namespace changes, in the same transaction. - On an overwrite that keeps the same table id, reuse the stored column ids by column name. - In the Lance refresh, reuse existing column ids and comments by name. - Add tests that fail without the fix. Target: `main`, backport to `branch-1.3`. ### Parent issue #13303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
