cshuo opened a new pull request, #20736: URL: https://github.com/apache/hudi/pull/20736
### Describe the issue this Pull Request addresses Flink preserves table options when renaming or dropping a column. For a table configured with `hoodie.vector.columns='embedding:3,features:2'`, renaming `embedding` left the old name in the option. The filesystem catalog committed the rename and then failed to refresh the properties with `VECTOR column 'embedding' does not exist in the table schema.` Retrying the statement failed because the column had already been renamed. Dropping `embedding` had the same effect. ### Summary and Changelog #### Commit 1: Update vector column descriptors before committing schema changes (`63a79d7999a`) - Apply column renames and drops to VECTOR descriptors, preserving their order and dimensions. Remove the option when the last VECTOR column is dropped. - Prepare and validate the filesystem catalog properties before committing the schema change, then persist the validated properties. - Read the existing `vector[dimension, elementType, storageBacking]` representation from internal schema history so a subsequent ALTER can use the committed schema. ### Impact Renaming `embedding` to `embedding_v2` now produces `embedding_v2:3,features:2`. Dropping `embedding` produces `features:2`. An invalid descriptor fails validation before an ALTER_SCHEMA instant is committed. ### Risk Level Low. The change touches filesystem catalog ALTER ordering and the common internal schema reader. On Flink 1.20.1, all 14 DFS catalog schema evolution tests and all 6 `TestSerDeHelper` tests passed. The SQL regressions cover rename, consecutive drops, and a stale descriptor leaving the timeline and catalog properties intact. Checkstyle passed for both affected modules. ### Documentation Update none ### Contributor's checklist - [ ] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [x] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
