cshuo opened a new pull request, #20736:
URL: https://github.com/apache/hudi/pull/20736

   ### Describe the issue this Pull Request addresses
   
   Flink preserves table options when renaming or dropping a column. For a 
table configured with `hoodie.vector.columns='embedding:3,features:2'`, 
renaming `embedding` left the old name in the option. The filesystem catalog 
committed the rename and then failed to refresh the properties with `VECTOR 
column 'embedding' does not exist in the table schema.` Retrying the statement 
failed because the column had already been renamed. Dropping `embedding` had 
the same effect.
   
   ### Summary and Changelog
   
   #### Commit 1: Update vector column descriptors before committing schema 
changes (`63a79d7999a`)
   
   - Apply column renames and drops to VECTOR descriptors, preserving their 
order and dimensions. Remove the option when the last VECTOR column is dropped.
   - Prepare and validate the filesystem catalog properties before committing 
the schema change, then persist the validated properties.
   - Read the existing `vector[dimension, elementType, storageBacking]` 
representation from internal schema history so a subsequent ALTER can use the 
committed schema.
   
   ### Impact
   
   Renaming `embedding` to `embedding_v2` now produces 
`embedding_v2:3,features:2`. Dropping `embedding` produces `features:2`. An 
invalid descriptor fails validation before an ALTER_SCHEMA instant is committed.
   
   ### Risk Level
   
   Low. The change touches filesystem catalog ALTER ordering and the common 
internal schema reader. On Flink 1.20.1, all 14 DFS catalog schema evolution 
tests and all 6 `TestSerDeHelper` tests passed. The SQL regressions cover 
rename, consecutive drops, and a stale descriptor leaving the timeline and 
catalog properties intact. Checkstyle passed for both affected modules.
   
   ### Documentation Update
   
   none
   
   ### Contributor's checklist
   
   - [ ] Read through [contributor's 
guide](https://hudi.apache.org/contribute/how-to-contribute)
   - [x] Enough context is provided in the sections above
   - [x] Adequate tests were added if applicable
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to