lihanyu1715446406-afk opened a new pull request, #20561:
URL: https://github.com/apache/hudi/pull/20561

   ### Describe the issue this Pull Request addresses
   
   Closes #20528. Hive/MR snapshot reads fail when the file group reader reads 
a union column that holds a non-null value: 
`HoodieArrayWritableSchemaUtils#rewriteRecordWithNewSchemaInternal` has no 
`UNION` handler and throws `IllegalArgumentException: should not be here?`.
   
   Parquet stores a union column as a record of optional `member0..memberN` 
fields, so the file schema is that record while the required table schema is 
still the union. The legacy reader already returns the member record shape (a 
struct with the populated branch and the others null), which is what Hive 
expects.
   
   ### Summary and Changelog
   
   - Handle a `UNION` target in 
`HoodieArrayWritableSchemaUtils#rewriteRecordWithNewSchemaInternal` when the 
source schema is the member record: the writable already has the required 
representation, so it is kept as-is.
   - Added unit tests for a direct union column and an array of union values.
   
   ### Impact
   
   Fixes reads of union columns through the file group reader. No public API or 
storage format change.
   
   ### Risk Level
   
   Low. The new branch only applies when the source schema is a `RECORD` and 
the target is a `UNION`; every other `UNION` case still throws as before.
   
   ### Documentation Update
   
   none
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to