hudi-agent commented on code in PR #19582:
URL: https://github.com/apache/hudi/pull/19582#discussion_r3763786020
##########
hudi-client/hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieMergeHelper.java:
##########
@@ -93,8 +94,15 @@ public void runMerge(HoodieTable<?, ?, ?, ?> table,
// Check whether the writer schema is simply a projection of the file's
one, ie
// - Its field-set is a proper subset (of the reader schema)
// - There's no schema evolution transformation necessary
+ // Shredded variant columns are aligned to the writer side first: their
footer-derived file
+ // schema surfaces a plain {metadata, value, typed_value} record where the
writer schema has a
+ // variant, which can never pass the strict check even though the reader
reconstructs such
+ // columns when handed a variant-bearing requested schema. Without the
alignment recordSchema
+ // degenerates to the footer schema, reconstruction cannot anchor on a
variant requested
Review Comment:
🤖 The alignment only takes effect when
`schemaEvolutionTransformerOpt.isEmpty()`. If Advanced Schema Evolution
(schema-on-read) is enabled on a table with a shredded variant column, wouldn't
this same small-file merge fall back to `recordSchema = readerSchema` (the
footer schema, no variant to anchor on) and drop `typed_value` again via the
transformer's rewrite? Is that combination out of scope here, or worth a
follow-up?
<sub><i>⚠️ AI-generated; verify before applying. React 👍/👎 to flag
quality.</i></sub>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
