nada-attia opened a new pull request, #20763: URL: https://github.com/apache/hudi/pull/20763
### Describe the issue this Pull Request addresses A `MERGE INTO ... WHEN MATCHED THEN UPDATE` that goes through `HoodieWriteMergeHandle` can fail with `HoodieIOException: Failed to parse record into provided schema` (caused by an `EOFException` or `Invalid int encoding`), or silently write shifted values. This happens when the MERGE source columns are `NOT NULL` (for example a subquery of constants) and the table columns are nullable. ### Summary and Changelog Fixes MERGE INTO updates through the payload-based merge handle when source and table nullability differ. - `HoodieWriteMergeHandle#write` wrapped the merge result, which is already in the table schema, in the configured payload (`ExpressionPayload` for MERGE INTO) via `constructHoodieRecord`. `shouldIgnore` and `writeUpdateRecord` then called `ExpressionPayload#getInsertValue` on it, which decodes the bytes with the MERGE source schema (`hoodie.payload.record.schema`) instead of the table schema, so the nullable-union bytes were read as plain fields. - The combined record is now built with `constructFinalHoodieRecord`, so it is a plain record. The record merger already resolves a payload `SENTINEL` result by returning the stored record, so no payload evaluation is needed on the merge result. No code was copied. ### Impact No public API change. MERGE INTO updates through `HoodieWriteMergeHandle` no longer fail or write shifted values for the scenario above. The `numUpdates` stat may differ when the stored row is kept, since the combined record is now a plain record rather than a payload-wrapped one. ### Risk Level low. This is a one-line change limited to the payload-based merge handle (`HoodieWriteMergeHandle`), not the default `FileGroupReaderBasedMergeHandle`. The same fix was verified on a Hudi 1.2 branch: a MERGE INTO with a NOT NULL constants source, a nullable source, and a conditional update whose condition does not match (the stored row is kept) all passed. This PR itself has not been built or tested against `master`, and no new test is added here. ### Documentation Update none ### Contributor's checklist - [x] Read through [contributor's guide](https://hudi.apache.org/contribute/how-to-contribute) - [x] Enough context is provided in the sections above - [ ] Adequate tests were added if applicable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
