rangareddy commented on issue #17273: URL: https://github.com/apache/hudi/issues/17273#issuecomment-5365810478
This issue was reviewed as part of the JIRA-migrated backlog triage (HUDI-8241). **Findings: needs a reproduction on current `master`; the premise may have shifted.** The description reads as a known-limitation note rather than a defect report: `MERGE INTO` uses the Avro record merger so `ExpressionPayload` works, merger implementations get overwritten, and therefore a user-supplied custom merger is not honoured when records are read during a cross-partition (global index) update. It also explicitly defers the fix - *"There might be some changes to how we do merger impls so I don't want to address this issue yet."* Those changes have since happened. The merge layer was rewritten around table versions 8 and 9, with the `BufferedRecordMerger` family replacing payload-based merging, and merge mode became the primary merging config (#11943). Whether the limitation survives in the same shape is not determinable by reading the current code, and asserting either way would be a guess. To settle it, a `MERGE INTO` test on current `master` with a global index, an update that moves a record across partitions, and a custom `HoodieRecordMerger` - checking whether the custom merger is invoked on the read of the existing record. Keeping this open. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
