codope opened a new pull request, #13083:
URL: https://github.com/apache/hudi/pull/13083
### Change Logs
The USP of partial updates is that users don't have to specify all fields in
the merge into command. However, with global index and partition path updates,
merge into command will fail because the expectation is that full record is
provided to
`HoodieIndexUtils::mergeIncomingWithExistingRecordWithExpressionPayload`. This
PR attempts to fix the behavior by doing partial merge and building full record
i.e. get the merged record and then fill in the missing fields from existing
record. The PR still uses record merger API in both the above method as well as
`HoodieMergedReadHandle#doMergedRead`. Ideally, we would want to use the
filegroup reader in `HoodieMergedReadHandle` instad of record merger. That's a
larger refactoring and for now, maybe we can just error out with message that
merge into partial updates are not supported with global index.
### Impact
Fix Merge Into Partial Updates with Global Index
### Risk level (write none, low medium or high below)
low
### Documentation Update
_Describe any necessary documentation update if there is any new feature,
config, or user-facing change. If not, put "none"._
- _The config description must be updated if new configs are added or the
default value of the configs are changed_
- _Any new feature or user-facing change requires updating the Hudi website.
Please create a Jira ticket, attach the
ticket number here and follow the
[instruction](https://hudi.apache.org/contribute/developer-setup#website) to
make
changes to the website._
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
