nsivabalan opened a new pull request #3315:
URL: https://github.com/apache/hudi/pull/3315
## What is the purpose of the pull request
- Adding virtual keys support to MOR table
- Compaction
- Realtime read
- Clustering
Constraints:
Only SimpleKeyGen is supported because during real time read (snapshot
read), we can't afford to generated keys using complex key gens. Query times
will shoot up and its unusable from a user standpoint.
## Brief change log
- Introduced 2 additional configs to HoodieTableConfig to serialize the
simple record field and simple partition field for the table.
- Added virutal keys support to MOR table for compaction, clustering and
realtime read.
- Metadata table is ensured to work w/ virtual keyed MOR table.
## Verify this pull request
*(Please pick either of the following options)*
This change added tests and can be verified as follows:
- Fixed TestHoodieMergeOnReadTable and TestHoodieBackedMetadata for virtual
keys with MOR table.
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
