prashantwason commented on pull request #4067:
URL: https://github.com/apache/hudi/pull/4067#issuecomment-977583545
Concept looks good. But why introduce a new block type and not do it for the
HoodieHFileDataBlock itself?
When the HFile format is used, whether for Metadata Table or elsewhere in
HUDI, there will always be a key for the HFile and they will be derived from
some field of the record. Hence, this HFile key will always be redundant.
Therefore this optimization needs to performed for HoodieHFileDataBlock itself.
HoodieHFileDataBlock already accepts a "keyField". We can simply this change
by:
1. If keyField is not None:
- set keyField to "null" and do not save it
- materialize the keyField from HFile key
2. If keyField is None - no need to do the above
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
