hudi-bot opened a new issue, #15970:
URL: https://github.com/apache/hudi/issues/15970
For COW, col stats processing/evaluation is straight forward. but for MOR
tables, it may not be.
We store file name as one of the component in record key in col stats
entries. So, each file will have an entry in col stats index in MDT. But for
MOR table, expectation is, we need one col stats value for entire file slice.
(base file + bunch of log files).
we should also include merged file slices (last but one file slice's base
file + last but one file slice's log files + latest slice's log files)
We need to go through the flow end to end and verify these are all intact.
* Ensure we are able to evaluate one col stats value for one file slice
* Ensure merge file slices are in-corporated
* Ensure delete log blocks are in-corporated as well.
* Ensure custom deletes (custom payloads) are also incorporated.
## JIRA info
- Link: https://issues.apache.org/jira/browse/HUDI-6248
- Type: Bug
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
