sivabalan narayanan created HUDI-6248:
-----------------------------------------
Summary: Validate col stats for MOR data table
Key: HUDI-6248
URL: https://issues.apache.org/jira/browse/HUDI-6248
Project: Apache Hudi
Issue Type: Bug
Components: reader-core, writer-core
Reporter: sivabalan narayanan
For COW, col stats processing/evaluation is straight forward. but for MOR
tables, it may not be.
We store file name as one of the component in record key in col stats entries.
So, each file will have an entry in col stats index in MDT. But for MOR table,
expectation is, we need one col stats value for entire file slice. (base file +
bunch of log files).
we should also include merged file slices (last but one file slice's base file
+ last but one file slice's log files + latest slice's log files)
We need to go through the flow end to end and verify these are all intact.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
