[
https://issues.apache.org/jira/browse/HUDI-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-6206:
--------------------------------------
Description:
I happened to check for col stats tests in TestHoodieBackedMetadata and did not
find any good tests that validates col stats records from MDT and bloom
partition records. I do see some validations for log files, but none for base
files. we do have one test named testColStatsPrefixLookup which tests col
stats. but if we can add a separate validateColStats() method we can call it
with validateMetadata() for any tests we have to gain more coverage.
We do a sparate test class TestColumnStatsIndex for testing col stats. But
would be good to enhance the validations in TestHoodieBackedMetadata tests.
we should enhance them for sure.
was:
I happened to check for col stats tests in TestHoodieBackedMetadata and did not
find any good tests that validates col stats records from MDT and bloom
partition records. I do see some validations for log files, but none for base
files. we do have one test named testColStatsPrefixLookup which tests col
stats. but if we can add a separate validateColStats() method we can call it
with validateMetadata() for any tests we have to gain more coverage.
we should enhance them for sure.
> Enhance Metadata tests to validate col stats
> --------------------------------------------
>
> Key: HUDI-6206
> URL: https://issues.apache.org/jira/browse/HUDI-6206
> Project: Apache Hudi
> Issue Type: Improvement
> Components: metadata
> Reporter: sivabalan narayanan
> Priority: Critical
> Labels: release-0.14.0-blocker
>
> I happened to check for col stats tests in TestHoodieBackedMetadata and did
> not find any good tests that validates col stats records from MDT and bloom
> partition records. I do see some validations for log files, but none for base
> files. we do have one test named testColStatsPrefixLookup which tests col
> stats. but if we can add a separate validateColStats() method we can call it
> with validateMetadata() for any tests we have to gain more coverage.
>
> We do a sparate test class TestColumnStatsIndex for testing col stats. But
> would be good to enhance the validations in TestHoodieBackedMetadata tests.
> we should enhance them for sure.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
