[
https://issues.apache.org/jira/browse/HUDI-7971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Surya Prasanna Yalla updated HUDI-7971:
---------------------------------------
Summary: Test and Certify 0.14.x tables are readable in 1.x Hudi reader
(was: Test and Certify 0.14.x to 0.16.x tables are readable in 1.x Hudi reader )
> Test and Certify 0.14.x tables are readable in 1.x Hudi reader
> ---------------------------------------------------------------
>
> Key: HUDI-7971
> URL: https://issues.apache.org/jira/browse/HUDI-7971
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: sivabalan narayanan
> Assignee: Lokesh Jain
> Priority: Major
> Fix For: 1.0.0
>
>
> Lets ensure 1.x reader is fully compatible w/ reading any of 0.14.x to 0.16.x
> tables
>
> Readers : 1.x
> # Spark SQL
> # Spark Datasource
> # Trino/Presto
> # Hive
> # Flink
> Writer: 0.16
> Table State:
> * COW
> ** few write commits
> ** Pending clustering
> ** Completed Clustering
> ** Failed writes with no rollbacks
> ** Insert overwrite table/partition
> ** Savepoint for Time-travel query
> * MOR
> ** Same as COW
> ** Pending and completed async compaction (with log-files and no base file)
> ** Custom Payloads (for MOR snapshot queries) (e:g SQL Expression Payload)
> ** Log block formats - DELETE, rollback block
> Other knobs:
> # Metadata enabled/disabled (all combinations)
> # Column Stats enabled/disabled and data-skipping enabled/disabled
> # RLI enabled with eq/IN queries
> # Non-Partitioned dataset (all combinations)
> # CDC Reads
> # Incremental Reads
> # Time-travel query
>
> What to test ?
> # Query Results Correctness
> # Performance : See the benefit of
> # Partition Pruning
> # Metadata table - col stats, RLI,
>
> Corner Case Testing:
>
> # Schema Evolution with different file-groups having different generation of
> schema
> # Dynamic Partition Pruning
> # Does Column Projection work correctly for log files reading
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
