[
https://issues.apache.org/jira/browse/HUDI-9684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rahil Chertara updated HUDI-9684:
---------------------------------
Description:
Currently our existing tests for upgrade/downgrade have complicated logic in
terms of validating the many different upgrade/downgrade handlers we have.
Discussed with Ethan that we should have a new approach of validating
upgrade/downgrade process with a new test framework.
The framework will use specific hudi binaries with spark to create each each
table version.
* master -> version 9
* Hudi 1.0.2 -> version 8
* Hudi 0.14 -> version 6
* Hudi 0.12.2 -> version 5
* Hudi 0.11.1 -> version 4
Precreated tables for version 0.14.0(table ver 6) 1.0.2(table version 8) 1.1
(table version 9 current master)
* For now a MOR table with mdt enabled (with small data and simple schema)
* Table should have existing log files and an inflight commit
* Running upgrade/downgrade on each of these versions
* Test autoUpgrade flag is disbaled to ensure no process is ran
* Verify If rollback was exceuted, if compaction was executed for specific
handlers
* Validate table configs and write config(write.table.version) at the end of
the process
was:
Recently we have opened the following test fixtures PR:
[https://github.com/apache/hudi/pull/13669]
We would like to add the following tests to this test class in the future.
* Add a test on a table with default/null partition for default partition
change (see {{{}FourToFiveUpgradeHandler{}}}).
* Add a test for verifying the auxiliary folder. (See FixToSixUpgradeHandler)
> Add Uprgade/Downgrade Test Fixtures Framework
> ---------------------------------------------
>
> Key: HUDI-9684
> URL: https://issues.apache.org/jira/browse/HUDI-9684
> Project: Apache Hudi
> Issue Type: Improvement
> Reporter: Rahil Chertara
> Priority: Major
> Fix For: 1.2.0
>
>
> Currently our existing tests for upgrade/downgrade have complicated logic in
> terms of validating the many different upgrade/downgrade handlers we have.
> Discussed with Ethan that we should have a new approach of validating
> upgrade/downgrade process with a new test framework.
> The framework will use specific hudi binaries with spark to create each each
> table version.
> * master -> version 9
> * Hudi 1.0.2 -> version 8
> * Hudi 0.14 -> version 6
> * Hudi 0.12.2 -> version 5
> * Hudi 0.11.1 -> version 4
> Precreated tables for version 0.14.0(table ver 6) 1.0.2(table version 8) 1.1
> (table version 9 current master)
> * For now a MOR table with mdt enabled (with small data and simple schema)
> * Table should have existing log files and an inflight commit
> * Running upgrade/downgrade on each of these versions
>
> * Test autoUpgrade flag is disbaled to ensure no process is ran
> * Verify If rollback was exceuted, if compaction was executed for specific
> handlers
> * Validate table configs and write config(write.table.version) at the end of
> the process
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
