Rahil Chertara created HUDI-9660:
------------------------------------
Summary: Fix Upgrade/Downgrade Command To Keep MDT Table and Data
Table (max one version difference apart)
Key: HUDI-9660
URL: https://issues.apache.org/jira/browse/HUDI-9660
Project: Apache Hudi
Issue Type: Improvement
Reporter: Rahil Chertara
Currently within our code for Upgrade/Downgrade we first either fully
uprgade/downgrade the mdt table from a startVersion to an endVersion. Then we
do the same for the data table. For example say a user would like to go from
table version 6 to now table version 9.
We would first migrate the MDT table to 6 -> 7, then 7-> 8, and finally 8->9 .
Then we repeat the same process for the data table.
The issue is that if there is a crash after upgrading the mdt table say during
the process of finishing 9, the problem is the data table is still stuck on 6.
We should ideally increment both mdt table and data table together to a version
such as doing mdt table 6->7 and dt table 6->7, then move on to next increment
for both.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
