[
https://issues.apache.org/jira/browse/HUDI-7900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17882205#comment-17882205
]
Ethan Guo edited comment on HUDI-7900 at 9/16/24 11:13 PM:
-----------------------------------------------------------
# I have a couple of questions while reviewing the PR: # This is the first time
where the MDT storage format (e.g., completion time, instant time format, new
indexes, etc) is changed, from Hudi 0.14 (table version 6) to 1.0 (table
version 8). Should MDT be completely deleted during upgrade/downgrade and to
be recreated later, or could we preserve the FILES/COL_STATS/BLOOM_FILTER/RLI
partitions and transform the timeline in some way?
# When deleting unsupported MDT partitions, should this happen in the data
table downgrade process or MDT downgrade process?
was (Author: guoyihua):
I have a couple of questions while reviewing the PR: # This is the first time
where the MDT storage format (e.g., completion time, instant time format, new
indexes, etc) is changed, from Hudi 0.14 (table version 6) to 1.0 (table
version 8). Should MDT be completely deleted during upgrade/downgrade and to
be recreated later, or could we preserve the FILES/COL_STATS/BLOOM_FILTER/RLI
partitions and transform the timeline in some way?
# When deleting unsupported MDT partitions, should this happen in the data
table downgrade process or MDT downgrade process?
> Downgrade handles to delete new MDT partitions and cleanup MDT table
> properties
> -------------------------------------------------------------------------------
>
> Key: HUDI-7900
> URL: https://issues.apache.org/jira/browse/HUDI-7900
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Sagar Sumit
> Assignee: Lin Liu
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.0.0
>
>
> {code:java}
> {code}
> The newer metadata partitions such as secondary_index or partitin_stats etc
> are only meant for 1.x . So if someone had created these indexes and then
> later downgraded to 0.x then we need to ensure that hoodie.properties should
> not show these indexes as available. Ideally, we would want to drop the
> metadata partition altogether.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
