[
https://issues.apache.org/jira/browse/HUDI-9097?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
sivabalan narayanan updated HUDI-9097:
--------------------------------------
Description:
This ticket tracks the design of such b/w compatible support.
Reqs:
* Able to write using tbl version 6 with mdt enabled.
* 1.0 binary able to read any 0.x table w/ mdt enabled.
* Seamless upgrade to 1.0 w/o nuking entire mdt.
* Also provide a downgrade path w/o nuking entire MDT if feasible.
Expectations:
* All storage changes should be accounted for
* In memory computation should be retained as much as possible to align w/ 0.x
behavior.
* Its ok to force full compaction in MDT during upgrade. Anyways, we only do
snapshot read on MDT. So, once compacted, we should not have any issues w/ MDT.
no one will ever read older file slices from MDT.
was:
This ticket tracks the design of such b/w compatible support.
Reqs:
* Able to write using tbl version 6 with mdt enabled.
* 1.0 binary able to read any 0.x table w/ mdt enabled.
* Seamless upgrade to 1.0 w/o nuking entire mdt.
* Also provide a downgrade path w/o nuking entire MDT if feasible.
Expectations:
* All storage changes should be accounted for
* Inmemory computation should be retained as much as possible to align w/ 0.x
behavior.
* Its ok to force full compaction in MDT during upgrade. Anyways, we only do
snapshot read on MDT. So, once compacted, we should not have any issues w/ MDT.
no one will ever read older file slices from MDT.
Here is the impl jira: https://issues.apache.org/jira/browse/HUDI-9013
> Design: B/w compatible MDT writer support in 1.0
> -------------------------------------------------
>
> Key: HUDI-9097
> URL: https://issues.apache.org/jira/browse/HUDI-9097
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: sivabalan narayanan
> Assignee: sivabalan narayanan
> Priority: Major
> Fix For: 1.0.2
>
>
> This ticket tracks the design of such b/w compatible support.
> Reqs:
> * Able to write using tbl version 6 with mdt enabled.
> * 1.0 binary able to read any 0.x table w/ mdt enabled.
> * Seamless upgrade to 1.0 w/o nuking entire mdt.
> * Also provide a downgrade path w/o nuking entire MDT if feasible.
> Expectations:
> * All storage changes should be accounted for
> * In memory computation should be retained as much as possible to align w/
> 0.x behavior.
> * Its ok to force full compaction in MDT during upgrade. Anyways, we only do
> snapshot read on MDT. So, once compacted, we should not have any issues w/
> MDT. no one will ever read older file slices from MDT.
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
