sivabalan narayanan created HUDI-9026:
-----------------------------------------
Summary: Add backwards compatible MDT writer support and reader
support with tbl v6 - Design
Key: HUDI-9026
URL: https://issues.apache.org/jira/browse/HUDI-9026
Project: Apache Hudi
Issue Type: Improvement
Reporter: sivabalan narayanan
Assignee: sivabalan narayanan
We wanted to add b/w compatible mdt write support w/ tbl version 6 in 1.0.
This ticket tracks the design of such b/w compatible support.
Reqs:
* Able to write using tbl version 6 with mdt enabled.
* 1.0 binary able to read any 0.x table w/ mdt enabled.
* Seamless upgrade to 1.0 w/o nuking entire mdt.
* Also provide a downgrade path w/o nuking entire MDT if feasible.
Expectations:
* All storage changes should be accounted for
* Inmemory computation should be retained as much as possible to align w/ 0.x
behavior.
* Its ok to force full compaction in MDT during upgrade. Anyways, we only do
snapshot read on MDT. So, once compacted, we should not have any issues w/ MDT.
no one will ever read older file slices from MDT.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
