[
https://issues.apache.org/jira/browse/HUDI-7665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vinoth Chandar updated HUDI-7665:
---------------------------------
Description:
*Supported migration combinations*
Obviously, within 1.x code line - writers, readers, table services all need to
co-exist for both table versions (tv) 6 & 8.
|| ||Reader 0.x ||Reader 1.x tv=6/8||Table Services 0.x||Table Services 1.x
tv=6||Table Services 1.x tv=8||
|*Writer 1.x tv=6*|(/)|(/)|(x)|(/)|(x)|
|*Writer 1.x tv=8*|(x)|(/)|(x)|(x)|(/)|
|*Table Services 1.x tv=6*|(/)|(/)|(x)|(/)|(x)|
|*Table Services 1.x tv=8*|(x)|(/)|(x)|(x)|(/)|
We are making the practical restriction that table services 0.x, need to be
stopped first, upgraded to new binary 1.x, before resuming operations even for
tv=6 to be able to co-exist with a 1.x tv=6 writer.
*Things to be done during rolling upgrade*
the 6 to 8 upgrade handler, need to handle the following.
| * Migrating table properties including key generators, payloads.
* Migrating timeline to new layout
* Full compact the table and clean, to get rid of log files?
* Migration path for CDC and incremental queries.
* Handle differences between 0.x and 1.x for stuff needed in upgrade e.g
compaction (need to compact older file slice), rollback (marker differences b/w
0.14 and 0.15)|
*Safe migration protocol :*
<wip>
*Downgrade tooling :*
<wip>
was:
*Supported migration combinations*
Obviously, within 1.x code line - writers, readers, table services all need to
co-exist for both table versions (tv) 6 & 8.
|| ||Reader 0.x ||Reader 1.x tv=6/8||Table Services 0.x||Table Services 1.x
tv=6||Table Services 1.x tv=8||
|*Writer 1.x tv=6*|(/)|(/)|(x)|(/)|(x)|
|*Writer 1.x tv=8*|(x)|(/)|(x)|(x)|(/)|
|*Table Services 1.x tv=6*|(/)|(/)|(x)|(/)|(x)|
|*Table Services 1.x tv=8*|(x)|(/)|(x)|(x)|(/)|
We are making the practical restriction that table services 0.x, need to be
stopped first, upgraded to new binary 1.x, before resuming operations even for
tv=6 to be able to co-exist with a 1.x tv=6 writer.
*Things to be done during rolling upgrade*
the 6 to 8 upgrade handler, need to handle the following.
| * Migrating table properties including key generators, payloads.
* Migrating timeline to new layout
* Full compact the table and clean, to get rid of log files?
* Migration path for CDC and incremental queries.|
*[WIP] Safe migration protocol :*
*[WIP] Downgrade tooling :*
> Rolling upgrade to table version 8
> ----------------------------------
>
> Key: HUDI-7665
> URL: https://issues.apache.org/jira/browse/HUDI-7665
> Project: Apache Hudi
> Issue Type: Task
> Reporter: Sagar Sumit
> Assignee: Sagar Sumit
> Priority: Major
> Labels: hudi-1.0.0-beta2
> Fix For: 1.0.0
>
>
>
> *Supported migration combinations*
> Obviously, within 1.x code line - writers, readers, table services all need
> to co-exist for both table versions (tv) 6 & 8.
> || ||Reader 0.x ||Reader 1.x tv=6/8||Table Services 0.x||Table Services 1.x
> tv=6||Table Services 1.x tv=8||
> |*Writer 1.x tv=6*|(/)|(/)|(x)|(/)|(x)|
> |*Writer 1.x tv=8*|(x)|(/)|(x)|(x)|(/)|
> |*Table Services 1.x tv=6*|(/)|(/)|(x)|(/)|(x)|
> |*Table Services 1.x tv=8*|(x)|(/)|(x)|(x)|(/)|
> We are making the practical restriction that table services 0.x, need to be
> stopped first, upgraded to new binary 1.x, before resuming operations even
> for tv=6 to be able to co-exist with a 1.x tv=6 writer.
>
> *Things to be done during rolling upgrade*
> the 6 to 8 upgrade handler, need to handle the following.
> | * Migrating table properties including key generators, payloads.
> * Migrating timeline to new layout
> * Full compact the table and clean, to get rid of log files?
> * Migration path for CDC and incremental queries.
> * Handle differences between 0.x and 1.x for stuff needed in upgrade e.g
> compaction (need to compact older file slice), rollback (marker differences
> b/w 0.14 and 0.15)|
>
>
> *Safe migration protocol :*
> <wip>
>
> *Downgrade tooling :*
> <wip>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
