[
https://issues.apache.org/jira/browse/HUDI-9066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Y Ethan Guo updated HUDI-9066:
------------------------------
Priority: Blocker (was: Major)
> Auto upgrade and stale metaclient issue
> ---------------------------------------
>
> Key: HUDI-9066
> URL: https://issues.apache.org/jira/browse/HUDI-9066
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Davis Zhang
> Assignee: Y Ethan Guo
> Priority: Blocker
> Fix For: 1.0.2
>
>
> for SIX to EIGHT upgrade, today we must do via write client, so either DS
> ingest/SQL will be responsible for that.
>
> During test we found the corner case:
> * Writer do the write and upgrade, table version is 8 with timeline layout 2
> * writer continue to do something else, but the metaclient it holds is still
> a stale one using version 6 and layout 1
> * So the subsequent logic will fail when parsing instants. Here the upgrade
> succeeds not gracefully.
> * For DS it will auto retry and meta client is recreated with the new
> versions. For SQL writers / data frame writers they need to restart the
> session to refresh metaclient.
>
> In general, the impact is limited. Given the impact, it might not be
> desirable to tangle auto upgrade and write operations unless we compensate
> with test coverage and extra design. I know this decision is made out of
> usability - user set auto upgrade flag and things will flow with no further
> action required.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
