Davis Zhang created HUDI-9066:
---------------------------------
Summary: Auto upgrade and stale metaclient issue
Key: HUDI-9066
URL: https://issues.apache.org/jira/browse/HUDI-9066
Project: Apache Hudi
Issue Type: Bug
Reporter: Davis Zhang
for SIX to EIGHT upgrade, today we must do via write client, so either DS
ingest/SQL will be responsible for that.
During test we found the corner case:
* Writer do the write and upgrade, table version is 8 with timeline layout 2
* writer continue to do something else, but the metaclient it holds is still a
stale one using version 6 and layout 1
* So the subsequent logic will fail when parsing instants. Here the upgrade
succeeds not gracefully.
* For DS it will auto retry and meta client is recreated with the new
versions. For SQL writers / data frame writers they need to restart the session
to refresh metaclient.
In general, the impact is limited. Given the impact, it might not be desirable
to tangle auto upgrade and write operations unless we compensate with test
coverage and extra design. I know this decision is made out of usability - user
set auto upgrade flag and things will flow with no further action required.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
