[
https://issues.apache.org/jira/browse/HUDI-9675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Timothy Brown updated HUDI-9675:
--------------------------------
Description:
You can currently savepoint to an invalid commit when the
{{KEEP_LATEST_FILE_VERSIONS}} is enabled. The earliest commit to retain will
never be set in the clean metadata so the check allows you to savepoint any
commit, regardless of whether the files are present.
We should make a very conservative check here and inspect whether there is any
clean commit after the instant of interest. If there is and it is not the
commit immediately after the commit of interest, then fail.
was:
{{}}
You can currently savepoint to an invalid commit when the
{{KEEP_LATEST_FILE_VERSIONS}} is enabled. The earliest commit to retain will
never be set in the clean metadata so the check allows you to savepoint any
commit, regardless of whether the files are present.
{{}}
{{}}
We should make a very conservative check here and inspect whether there is any
clean commit after the instant of interest. If there is and it is not the
commit immediately after the commit of interest, then fail.
> Savepoint safety checks are missing for cleaner policy
> KEEP_LATEST_FILE_VERSIONS
> --------------------------------------------------------------------------------
>
> Key: HUDI-9675
> URL: https://issues.apache.org/jira/browse/HUDI-9675
> Project: Apache Hudi
> Issue Type: Bug
> Reporter: Timothy Brown
> Assignee: Timothy Brown
> Priority: Major
>
> You can currently savepoint to an invalid commit when the
> {{KEEP_LATEST_FILE_VERSIONS}} is enabled. The earliest commit to retain will
> never be set in the clean metadata so the check allows you to savepoint any
> commit, regardless of whether the files are present.
> We should make a very conservative check here and inspect whether there is
> any clean commit after the instant of interest. If there is and it is not the
> commit immediately after the commit of interest, then fail.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
