[
https://issues.apache.org/jira/browse/HUDI-8077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Chen updated HUDI-8077:
-----------------------------
Description:
Currently, the incremental cleaning will remember a marker instant of last
retained in the commit metadata, both the marker and the filtering instant on
fs view are start times(instant times), while this is okay for most of the
cases because we actually have some buffer time for cleaning(30 commits
retained by default), but if the user sets up a very redical strategy like
clean for every commits, then there might be isses in NB-CC mode:
A instant that starts very early but finished recently might be skipped by the
cleaning table service.
> Fix the incremental cleaning to base on completion time
> -------------------------------------------------------
>
> Key: HUDI-8077
> URL: https://issues.apache.org/jira/browse/HUDI-8077
> Project: Apache Hudi
> Issue Type: Sub-task
> Reporter: Danny Chen
> Priority: Major
> Fix For: 1.0.0
>
>
> Currently, the incremental cleaning will remember a marker instant of last
> retained in the commit metadata, both the marker and the filtering instant on
> fs view are start times(instant times), while this is okay for most of the
> cases because we actually have some buffer time for cleaning(30 commits
> retained by default), but if the user sets up a very redical strategy like
> clean for every commits, then there might be isses in NB-CC mode:
> A instant that starts very early but finished recently might be skipped by
> the cleaning table service.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
