Hi Prashant, Not sure if there is a specific reason. Mostly, it because until recently, the clean metadata was not actually used. Currently, incremental cleaning will use it, but even then, it only relies on the partition paths being touched there.. So should be fine..
+100 though on consistently cleaning all of this up. Some of these inconsistencies exist actually to ensure the old timelines for old users (e.g uber) continue to work. So I would like to actually have a conversation on streamlining all this, so the system implementation is as simple/close to the design.. On Wed, Mar 18, 2020 at 11:23 AM Prashant Wason <[email protected]> wrote: > HI Team, > > I noticed that when a table is restored to a previous commit ( > HoodieWriteClient::restoreToInstant > < > https://github.com/apache/incubator-hudi/blob/master/hudi-client/src/main/java/org/apache/hudi/client/HoodieWriteClient.java#L735 > >), > only the COMMIT, DELTA_COMMIT and COMPACTION instants are rolled back and > their corresponding files are deleted from the timeline. If there are some > CLEAN instants, they are left over. > > Is there a reason why CLEAN are not removed? Won't they be referring to > files which are no longer present and hence not useful? > > Thanks > Prashant >
