+1 One caveat here is that long queries (e.g Hive) may still be accessing these older files and may fail when cleaning very aggressively like this.
On Mon, Aug 26, 2019 at 2:24 AM Gary Li <[email protected]> wrote: > Hello, > > You can achieve this by changing the hudi config. > hoodie.cleaner.policy=KEEP_LATEST_FILE_VERSIONS > hoodie.cleaner.commits.retained=1 > > Best, > Gary > > > On Mon, Aug 26, 2019 at 10:06 AM Максим Радионов <[email protected] > > > wrote: > > > Hi all, > > > > I use COW and HoodieDeltaStreamer and after successful execution of a new > > job I would like for the old data with commits to be deleted and there > > would be a data snapshot without the history of commits, please tell me > the > > correct way to solve this task. > > > > example: (i whould like to delete old file commit after succesed job) > > fe57cbbe-fea9-493d-8c20-b74c5b699597_200_20190803114918.parquet > > fe57cbbe-fea9-493d-8c20-b74c5b699597_1479_20190804125520.parquet > > > > Kind regards, > > Maksim > > >
