Any ideas on this? On Sun, 4 Sep, 2022, 5:56 pm Taher Koitawala, <taher...@gmail.com> wrote:
> Hi All, > Need your help with deleting a positionDelete file that has been > committed before and writing a new one instead. > > The use case is we produce a Position delete file and commit it > to the table using rowDelta.addDeletes(posDeleteFile). Let us say this is > snapshot 1, when we do snapshot 2 what I want to do is delete posDeleteFile > from snapshot 1 and add a new file to snapshot 2. > > To plan for files I am using > > CloseableIterator<CombinedScanTask> iterator = table.newScan() > .useSnapshot(table.currentSnapshot().snapshotId()) > .planTasks().iterator(); > > StreamSupport.stream(Spliterators > .spliteratorUnknownSize(iterator, > Spliterator.ORDERED) > , false) > .flatMap(f -> f.deletes().stream()) > .collect(Collectors.toSet()); > > however, this gives me all the delete files. My specific usecase is to > keep rolling the delete file, so on a new snapshot, I want to do is delete > the delete file from snapshot - 1 and add a new delete file to the current > snapshot. > The current snapshot should then only read the new file committed and not > the ones from snapshot - 1. > > Also, can you tell me how to delete the physical file along with snapshot > and manifest entries for the older delete file. > > Regards, > Taher Koitawala >
