aokolnychyi edited a comment on pull request #1313: URL: https://github.com/apache/iceberg/pull/1313#issuecomment-671514522
I would like to make sure I get the use case correctly. Is the main purpose to use `RemoveOrphanFilesAction` to clean up expired files? If so, I think that's a misuse of the API. Instead, one should use `ExpireSnapshotsAction` that Russell is working on in PR #1264. The most expensive part in `RemoveOrphanFilesAction` is listing. The bigger table the more pressure you put on the storage. The number of orphan files should be usually small so the delete should be quick. This action should be called maybe once a month or so. `ExpireSnapshotsAction`, on the other hand, is meant to be called regularly. It will have an executor service to parallelize deletes. `ExpireSnapshotsAction` is supposed to delete a lot of files. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
