rdblue opened a new pull request #352: Fix transaction cleanup. URL: https://github.com/apache/incubator-iceberg/pull/352 #218 refactored how intermediate metadata files are handled in transactions by adding a callback to collect files to delete in each transaction attempt and delete just that set of files when the transaction successfully commits. This avoided deleting files that were deleted in an early attempt and eventually used, but it fails to delete other metadata files that are deleted once and removed from operation caches. A better solution is to track all of the metadata files that are deleted in any attempt and delete the ones that are not referenced by table metadata after the commit succeeds. This uses a set to track deleted files, but does not clear that set between transaction commit attempts. Instead, all manifests and manifest lists that are committed are collected and used to block deletes. Fixes #330.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@iceberg.apache.org For additional commands, e-mail: issues-h...@iceberg.apache.org
