dhananjaykrutika commented on code in PR #677:
URL: https://github.com/apache/iceberg-go/pull/677#discussion_r2694094852
##########
table/transaction.go:
##########
@@ -184,9 +185,15 @@ func WithOlderThan(t time.Duration) ExpireSnapshotsOpt {
}
}
+func WithPostCommit(postCommit bool) ExpireSnapshotsOpt {
+ return func(cfg *expireSnapshotsCfg) {
+ cfg.postCommit = postCommit
+ }
+}
Review Comment:
@zeroshade
So my idea was this:
PostCommit() is a no-op for all updates barring `RemoveSnapshotsUpdate`,
where it immediately deletes files (manifest files, manifest list files, data
files etc) that became orphaned/unreferenced by virtue of **this** snapshot's
expiration from storage. So queries that are scanning data files potentially
referenced by **this** snapshot would fail or time-travel queries could see
files deleted mid-query.
So the use-case I am using this code for, revolves around delegating this
decision of deleting orphaned/unreferenced files to a separate maintenance job,
that would run periodically, and use a different logic, such as adding a grace
period of `X` days since the files became orphaned, before to deleting them
permanently from storage.
Let me know what you think.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
