mengna-lin opened a new issue, #17715:
URL: https://github.com/apache/iceberg/issues/17715
### Feature Request / Improvement
Feature: best-effort file deletion in expire snapshots when a referenced
metadata file is missing
Today, ExpireSnapshots / ExpireSnapshotsSparkAction aborts the entire
operation if a referenced manifest or manifest-list file is missing from
storage (NotFoundException). The failure happens while computing the set of
files to delete, so no cleanup runs at all — even for the files it could have
safely deleted.
To let expiration make progress instead of failing outright, while staying
safe:
- If the missing manifest is reachable only from snapshots being expired,
delete every file that was fully resolved, then surface a single aggregated
error/warning listing the missing files. Under-deleting here only leaves
orphans, which remove_orphan_files can clean up later — it never removes live
data.
- If the missing manifest is reachable from a retained snapshot, the delete
set can't be computed safely (risk of deleting live files), so fail fast before
deleting anything.
### Query engine
None
### Willingness to contribute
- [x] I can contribute this improvement/feature independently
- [ ] I would be willing to contribute this improvement/feature with
guidance from the Iceberg community
- [ ] I cannot contribute this improvement/feature at this time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
