ferrari6666 opened a new issue, #6989: URL: https://github.com/apache/iceberg/issues/6989
### Query engine Spark ### Question When spark abort is called from SparkWrite if few files are deleted but manifest,manifestLists and metadata not deleted this is causing table corruption with AWS glue catalog TableOperations Pls can throw some light on this. Catalog commit was in UNKOWN state which causes cleanupOnAbort = false; ideally this should not attempt file cleanup. It attempted for file cleanup as S3 logs shows file marked as deleted. (edited) [8:30](https://apache-iceberg.slack.com/archives/C03LG1D563F/p1677769243280359) if (deletedFilesCount.get() < paths.size()) { LOG.warn("Deleted only {} of {} file(s) ({})", deletedFilesCount, paths.size(), context); } [8:31](https://apache-iceberg.slack.com/archives/C03LG1D563F/p1677769291365349) manual rollback needs to be done? @Anton -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
