developer-rpai opened a new pull request, #18407: URL: https://github.com/apache/iceberg/pull/18407
When a catalog applies a commit but reports it as failed (e.g. a 409 for an already-applied commit), the commit retry can fail validation after the snapshot is already on the table. SnapshotProducer's failure-path cleanup then deleted the staged manifest list and manifests even though the committed snapshot references them, corrupting the table. This guards the failure cleanup: before deleting, it refreshes and, if the staged snapshot id is on the table, cleans up exactly as the success path does. If the table state cannot be determined, cleanup is skipped rather than risking deletion of referenced files. One reviewer note: this adds a refresh() on the failure path only; the success path is extracted unchanged into a shared helper. Fixes #18402 Test plan: new test committedSnapshotFilesSurviveFailedCommitCleanup in TestSnapshotProducer reproduces the scenario with a TableOperations that applies the commit then throws CommitFailedException, and a validator that throws ValidationException on retry. It asserts the commit throws, the snapshot is current, and its manifest list and manifests still exist. I could not run the Gradle build in this environment, so CI is the authority for compile and test results. --- **AI Disclosure** - Model: Muse Spark - Platform/Tool: Hatch agent - Human Oversight: [unknown - human to fill in] - Prompt Summary: Fix apache/iceberg#18402 by guarding SnapshotProducer failure-path cleanup so it never deletes files referenced by an already-committed snapshot, plus a regression test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
