mengna-lin opened a new pull request, #19071: URL: https://github.com/apache/iceberg/pull/19071
Backport of #18477 to Spark 3.5, 4.0 and 4.1. ExpireSnapshotsSparkAction fails when a manifest list or manifest reachable from an expired snapshot is already gone. The snapshots are already expired at that point, so retrying cannot make progress and orphans accumulate. Such missing metadata is now skipped and logged, and the action returns the result of the files it deleted. Missing metadata reachable from a retained snapshot still fails. The code is identical to the Spark 4.2 change. In Spark 4.0, adaptive execution reports several failed stages as one exception, so the fail-fast test checks the full stack trace instead of the message. Issue: https://github.com/apache/iceberg/issues/17715 Depends on #18477. Test plan: TestExpireSnapshotsAction, TestDeleteReachableFilesAction and TestRemoveOrphanFilesAction pass on each version. --- **AI Disclosure** - Model: Claude Sonnet 5.5 - Platform/Tool: Claude Code - Human Oversight: [unknown - human to fill in] - Prompt Summary: Backport the best-effort ExpireSnapshots change from Spark 4.2 to Spark 3.5, 4.0 and 4.1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
