zheliu2 opened a new pull request, #15571: URL: https://github.com/apache/iceberg/pull/15571
Fixes #14699 ## Summary When `replacePartitions` mode is enabled and the write result contains delete files (e.g., from equality delete writers mixed with overwrite mode), the Flink committer throws `IllegalStateException: Cannot overwrite partitions with delete files`. This is overly aggressive because delete files are not applicable during a dynamic partition overwrite -- the entire partition is being replaced. This change: - Replaces the hard `Preconditions.checkState` with a warning log when delete files are present (they are safely dropped since partition replacement makes them unnecessary) - Adds an early return when there are no data files to commit (empty table or delete-only scenario), avoiding `Cannot determine partition specs: no data files have been added` - Applies the fix to `IcebergCommitter` and `IcebergFilesCommitter` across all Flink versions (v1.20, v2.0, v2.1) ## Test plan - Added `testReplacePartitionsWithDeleteFiles` -- verifies overwrite succeeds when both data files and delete files are present (delete files are dropped) - Added `testReplacePartitionsOnEmptyTableWithDeleteFiles` -- verifies overwrite succeeds when only delete files are present with no data files (the commit is skipped) - All 288 existing `TestIcebergCommitter` tests continue to pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
