zhengxiang378928908-code opened a new pull request, #18844: URL: https://github.com/apache/iceberg/pull/18844
Fixes #18843. With `overwrite(true)`, `DynamicCommitter` added only the data files of each write result to the `ReplacePartitions` commit, so delete files written for upserts or CDC deletes were silently dropped. It now fails before committing when delete files are present, matching the check in `IcebergCommitter`. Unlike `IcebergSink`, the dynamic sink cannot reject overwrite with upsert when the job is built, because upsert mode is set per record. This PR changes Flink 2.3 only; backports to 1.20 and 2.2 can follow. Test plan: the new `TestDynamicCommitter.replacePartitionsRejectsDeleteFiles` fails with `DynamicCommitter.java` reverted to `main` and passes with this change. `TestDynamicCommitter` (15 tests) and `TestDynamicIcebergSink` (49 tests) pass, as do `spotlessCheck`, `checkstyleMain` and `checkstyleTest`. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: partially reviewed - Prompt Summary: Fix #18843 by rejecting delete files in DynamicIcebergSink overwrite commits. - Generated parts: the fix in `DynamicCommitter`, the new test, the test runs and this description were done with AI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
