hameizi commented on pull request #3095: URL: https://github.com/apache/iceberg/pull/3095#issuecomment-1017216735
> 1. flink writing cdc data to iceberg will ignore UPDATE_BEFORE and treat UPDATE_AFTER as retract message, it's completely not correct. The relevant codes are as follows: For this issue, config `write.upsert.enabled` is not suitable for CDC. So if your scene is CDC you can config it false that result `upsert` false. > 2\. when there is only delete operation during the period of one snapshot，the snapshot of iceberg will contains only equality delete files and no data file(data file is deleted), flink will ignore equality delete files and miss all -D . function org.apache.iceberg.ManifestGroup.planStreamingFiles() create FileScanTask only when data file exists, so flink can not process equality delete files. I will do more test for this issue. Thanks for your feedback. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
