aokolnychyi commented on a change in pull request #3069:
URL: https://github.com/apache/iceberg/pull/3069#discussion_r714413810
##########
File path: spark3/src/main/java/org/apache/iceberg/spark/source/SparkWrite.java
##########
@@ -375,6 +376,14 @@ private void
commitWithSerializableIsolation(OverwriteFiles overwriteFiles,
private void commitWithSnapshotIsolation(OverwriteFiles overwriteFiles,
int numOverwrittenFiles,
int numAddedFiles) {
+ Long scanSnapshotId = scan.snapshotId();
+ if (scanSnapshotId != null) {
+ overwriteFiles.validateFromSnapshot(scanSnapshotId);
+ }
+
+ Expression conflictDetectionFilter = conflictDetectionFilter();
+ overwriteFiles.validateNoConflictingDeleteFiles(conflictDetectionFilter);
Review comment:
Consider we have a data file A and there is a copy-on-write operation
that overwrites it with a data file B. If a concurrent operation adds a delete
file that references records from the data file A, committing the original
copy-on-write operation (i.e. overwrite) would undelete the rows that were
deleted concurrently.
It seems we always have to validate the delete files whenever we overwrite
specific files during DELETE/MERGE.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
