rdblue commented on code in PR #18546:
URL: https://github.com/apache/iceberg/pull/18546#discussion_r4331667798
##########
core/src/main/java/org/apache/iceberg/BaseOverwriteFiles.java:
##########
@@ -160,28 +173,29 @@ protected void validate(TableMetadata base, Snapshot
parent) {
}
}
+ Expression validationFilter = validationFilter();
+
if (validateNewDataFiles) {
- validateAddedDataFiles(base, startingSnapshotId,
dataConflictDetectionFilter(), parent);
+ validateAddedDataFiles(base, startingSnapshotId, validationFilter,
parent);
}
if (validateNewDeletes) {
if (rowFilter() != Expressions.alwaysFalse()) {
- Expression filter = conflictDetectionFilter != null ?
conflictDetectionFilter : rowFilter();
- validateNoNewDeleteFiles(base, startingSnapshotId, filter, parent);
- validateDeletedDataFiles(base, startingSnapshotId, filter, parent);
+ validateNoNewDeleteFiles(base, startingSnapshotId, validationFilter,
parent);
+ validateDeletedDataFiles(base, startingSnapshotId, validationFilter,
parent);
}
if (!deletedDataFiles.isEmpty()) {
validateNoNewDeletesForDataFiles(
- base, startingSnapshotId, conflictDetectionFilter,
deletedDataFiles, parent);
+ base, startingSnapshotId, validationFilter, deletedDataFiles,
parent);
Review Comment:
The current behavior is that `rowFilter()` is never passed here. Either
`conflictDetectionFilter` is passed or it is null, which results in not
configuring a filter (equivalent to `alwaysTrue()`).
The new behavior is that `validationFilter()` method will return
`rowFilter()` if it is set regardless of whether `deletedDataFiles` is empty.
So if `rowFilter()` is set and there are deleted data files, it will be passed
here.
The new precondition disallows deleting specific files using
`deleteFile(DataFile)` when `rowFilter` is set to avoid a behavior change here,
but is a behavior change for the API.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
