aokolnychyi commented on code in PR #8172:
URL: https://github.com/apache/iceberg/pull/8172#discussion_r1277173170
##########
core/src/test/java/org/apache/iceberg/TestDeleteFileIndex.java:
##########
@@ -127,43 +175,40 @@ public void testUnpartitionedDeletes() {
DataFile partitionedFileA = FILE_A.copy();
((BaseFile<?>) partitionedFileA).setSpecId(1);
Assert.assertArrayEquals(
- "All global deletes should apply to a partitioned file",
- DELETE_FILES,
+ "All global equality deletes should apply to a partitioned file",
+ Arrays.copyOfRange(deleteFiles, 0, 2),
Review Comment:
The old test specified the indexed delete files directly and was broken. It
assumed global deletes can be position deletes. It is not how the delete index
builder works. Position deletes are always scoped to a particular partition
spec, unlike equality deletes.
Here is a quote from the spec:
```
A position delete file must be applied to a data file when all of the
following are true:
- The data file’s data sequence number is less than or equal to the delete
file’s data sequence number
- The data file’s partition (both spec and partition values) is equal to the
delete file’s partition
An equality delete file must be applied to a data file when all of the
following are true:
- The data file’s data sequence number is strictly less than the delete’s
data sequence number
- The data file’s partition (both spec and partition values) is equal to the
delete file’s partition or the delete file’s partition spec is unpartitioned
```
##########
core/src/test/java/org/apache/iceberg/TestDeleteFileIndex.java:
##########
@@ -127,43 +175,40 @@ public void testUnpartitionedDeletes() {
DataFile partitionedFileA = FILE_A.copy();
((BaseFile<?>) partitionedFileA).setSpecId(1);
Assert.assertArrayEquals(
- "All global deletes should apply to a partitioned file",
- DELETE_FILES,
+ "All global equality deletes should apply to a partitioned file",
+ Arrays.copyOfRange(deleteFiles, 0, 2),
Review Comment:
The old test specified the indexed delete files directly and was broken. It
assumed global deletes can be position deletes. It is not how the delete index
builder works. Position deletes are always scoped to a particular partition
spec, unlike equality deletes.
Here is a quote from the spec:
```
A position delete file must be applied to a data file when all of the
following are true:
- The data file’s data sequence number is less than or equal to the delete
file’s data sequence number
- The data file’s partition (both spec and partition values) is equal to the
delete file’s partition
An equality delete file must be applied to a data file when all of the
following are true:
- The data file’s data sequence number is strictly less than the delete’s
data sequence number
- The data file’s partition (both spec and partition values) is equal to the
delete file’s partition or the delete file’s partition spec is unpartitioned
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
