karubian opened a new issue, #19035: URL: https://github.com/apache/iceberg/issues/19035
### Apache Iceberg version 1.12.0 (latest release) ### Query engine Spark ### Please describe the bug 🐞 `rewrite_position_delete_files` can drop position delete entries whose data file is still live, when a manifest partition summary wrongly excludes that data file's partition. The rewrite then removes the input delete file, so the deleted rows reappear for every reader that plans the data file, such as an unfiltered scan. Such a summary violates the spec, since its bounds must cover the manifest's partition values, [Manifest Lists](https://iceberg.apache.org/spec/#manifest-lists), but Iceberg Java wrote them before 1.11.0. In #15128, a manifest filter or merge that copies entries on a table with a binary partition field records bounds from a byte array the manifest reader reuses: [PartitionData](https://github.com/apache/iceberg/blob/apache-iceberg-1.10.0/core/src/main/java/org/apache/iceberg/PartitionData.java#L137-L139), [filter](https://github.com/apache/iceberg/blob/apache-iceberg-1.10.0/core/src/main/java/org/apache/iceberg/ManifestFilterManager.java#L486-L536), [merge](https://github.com/apache/iceberg/blob/apache-iceberg-1.10.0/core/src/main/java/org/apache/iceberg/ManifestMergeManager.java#L171-L188). #15127 fixed new writes in 1.11.0, but later snapshots keep the wrong summary as long as they reuse the manifest, so a 1.12.0 rewrite still drops the deletes. On Spark 3.5.3 with a Hadoop catalog, the first five statements on Iceberg 1.10.0, then the Spark session stopped and started again with the 1.12.0 runtime on the same warehouse: ```sql -- Iceberg 1.10.0 CREATE TABLE t (id bigint NOT NULL, data binary) USING iceberg PARTITIONED BY (data) TBLPROPERTIES ('format-version' = '2', 'write.delete.mode' = 'merge-on-read'); INSERT INTO t VALUES (1, X'e3bcd1'), (2, X'e3bcd1'), (3, X'bcd1'); DELETE FROM t WHERE data = X'bcd1'; -- metadata delete; the manifest summary becomes [bcd1, bcd1d1] DELETE FROM t WHERE id = 1; -- one position delete for the e3bcd1 data file SELECT * FROM t; -- (2, X'e3bcd1') -- Iceberg 1.12.0, same warehouse CALL system.rewrite_position_delete_files(table => 't', options => map('rewrite-all', 'true')); -- rewritten_delete_files_count 1, added_delete_files_count 0 SELECT * FROM t; -- (1, X'e3bcd1'), (2, X'e3bcd1') ``` Row 1 is back and the table has no delete file left. `rewrite-all` makes the planner pick the single small delete file. A filtered read, `WHERE data = X'e3bcd1'`, returns nothing before and after the rewrite, because the wrong summary still prunes the manifest. Running every statement on 1.10.0 gives the same result. Running every statement on 1.12.0 writes a right summary and the rewrite keeps the delete. The runner keeps only the position delete entries whose `file_path` appears in a `data_files` scan filtered to the file group's partition: [SparkRewritePositionDeleteRunner](https://github.com/apache/iceberg/blob/apache-iceberg-1.12.0/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/SparkRewritePositionDeleteRunner.java#L113-L149). That scan skips manifests by their partition summaries, [BaseFilesTable](https://github.com/apache/iceberg/blob/apache-iceberg-1.12.0/core/src/main/java/org/apache/iceberg/BaseFilesTable.java#L59-L87) and [ManifestEvaluator](https://github.com/apache/iceberg/blob/apache-iceberg-1.12.0/api/src/main/java/org/apache/iceberg/expressions/ManifestEvaluator.java#L245-L265), as [Scan Planning](https://iceberg.apache.org/spec/#scan-planning) allows. A live data file in a skipped manifest looks gone, so its entries are dropped as dangling, although the spec deletes a row when "there is an entry in a position delete file for the row's file and posit ion": [Position Delete Files](https://iceberg.apache.org/spec/#position-delete-files). Only file groups whose data file sits in a skipped manifest are affected. `rewrite_manifests` repairs filtered reads when it rewrites the manifests, since new manifests get summaries from their entries, but it leaves a single data manifest as it is, as in this table, and a delete the rewrite has dropped is missing from every later snapshot; rolling back to an earlier snapshot, or adding the old delete file back while it still exists, restores it. `RemoveDanglingDeleteFilesAction` and `RemoveDanglingDeletesSparkAction` plan unfiltered scans, [core](https://github.com/apache/iceberg/blob/apache-iceberg-1.12.0/core/src/main/java/org/apache/iceberg/actions/RemoveDanglingDeleteFilesAction.java#L172-L173) and [Spark](https://github.com/apache/iceberg/blob/apache-iceberg-1.12.0/spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/actions/RemoveDanglingDeletesSparkAction.java#L89-L90), and keep these d eletes. The runner on main is the same for Spark 3.5, 4.0, 4.1 and 4.2, and the same filter decides which entries survive when it converts v2 position deletes to deletion vectors on a table upgraded to v3. ### Willingness to contribute - [x] I can contribute a fix for this bug independently - [ ] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
