mbutrovich opened a new pull request, #2781: URL: https://github.com/apache/iceberg-rust/pull/2781
## Which issue does this PR close? - Closes #2780. ## What changes are included in this PR? Equality deletes are applied by building a predicate that keeps rows not matched by any delete row. For each equality column the keep term was `col != v`, produced by negating the equality match. Applied as a row filter, `null != v` evaluates to null, so a data row whose equality column is null was dropped. Per the spec ([Equality Delete Files](https://iceberg.apache.org/spec/#equality-delete-files)), a null value matches only a null delete value, so such rows should be kept. This builds the keep predicate directly in `parse_equality_deletes_record_batch_stream`: `col IS NULL OR col != v` for a non-null delete value, and `col IS NOT NULL` for a null delete value. Rows with a null value in an equality column are no longer deleted. ## Are these changes tested? Yes, unit tests in `caching_delete_file_loader` covering the predicate construction: - `test_equality_delete_predicate_preserves_null_rows`: a non-null delete value on a nullable column produces `(col IS NULL) OR (col != v)`. - `test_equality_delete_predicate_matches_null_delete_value`: a null delete value produces `col IS NOT NULL`. - `test_equality_delete_predicate_multiple_columns`: per-column keep terms are OR-ed. - `test_equality_delete_predicate_multiple_delete_rows`: per-row keep predicates are AND-ed. - `test_delete_file_loader_parse_equality_deletes` (existing, updated): the combined case with multiple columns, a required column, and null delete values. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
