mbutrovich opened a new pull request, #1778: URL: https://github.com/apache/iceberg-rust/pull/1778
## Rationale for this change This PR fixes a bug in delete file loading when a `FileScanTask` contains both positional and equality delete files. We hit this when running Iceberg Java test suite via Comet in https://github.com/apache/datafusion-comet/pull/2528. I also confirmed that it fixes the tests in Iceberg Java's suite. **The Bug:** The condition in `try_start_eq_del_load` (delete_filter.rs:71-73) was inverted. It returned `None` when the equality delete file was not in the cache, causing the loader to skip loading it. When `build_equality_delete_predicate` was later called, it would fail with "Missing predicate for equality delete file". ## What changes are included in this PR? **The Fix:** - Inverted the condition so it returns `None` when the file is already in the cache (being loaded or loaded), preventing duplicate work across concurrent tasks - When the file is not in the cache, mark it as Loading and proceed with loading **Additional Changes:** - Added test case `test_load_deletes_with_mixed_types` that reproduces the bug scenario ## Are these changes tested? Yes, this PR includes a new unit test `test_load_deletes_with_mixed_types` that: - Creates a `FileScanTask` with both a positional delete file and an equality delete file - Verifies that `load_deletes` successfully processes both types - Verifies that `build_equality_delete_predicate` succeeds without the "Missing predicate" error - We hit this when running Iceberg Java test suite via Comet in https://github.com/apache/datafusion-comet/pull/2528. I also confirmed that it fixes the tests in Iceberg Java's suite. The test would fail before this fix and passes after. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
