rajendrag opened a new pull request, #18673: URL: https://github.com/apache/iceberg/pull/18673
Metadata deletes using `<`or `<=` can incorrectly remove NULL-partition files when they share a manifest with matching files. This returns `false` for those comparisons on NULL partition values. A related PR #18617 adds guards in Spark 4.2; and this fixes the underlying residual evaluator independently of Spark. Fixes #18672. **Test plan:** - Verify `<` and `<=` return false residuals for NULL date partitions. - Verify metadata deletes preserve NULL-partition files in shared and separate manifests across format versions and branches. - Residual and delete-file suites: 125 passed, 18 skipped. Spotless and Checkstyle passed. **AI Disclosure** - [X] AI generated The issue was observed in our prod env and human was able to reproduce. AI was used in fixing the bug, adding tests and regression. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
