karubian opened a new pull request, #19056: URL: https://github.com/apache/iceberg/pull/19056
Fixes #19055. Iceberg before 1.11.0 can write manifest list bounds for binary and fixed partition fields that leave out the manifest's own values, #15128, and later snapshots keep those manifests. Planning and commits still skip manifests by those bounds, so filtered scans lose delete files or live data files, filtered writes bring deleted rows back or miss rows, and dynamic overwrite keeps the replaced rows. `ManifestEvaluator` and `ManifestFileUtil.canContainAny` no longer compare values with the bounds of binary and fixed fields, and still prune fields that hold only nulls. Tables partitioned by those types lose manifest pruning on them. This also covers the `rewrite_position_delete_files` case of #19035. Test plan: for binary and fixed partition fields, `ManifestEvaluator` reads a manifest whose bounds exclude the value under every comparison and set predicate, and still prunes fields holding only nulls and by another partition field; `ManifestFileUtil.canContainAny` keeps a partition outside the bounds and still prunes against null-only fields. The bounds cases fail without the change. --- **AI Disclosure** - Model: Claude Opus 5.5 - Platform/Tool: Claude Code - Human Oversight: [unknown - human to fill in] - Prompt Summary: Investigate why filtered scans on a binary-partitioned table written by Iceberg 1.10 return deleted rows, then write regression tests and a change to `ManifestEvaluator` and `ManifestFileUtil`. The code change, the tests and this description are generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
