paultmathew commented on PR #3387: URL: https://github.com/apache/iceberg-python/pull/3387#issuecomment-4539710229
@abnobdoss You're right. My goal was to replace the Or(And(EqualTo, ...)) predicate with an O(1) range so the projection visitor would do less work, but I didn't account for the fact that inclusive_projection walks the full N-disjunct tree regardless of what I AND onto it — so the augmentation adds work rather than substituting for it, and create_match_filter's own per-disjunct projection is already producing the prune target. I'll drop the partition augmentation and keep just the selected_fields=tuple(join_cols) change. The column-projection win is independent of partition spec and stands on its own. The underlying gap that motivated the augmentation — large flushes against partition-source-not-in-join-cols tables hitting full-table scans — is a separate problem that needs a different primitive. Equality deletes by primary key are the cleanest Iceberg-native fit for it, so I'll follow up on that thread. Will trim the diff and update the PR description shortly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
