azwanzuharimi commented on issue #4003: URL: https://github.com/apache/iceberg-python/issues/4003#issuecomment-5916360468
I am working on this and will open a PR this week. Plan: - Keep the exact literal check when the IN list has 200 values or fewer. - Above 200 values, compare only `min(literals)` and `max(literals)` against the file lower and upper bounds. - Apply the same change to the strict visitor, which has the same early return. - Add tests for the 201 value boundary on both visitors. This relates to #4027. That PR makes the upsert match filter flat, but the planner still receives a large IN, so pruning stays off without this fix. @Ayoubhm07 @avinash-git-24 tell me if you already have a branch for this, so we do not do the work twice. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
