Ayoubhm07 opened a new pull request, #4004: URL: https://github.com/apache/iceberg-python/pull/4004
**What changed** While profiling In / NotIn, I noticed that the literal set was being built twice. In.__new__ already creates the set to decide whether the predicate can collapse to AlwaysFalse / EqualTo, but SetPredicate.__init__ rebuilt it immediately afterwards. For 100k values, that means creating 200k Literal models instead of 100k. I changed __new__ to pass the set it already built to the instance, and made SetPredicate.__init__ return early when it is already populated. This follows the same pattern already used by And. I also moved bound_term.ref().field.field_type outside the bind() comprehension so it is resolved once instead of once per literal. **Result** On 100k values: 1k -15% 10k -20% 100k -42% The 100k case went from 1.91s → 1.11s in my benchmark. The gain is mainly in predicate construction. bind() still has to convert each literal because those conversions are semantically meaningful, so I would not claim this patch makes binding faster end-to-end. **Scope** This only targets the duplicated work in In / NotIn. It does not fix the larger pruning issue from #3129: above IN_PREDICATE_LIMIT, file pruning is still disabled. That's a separate change. **Validation** Existing expression tests pass unchanged: 582 passed before and after. ruff passes, and mypy shows the same pre-existing findings. I also checked the constructor/deserialization paths, including duplicates, empty/single-value predicates, values=, JSON round-trips, pickling and inversion. No intentional user-facing behavior changes. AI disclosure: I used Claude to help profile the path and draft the change. I ran the benchmarks and reviewed the implementation myself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
