fallintoplace opened a new pull request, #2380: URL: https://github.com/apache/iceberg-go/pull/2380
## What - Group single-field identity partition values in equality-delete conflict filters. - Balance the remaining OR terms. ## Why - Large sets of partitions create deep expressions and expensive schema binding. ## Implementation - Group values by partition spec and field, with at most 200 values per `IN` predicate. - Keep small inputs, NaNs, binary values, and values requiring casts on the equality path. - Preserve null handling and conservative handling of non-identity transforms. ## Benchmark Predicate construction and binding only, Apple M1 Pro, Go 1.26.3, median of 3 runs, `GOMAXPROCS=1`: | Identity partitions | Before | After | | --- | ---: | ---: | | 1 | 2.97 µs | 3.00 µs | | 16 | 13.97 µs | 14.28 µs | | 1,000 | 0.756 ms | 0.514 ms | | 4,096 | 3.05 ms | 2.01 ms | These timings exclude manifest reads and commit I/O. The 16-partition difference is small relative to run-to-run variation. `GOMAXPROCS=1 go test ./table -run '^$' -bench '^BenchmarkEqDeleteConflictPredicateScale$' -benchmem -benchtime=150ms -count=3 -cpu=1` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
