fallintoplace opened a new pull request, #2117: URL: https://github.com/apache/iceberg-go/pull/2117
## What Skip an `IN` predicate when file bounds cannot match any value in the set. ## Why Manifest evaluation repeats member scans and allocations for every file. ## Implementation Compare file bounds with cached set extrema first. Keep exact member filtering when bounds overlap, including sparse sets. Benchmarks compare base `fa5309b` with this change (`078dce9`). Each operation evaluates 10,000 files; values are medians from 7 runs of 1 second on Go 1.26.3, darwin/arm64 (Apple M1 Pro). | IN values | File bounds | Time, base → head | B/op, base → head | Allocs/op, base → head | | ---: | --- | ---: | ---: | ---: | | 10 | Below set | 7.25 → 1.84 ms | 2,320,008 → 720,002 | 40,000 → 30,000 | | 10 | Above set | 5.89 → 1.34 ms | 2,240,006 → 640,001 | 20,000 → 10,000 | | 50 | Below set | 24.38 → 1.91 ms | 9,680,027 → 720,002 | 40,000 → 30,000 | | 50 | Above set | 19.62 → 1.38 ms | 9,600,018 → 640,001 | 20,000 → 10,000 | | 200 | Below set | 88.60 → 1.85 ms | 35,280,113 → 720,002 | 40,000 → 30,000 | | 200 | Above set | 73.06 → 1.50 ms | 35,240,105 → 680,002 | 30,000 → 20,000 | Overlap and sparse-gap controls retain exact member filtering and unchanged allocation counts. Command: ```sh go test ./table -run '^$' -bench '^BenchmarkInclusiveMetricsEvalInPredicate/int32/in=(10|50|200)/(below-set|above-set)$' -benchmem -benchtime=1s -count=7 -cpu=1 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
