fallintoplace opened a new pull request, #1936: URL: https://github.com/apache/iceberg-go/pull/1936
## Summary - **Cache Arrow filter setup per physical file schema during one scan.** - Reuse column translation, binding, Substrait expression conversion, rewritten stats filters, and bloom predicates. - Keep row-group stats evaluator creation per file because it owns mutable row-group maps. - Build a fresh record processor for each task context. - Use a low-allocation structural key that includes nested list/map field IDs. - Keep residual task filters on the existing path. - Keep the missing initial-default pruning fallback. Pruning plans are built only for Parquet files. ⚡ ## Benchmark Command: ```text go test ./table -run '^$' -bench '^BenchmarkArrowScanFilterPlanSetup$' -benchmem -benchtime=1s -count=1 ``` Apple M1 Pro, darwin/arm64: | Case | ns/op | B/op | allocs/op | | --- | ---: | ---: | ---: | | recompile_each_file | 22,898 | 18,984 | 414 | | cache_hit | 1,821 | 1,064 | 8 | This benchmark isolates repeated filter and pruning setup from file I/O. ## Tests - `go test ./...` - `go test -race ./table -run '^TestCompiledFileFilterPlans|TestPhysicalSchemaKey|TestProcessRecords' -count=1` - `go vet ./...` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
