fallintoplace opened a new pull request, #2118: URL: https://github.com/apache/iceberg-go/pull/2118
## What - Avoid split-offset copies during internal Parquet planning and scan reads. ## Why - Split tasks repeatedly clone the same file's offsets through `SplitOffsets()`. ## Implementation - Add a narrow internal borrow accessor without initializing statistics maps. - Custom DataFiles still use their public `SplitOffsets()` getter. Benchmarks compare `fa5309b` → `a71b303` with the same benchmark files on both commits. Go 1.26.3, darwin/arm64, Apple M1 Pro, CPU=1. Medians from 14 one-second samples per case, run in base/head/head/base order. Planning evaluates 1,024 files/op. The scan is a synthetic 512-row file with 512 row-group tasks/op. | Path | Offsets/file | Time, base → head | B/op, base → head | Allocs/op, base → head | | --- | ---: | ---: | ---: | ---: | | Planning | 1 | 27.67 → 10.99 µs | 8,192 → 0 | 1,024 → 0 | | Planning | 32 | 1.246 → 1.200 ms | 6,774,784 → 6,512,640 | 2,053 → 1,029 | | Planning | 128 | 2.078 → 2.014 ms | 21,454,848 → 20,406,272 | 2,053 → 1,029 | | Planning | 512 | 6.421 → 6.003 ms | 81,223,680 → 77,029,376 | 2,053 → 1,029 | | Scan | 512 | 39.009 → 38.641 ms | 37,001,984 → 34,904,420 | 181,079 → 180,567 | Command (run twice per commit, with the order reversed): ```sh go test ./table -run '^$' -bench '^(BenchmarkSplitRemoteScanTasksSplitOffsets|BenchmarkArrowScanPlanningSplitOffsets)$' -benchmem -benchtime=1s -count=7 -cpu=1 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
