fallintoplace opened a new pull request, #1404: URL: https://github.com/apache/arrow-go/pull/1404
## What Decode 1-bit and 2-bit Parquet levels directly into `int16` output. ## Why The current path unpacks values into `uint32` scratch space before converting them to `int16`. ## Implementation - Expand narrow packed bytes directly into the output. - Keep the existing path for wider levels and trailing values. - Add parity coverage for slices, truncated data, and sequential reads. ## Benchmark Paired benchmark of the copied previous `GetBatchLevels` implementation and the actual updated `BitReader.GetBatchLevels` method. Go 1.26.3, Apple M1 Pro (`darwin/arm64`), `GOMAXPROCS=1`; 10 runs of 150 ms per case. | Metric | Before | After | Change | | --- | ---: | ---: | ---: | | 1-bit, mostly defined, 1,024 levels | 742.9 ns/op | 324.6 ns/op | -56.32% | | 1-bit, mostly defined, 65,536 levels | 43.84 µs/op | 17.50 µs/op | -60.09% | | 1-bit, alternating, 1,024 levels | 740.1 ns/op | 330.5 ns/op | -55.34% | | 1-bit, alternating, 65,536 levels | 44.45 µs/op | 17.43 µs/op | -60.80% | | 1-bit, varied, 1,024 levels | 744.3 ns/op | 325.4 ns/op | -56.28% | | 1-bit, varied, 65,536 levels | 43.98 µs/op | 17.75 µs/op | -59.64% | | 2-bit, mostly defined, 1,024 levels | 753.7 ns/op | 572.3 ns/op | -24.07% | | 2-bit, mostly defined, 65,536 levels | 45.10 µs/op | 30.10 µs/op | -33.25% | | 2-bit, alternating, 1,024 levels | 753.9 ns/op | 521.0 ns/op | -30.90% | | 2-bit, alternating, 65,536 levels | 45.32 µs/op | 29.97 µs/op | -33.85% | | 2-bit, varied, 1,024 levels | 756.7 ns/op | 525.6 ns/op | -30.54% | | 2-bit, varied, 65,536 levels | 44.88 µs/op | 31.11 µs/op | -30.69% | | Geometric mean, 12 cases | 5.777 µs/op | 3.114 µs/op | -46.10% | All cases improved. Both paths use 48 B/op and 1 alloc/op. Command: `GOMAXPROCS=1 go test ./parquet/internal/utils -run '^$' -bench '^BenchmarkPackedLevelDecode$' -benchmem -benchtime=150ms -count=10 -cpu=1`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
