fallintoplace opened a new pull request, #1182: URL: https://github.com/apache/arrow-go/pull/1182
## What does this PR do? - decodes Parquet BOOLEAN values directly into a packed bitmap - supports both PLAIN and RLE boolean pages - expands nullable values inside the bitmap without creating a `[]bool` - passes the record reader value bitmap directly to the Arrow Boolean array ## Why? The read path currently stores decoded booleans as one byte per value. It then allocates an Arrow bitmap and packs the values again in `transferBool`. This finishes wiring the direct bitmap decoder API from #707 into the pqarrow read path. The record reader now keeps boolean values packed from decoding through Arrow array construction. The change also fixes decoder state after a partial PLAIN bitmap decode, so the next call resumes at the correct source bit. ## Benchmarks Apple M1 Pro, 1M values, uncompressed Parquet, `GOMAXPROCS=1`: | case | before | after | change | before B/op | after B/op | change | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | dense | 1.48 ms | 0.15 ms | -90% | 1.61 MB | 0.56 MB | -65% | | 50% null | 6.72 ms | 4.63 ms | -31% | 4.11 MB | 3.08 MB | -25% | ```text GOMAXPROCS=1 go test ./parquet/pqarrow -run '^$' -bench '^BenchmarkBooleanBitmapRead$' -benchmem -benchtime=500ms -count=5 ``` ## Tests - added direct bitmap tests for consecutive PLAIN decode calls - added nullable bitmap tests for PLAIN and RLE with unaligned offsets - ran `go test ./parquet/...` - ran race tests for the touched encoding, file, and pqarrow packages - ran `go vet` for the touched packages -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
