iceberg-kitten opened a new pull request, #1626: URL: https://github.com/apache/arrow-go/pull/1626
### What changes Recycle the compressed-input and decoded-data scratch buffers after closing a Parquet column chunk reader. Readers from the same file can reuse them when using Arrow's Go allocator. This applies to unencrypted reads and caches pairs only when each buffer is at most 1 MiB. Dictionary-page buffers keep their current lifetime because callers can request them independently of the current page. ### Benchmark `go test ./parquet/pqarrow -run '^$' -bench '^BenchmarkReadNumericRowGroups$' -benchmem -benchtime=650ms -cpu=1` The benchmark reads 262,144 rows from 32 Snappy row groups. Go 1.26.3 on Windows amd64, one pinned performance core, medians of six alternating runs against `4f4ac82`. | Metric | Base | This branch | | --- | ---: | ---: | | Elapsed time | 66.69 ms | 62.47 ms | | Allocated bytes/op | 40.03 MB | 33.43 MB | | Allocations/op | 8,913 | 8,553 | This branch was faster in all six pairs, with 6% lower median latency and 16% fewer allocated bytes on this fixture. ### Correctness `TestReadNumericRowGroupsRetainedBatch` exercises parallel column decoding and verifies that a record retained by the caller still has the correct values after reading the remaining row groups and closing the file. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
