iceberg-kitten opened a new pull request, #1626:
URL: https://github.com/apache/arrow-go/pull/1626

   ### What changes
   
   Recycle the compressed-input and decoded-data scratch buffers after closing 
a Parquet column chunk reader. Readers from the same file can reuse them when 
using Arrow's Go allocator. This applies to unencrypted reads and caches pairs 
only when each buffer is at most 1 MiB. Dictionary-page buffers keep their 
current lifetime because callers can request them independently of the current 
page.
   
   ### Benchmark
   
   `go test ./parquet/pqarrow -run '^$' -bench 
'^BenchmarkReadNumericRowGroups$' -benchmem -benchtime=650ms -cpu=1`
   
   The benchmark reads 262,144 rows from 32 Snappy row groups. Go 1.26.3 on 
Windows amd64, one pinned performance core, medians of six alternating runs 
against `4f4ac82`.
   
   | Metric | Base | This branch |
   | --- | ---: | ---: |
   | Elapsed time | 66.69 ms | 62.47 ms |
   | Allocated bytes/op | 40.03 MB | 33.43 MB |
   | Allocations/op | 8,913 | 8,553 |
   
   This branch was faster in all six pairs, with 6% lower median latency and 
16% fewer allocated bytes on this fixture.
   
   ### Correctness
   
   `TestReadNumericRowGroupsRetainedBatch` exercises parallel column decoding 
and verifies that a record retained by the caller still has the correct values 
after reading the remaining row groups and closing the file.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to