fallintoplace opened a new pull request, #1172: URL: https://github.com/apache/arrow-go/pull/1172
### Rationale for this change `BYTE_STREAM_SPLIT` decoding for `FIXED_LEN_BYTE_ARRAY` currently allocates a separate byte slice for every output value when the output has no reusable capacity. A 65,536-value batch therefore performs 65,536 decoder allocations. ### What changes are included in this PR? - Allocate one contiguous backing block for output values that need storage. - Keep reusing output slices that already have enough capacity. - Give each decoded value an exact-capacity slice. - Only prepare the output prefix that is actually decoded. - Add coverage for contiguous storage, mixed reusable output, output lifetime across decoder resets, partial output, and spaced decoding. - Add cold-output benchmarks for widths 2, 4, 8, 16, and 32. Apple M1 Pro, GOMAXPROCS=1, 65,536 values: | Width | Before | After | Change | Allocations | | ---: | ---: | ---: | ---: | ---: | | 16 | 2.42 ms | 1.55 ms | -35.7% | 65,537 to 2 | | 32 | 3.82 ms | 2.61 ms | -31.7% | 65,537 to 2 | Bytes allocated stay unchanged. The existing reusable-output benchmark also remains allocation-free and improved by about 8%. ### Are these changes tested? - `PARQUET_TEST_DATA=$PWD/parquet-testing/data go test ./parquet/...` - `go test -race ./parquet/internal/encoding` - `go vet -composites=false ./parquet/internal/encoding` - Cross-compiled the encoding package tests for linux/amd64 and linux/s390x. ### Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
