fallintoplace opened a new pull request, #1323: URL: https://github.com/apache/arrow-go/pull/1323
**What** - Concatenate List and LargeList offsets directly into the output buffer. **Why** - Avoid temporary fixed-width buffer wrappers for every input chunk. - Improve concatenation when arrays are split into many chunks. **Implementation** - Added `concatOffsetsData` using the existing direct 32-bit and 64-bit offset handlers. - Updated only the List and LargeList paths. Map and Binary paths are unchanged. - Added a benchmark covering both list types and 64, 1,024, and 8,192 chunks. Benchmark on an Apple M1 Pro with 65,536 values: | Case | ns/op before -> after | allocs/op before -> after | | --- | ---: | ---: | | List, 64 chunks | 86.5us -> 82.3us | 211 -> 145 | | List, 1,024 chunks | 274.1us -> 229.5us | 3,309 -> 2,273 | | List, 8,192 chunks | 1.512ms -> 1.220ms | 28,627 -> 19,656 | | LargeList, 64 chunks | 93.7us -> 87.5us | 211 -> 146 | | LargeList, 1,024 chunks | 262.6us -> 222.5us | 3,297 -> 2,254 | | LargeList, 8,192 chunks | 1.467ms -> 1.192ms | 28,244 -> 19,217 | Tests: - `go test ./arrow/array -count=1` - `go vet ./arrow/array` - `git diff --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
