fallintoplace opened a new pull request, #1187: URL: https://github.com/apache/arrow-go/pull/1187
## What - reserve List and ListView batches once - update parent validity in bulk - append repeated offsets and zero sizes through the unsafe builder path - cover List, LargeList, ListView, and LargeListView These bulk APIs currently call the scalar append path once per list slot. Empty and null variable-sized lists do not append child values, so only the parent validity and dimensions need updating. This follows the batching shape used by the C++ list builders: https://github.com/apache/arrow/blob/485499fd02ea2b0c323d67871fbe96aae4232504/cpp/src/arrow/array/builder_nested.h#L138-L160 For ListView, this keeps Go's current physical representation by repeating the current child offset with a zero size. ## Benchmark Apple M1 Pro, Go 1.26.3. Representative medians from three to five runs with 65,536 slots: | Builder | Operation | Before | After | Change | |---|---|---:|---:|---:| | List | nulls | 848 us | 201 us | -76% | | List | empty | 848 us | 197 us | -77% | | LargeList | nulls | 1.08 ms | 241 us | -78% | | LargeList | empty | 1.05 ms | 245 us | -77% | | ListView | nulls | 1.35 ms | 414 us | -69% | | ListView | empty | 1.24 ms | 415 us | -67% | | LargeListView | nulls | 1.47 ms | 509 us | -65% | | LargeListView | empty | 1.50 ms | 505 us | -66% | Regular-list allocations drop from 45 to 18 per batch. List-view allocations drop from 68 to 25. The `n=1` cases stay within benchmark noise. ## Tests - `go test ./arrow/array -count=1` - `go test -race ./arrow/array -run 'TestListBuilderBulkAppendNullsAndEmptyValues|TestList|TestLargeList' -count=1` - `go vet ./arrow/array` - `go test -p 1 ./...` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
