fallintoplace opened a new pull request, #1183: URL: https://github.com/apache/arrow-go/pull/1183
## What does this PR do? - reserves fixed-size list parent slots once in `AppendNulls` and `AppendEmptyValues` - updates the parent validity bitmap in bulk - appends all child null or empty values with one bulk builder call The child append count is `listSize * n`, so the parent and child lengths stay in sync. ## Why? Both methods currently loop over every parent value. Each parent append also loops over every child value. This removes the parent loop and lets child builders use their bulk append paths. The change is independent, but it also composes with #1173 and #1180. ## Benchmarks Apple M1 Pro, 65,536 `FixedSizeList<Int32>` rows, `GOMAXPROCS=1`. Standalone on `main`: | list width | nulls before | nulls after | change | empty before | empty after | change | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 4 | 1.10 ms | 0.86 ms | -22% | 1.55 ms | 1.07 ms | -31% | | 16 | 3.76 ms | 2.74 ms | -27% | 5.43 ms | 3.90 ms | -28% | | 64 | 13.03 ms | 11.07 ms | -15% | 19.35 ms | 14.86 ms | -23% | With #1173 and #1180 applied underneath: | list width | nulls before | nulls after | change | empty before | empty after | change | | ---: | ---: | ---: | ---: | ---: | ---: | ---: | | 4 | 1.56 ms | 0.18 ms | -88% | 2.01 ms | 0.20 ms | -90% | | 16 | 4.69 ms | 0.33 ms | -93% | 6.89 ms | 0.42 ms | -94% | | 64 | 16.27 ms | 0.74 ms | -95% | 22.18 ms | 0.92 ms | -96% | ```text GOMAXPROCS=1 go test ./arrow/array -run '^$' -bench '^BenchmarkFixedSizeListBuilderBulkAppend$/(nulls|empty)/rows=65536/width=(4|16|64)$' -benchmem -benchtime=300ms -count=5 ``` ## Tests - added coverage for unaligned parent validity bits - verifies parent validity and null counts - verifies child length, null slots, and zero-valued empty slots - ran `go test ./arrow/...` - ran `go test -race ./arrow/array` - ran `go vet ./arrow/array` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
