fallintoplace opened a new pull request, #1181: URL: https://github.com/apache/arrow-go/pull/1181
## Summary - Validate nullable dictionary indices using contiguous valid runs. - Keep the existing whole-slice min/max path for arrays without nulls. - Fall back to the whole-slice path for fragmented validity when all physical values are already in range. - Ignore out-of-range physical values at null positions. ## Why The current bounds check scans every physical index, including null positions. This does unnecessary work for clustered nullable arrays and can reject garbage payloads that are hidden by the validity bitmap. The existing TODO points to the set-bit run reader for this case. This change uses it for up to eight valid runs. More fragmented bitmaps keep the vectorized whole-slice fast path. If that scan finds an invalid physical value, valid runs are checked again so null payloads are still ignored. ## Benchmark 1M int32 indices on an Apple M1 Pro, 300 ms per sample, 6 samples: | Case | Before | After | Change | | --- | ---: | ---: | ---: | | all valid | 88.96 us/op | 92.68 us/op | no significant change | | 10% valid, clustered | 88.08 us/op | 45.53 us/op | 48.3% faster | | 50% valid, clustered | 90.51 us/op | 81.28 us/op | 10.2% faster | | alternating | 86.83 us/op | 88.58 us/op | no significant change | All cases remain at 0 allocations/op. ## Checks - `go test ./arrow/...` - `go vet ./arrow/array` - `git diff --check` No public API changes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
