fallintoplace opened a new pull request, #1174: URL: https://github.com/apache/arrow-go/pull/1174
### Rationale for this change Binary and string equality currently compares every non-null value separately. This repeats offset lookups and small byte or string comparisons even when a long valid run is stored contiguously. ### What changes are included in this PR? - Walk contiguous valid runs for Binary, String, LargeBinary, and LargeString. - Compare each run payload in one operation. - Verify every value length so equal concatenated bytes with different boundaries still compare unequal. - Keep null payloads ignored. - Sample validity fragmentation and retain the scalar path for many tiny runs. - Add benchmarks and tests for offsets, null patterns, mismatches, and value boundaries. Apple M1 Pro results for 65,536 values of 32 bytes with `-cpu=1`: | Case | Before | After | Speedup | | --- | ---: | ---: | ---: | | Binary equal | 644 us | 360 us | 1.79x | | Binary 10% null | 582 us | 439 us | 1.33x | | Binary mismatch last | 648 us | 350 us | 1.85x | | Binary different length | 453 us | 262 us | 1.73x | | String equal | 633 us | 390 us | 1.62x | | String 10% null | 604 us | 496 us | 1.22x | | String mismatch last | 664 us | 406 us | 1.64x | | String different length | 426 us | 235 us | 1.81x | The alternating 50% null case selects the scalar path and remains neutral. All benchmark cases stay at zero allocations. ### Are these changes tested? Yes. - `go test ./arrow/...` - `go test -race ./arrow/array` - `go vet -composites=false ./arrow/array` - Cross-compiled array tests for linux/amd64 and linux/s390x. ### Are there any user-facing changes? No. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
