fallintoplace opened a new pull request, #1258:
URL: https://github.com/apache/arrow-go/pull/1258
## What
- Precompute the xxHash values for `false` (`[]byte{0}`) and `true`
(`[]byte{1}`).
- Reuse them in dense and spaced boolean bitmap Bloom-filter paths.
- Keep custom `Hasher` implementations on the existing path.
This removes a `Sum64` call for every boolean while keeping the same hash
values.
## Benchmark
Apple M1 Pro, Go 1.26.3, 7 samples, one CPU:
`go test -vet=off ./parquet/metadata -run '^$' -bench
'^BenchmarkBloomFilter(HashingFromBitmap|BooleanBitmap)' -benchmem
-benchtime=150ms -count=7 -cpu=1`
The benchmark matrix covers 100K and 1M booleans, all false, all true,
alternating, random, and spaced values with 10% and 50% nulls.
Median results for alternating values:
| Case | Before | After |
| --- | --- | --- |
| Hash, 100K | 636.0 us, 802,819 B/op, 2 allocs | 181.6 us, 802,816 B/op, 1
alloc |
| Insert, 100K | 1.840 ms, 8,193 B/op, 2 allocs | 943.3 us, 8,193 B/op, 2
allocs |
| Hash, 1M | 7.206 ms, 8,003,596 B/op, 2 allocs | 1.510 ms, 8,003,584 B/op,
1 alloc |
| Insert, 1M | 18.832 ms, 8,193 B/op, 2 allocs | 9.767 ms, 8,193 B/op, 2
allocs |
For alternating data, hashing is 3.5x to 4.8x faster and insertion is about
1.9x faster.
## Tests
- `go test ./parquet/metadata`
- `PARQUET_TEST_DATA="$PWD/parquet-testing/data" go test ./...`
- `go vet ./parquet/metadata`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
