abstractdog opened a new pull request, #3782:
URL: https://github.com/apache/parquet-java/pull/3782
### Rationale for this change
`Binary#equals`, `Binary#lexicographicCompare`, and `Binary#hashCode` sit on
hot paths for statistics min/max maintenance, dictionary hash-map probing,
predicate evaluation, and bloom-filter build — so any per-byte overhead is
amortized across every reader and writer. The current implementations use
hand-rolled scalar byte loops; the JDK 9+ range overloads of `Arrays.equals` /
`Arrays.compareUnsigned` / `Arrays.hashCode` / `ByteBuffer.mismatch` route
through `ArraysSupport.vectorizedMismatch`, an `@IntrinsicCandidate` helper
HotSpot substitutes with a SIMD byte-scan (SSE / AVX2 / NEON).
### What changes are included in this PR?
- Replace the scalar loops in `Binary.java` for `equals`,
`lexicographicCompare`, and `hashCode` with the intrinsic-backed `Arrays.*` /
`ByteBuffer.mismatch` APIs across all four `Binary` shape pairs
(`byte[]`/`byte[]`, `byte[]`/`ByteBuffer`, `ByteBuffer`/`ByteBuffer`).
- Add a JMH suite `parquet-benchmarks/BinaryComparisonBenchmark` covering
`equals` / `compareTo` / `hashCode` at length regimes 8, 64, and 512, both
worst-case (full match) and average-case (mid-length mismatch).
Follows the same technique already accepted for `DeltaByteArrayWriter` in
#3465. Complementary to the `Binary#hashCode` caching in #3566 — caching
reduces call frequency, intrinsics speed up the calls that remain plus `equals`
/ `compareTo` which caching does not address.
### Are these changes tested?
Yes. Covered by the existing `Binary` unit tests (equality, comparison,
hashing across all shape pairs) plus the new JMH suite. `compareTo` semantics
are preserved — `Arrays.compareUnsigned` matches the contract of the previous
loop (unsigned byte order, shorter-runs-first on prefix match).
### Are there any user-facing changes?
No API changes. Behavior is identical; only the implementation is faster.
**Benchmark** — JDK 17, 1 fork, 3×1s warmup, 5×1s measurement, throughput
(ops/s):
| bench | len | before | after | speedup |
| --------------------------- | --: | -----: | ------: | ------: |
| `equalsMatch_bytesBytes` | 512 | 9.1 M | 39.1 M | 4.30× |
| `equalsMatch_bytesBytes` | 64 | 41.4 M | 160.9 M | 3.89× |
| `equalsMismatch_bytesBytes` | 512 | 14.6 M | 55.5 M | 3.79× |
| `compareTo_bytesBytes` | 512 | 15.1 M | 58.8 M | 3.91× |
| `compareTo_bytesBytes` | 64 | 59.0 M | 199.4 M | 3.38× |
| `equalsMismatch_bytesBuf` | 64 | 63.8 M | 190.2 M | 2.98× |
Wins scale with length — at len=512 the SIMD lane count amortizes fully
(≈4×); at len=64 it's ≈3–4×; len=8 sees only 1.1–1.7× because there's barely
enough work for one SIMD lane. `hashCode` is unchanged in JDK 17 numbers — the
`31*h+b` polynomial has a serial cross-iteration dependency; HotSpot only
gained a lane-split intrinsic (`ArraysSupport.vectorizedHashCode`) in JDK 21,
so the new `Arrays.hashCode` call is equivalent on 17 and picks up the
vectorization automatically on 21+.
<!-- Closes #${GITHUB_ISSUE_ID} -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
