LantaoJin opened a new pull request, #16807:
URL: https://github.com/apache/lucene/pull/16807
### Description
Since #15148, `PanamaVectorUtilSupport` computes `dotProduct(byte[],
byte[])` and `uint8DotProduct` with the same methods, and `squareDistance` and
`uint8SquareDistance` too. A `boolean signed` is passed down to the
128/256/512-bit loops, which choose their widening conversion with `signed ?
B2S : ZERO_EXTEND_B2S` (and the same for `S2I` and `B2I`). This is fine while a
JVM uses only one of the two kernels. Once it has used both, C2 compiles the
shared methods for both values of the flag, and the kernel that was used second
stays slow for the life of the JVM:
- On aarch64 (NEON, 128-bit) the conversion operator is no longer a constant
in the compiled loop, so the Vector API conversion is not intrinsified
(`-XX:+PrintIntrinsics` shows `missing constant ... etype_to=Phi` and
`VectorSupport::convert ... failed to inline (intrinsic)`) and runs the Java
fallback. The second kernel is **65x to 150x slower**.
- On x86 with 512-bit vectors the conversions are still intrinsified, but
the shared loop is recompiled out of line and its callers no longer inline it
(`already compiled into a big method`). The second kernel is **4x to 5x
slower**.
This is easy to hit. `Lucene104ScalarQuantizedVectorScorer` scores
`UNSIGNED_BYTE` (8-bit) fields with `uint8DotProduct` and `SEVEN_BIT` fields
with `dotProduct`, so a JVM that searches both kinds of field mixes the two
kernels; `KnnByteVectorField` uses the signed ones. JMH benchmarks do not show
it, because JMH forks a new JVM for each benchmark.
This PR gives the two kernels separate methods all the way down.
`dotProductBody`, `squareDistanceBody` and their per-width loops are again
exactly the code from before #15148. The unsigned kernels get their own
`uint8DotProductBody` and `uint8SquareDistanceBody` with their own copies of
the loops, which use the `ZERO_EXTEND_*` conversions. Each public method calls
one or the other.
**Reproduction**, with `--add-modules jdk.incubator.vector`:
```java
byte[] a = new byte[768], b = new byte[768];
new Random(42).nextBytes(a);
new Random(43).nextBytes(b);
long sink = 0;
for (int i = 0; i < 2_000_000; i++) sink += VectorUtil.uint8DotProduct(a, b);
for (int round = 0; round < 10; round++) {
long start = System.nanoTime();
for (int i = 0; i < 100_000; i++) sink += VectorUtil.dotProduct(a, b);
System.out.printf("dotProduct: %.1f ns/op%n", (System.nanoTime() - start)
/ 100_000.0);
}
```
On aarch64 this prints about 21,300 ns/op in every round before this change,
and 339 ns/op after.
**Benchmark,** 768-dimension vectors, ns per call of the second kernel once
it is steady, JDK 25. The x86 machine is a Xeon Platinum 8175M; it uses 256-bit
vectors by default, so the 512-bit rows use `-XX:UseAVX=3`.
| machine | first kernel, then second kernel | first kernel alone | second,
before | second, after |
|---|---|---|---|---|
| aarch64 (Neoverse-N1) | `uint8DotProduct`, then `dotProduct` | 332-343 |
21,468 | 343 |
| aarch64 (Neoverse-N1) | `dotProduct`, then `uint8DotProduct` | 334-344 |
50,087 | 342 |
| x86, 512-bit | `uint8DotProduct`, then `dotProduct` | 59-65 | 271-272 |
59-65 |
| x86, 512-bit | `dotProduct`, then `uint8DotProduct` | 59-61 | 327-328 |
61-64 |
| x86, 512-bit | `uint8SquareDistance`, then `squareDistance` | 73-80 |
217-224 | 73-80 |
| x86, 512-bit | `squareDistance`, then `uint8SquareDistance` | 74-80 |
258-262 | 73-81 |
With 256-bit vectors on x86, the second kernel runs at 109-122 ns before and
after this change. On aarch64, square distance did not slow down in my runs,
only because its loop happens to be inlined into a caller where the flag is a
constant; when the loop is kept out of line with
`-XX:CompileCommand=dontinline`, `squareDistance` followed by
`uint8SquareDistance` is 981 ns per call before and 441 ns after (418-427 ns
alone).
End to end on aarch64, I used one index with an `UNSIGNED_BYTE` field and a
`SEVEN_BIT` field (`Lucene104HnswScalarQuantizedVectorsFormat`, 10,000 docs,
384 dimensions, force-merged, k = 10). Querying each field alone in its own JVM
takes 236 and 237 µs per query. Querying both in one JVM takes 9,208 and 5,166
µs before this change, and 214 and 212 µs after. On x86 with 512-bit vectors,
the end-to-end effect is small (about 1.1x to 1.5x) and noisy, and I could not
see it after this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
