On Thu, 13 Aug 2026 22:48:49 GMT, Chad Rakoczy <[email protected]> wrote:
> Adds support for vectorized dot product on aarch64 (`sdot` and `udot`) > through the Vector API > > Dedicated dot product instructions have been shown to provide up to 10x > throughput for lucene > ([results](https://github.com/apache/lucene/pull/13572)) compared to > vectorized multiply and add. This PR adds two new methods to `ByteVector` to > leverage the aarch dot product instructions `sdot` and `udot` respectively. > - `IntVector dot(Vector<Byte> v, Vector<Integer> acc)` > - `IntVector dotUnsigned(Vector<Byte> v, Vector<Integer> acc)` > > Each int lane of the result holds the dot product of the corresponding group > of four bytes from the two operands, added to the matching accumulator lane. > For example: > > a = [a1, a2, a3, a4, ..., ..., a13, a14, a15, a16] > b = [b1, b2, b3, b4, ..., ..., b13, b14, b15, b16] > acc = [acc1, ..., ..., acc4] > > a.dot(b, acc) -> [ > acc1 + a1 * b1 + a2 * b2 + a3 * b3 + a4 * b4, > ..., > ..., > acc4 + a13 * b13 + a14 * b14 + a15 * b15 + a16 * b16 > ] > > > The equivalent instructions on x86 are `VPDPBSSD` and `VPDPBUUD` which > perform the same operations and match the proposed new functions however this > PR only includes aarch64. > > Graviton 2 > > Benchmark Mode Cnt Score Error Units > VectorDotBenchmark.dotScalar thrpt 25 2412.719 ± 0.014 ops/ms > VectorDotBenchmark.dotMulAdd thrpt 25 1497.953 ± 2.734 ops/ms > VectorDotBenchmark.dotVector thrpt 25 23243.016 ± 141.708 ops/ms > VectorDotBenchmark.dotUnsignedScalar thrpt 25 2402.966 ± 0.157 ops/ms > VectorDotBenchmark.dotUnsignedMulAdd thrpt 25 1498.898 ± 1.442 ops/ms > VectorDotBenchmark.dotUnsignedVector thrpt 25 23344.116 ± 222.150 ops/ms > > > Graviton 3 > > Benchmark Mode Cnt Score Error Units > VectorDotBenchmark.dotScalar thrpt 25 7946.950 ± 2.564 ops/ms > VectorDotBenchmark.dotMulAdd thrpt 25 3257.311 ± 5.308 ops/ms > VectorDotBenchmark.dotVector thrpt 25 41430.996 ± 688.936 ops/ms > VectorDotBenchmark.dotUnsignedScalar thrpt 25 2536.268 ± 0.438 ops/ms > VectorDotBenchmark.dotUnsignedMulAdd thrpt 25 3259.853 ± 3.511 ops/ms > VectorDotBenchmark.dotUnsignedVector thrpt 25 41263.821 ± 478.856 ops/ms > > > --------- > - [x] I confirm that I make this contribution in accordance with the [OpenJDK > Interim AI Policy](https://openjdk.org/legal/ai). Looking at Chad's proposal, it matches SVE closely enough that the instructions can almost directly be emitted. That there is an x86 equivalent and a use case, I'd be in favour of adding it. However, I appreciate it might be difficult to incorporate more specialist instructions in a way that is portable and predictable in terms of performance. The Vector API allows programmers to write vectorized code without the brittleness of auto-vectorization, but if Vector API code is to be transformed from existing calls into more esoteric instructions, such as the dot product, we are possibly going to hit similar issues. I was going to suggest providing a library of routines that we know C2 can transform, but that would seem to just be adding a layer of unnecessary complexity. I made an attempt at rewriting the routine from @iwanowww , but with the actual dot product routine behaviour. The SVE *DOT instructions calculate the dot product of 4 groups of 4 bytes from two vectors and store the results into 4x 32-bit lanes. For vectors longer that 128-bits, this is repeated over however many 128-bit slices there are in a vector: static IntVector sdot(ByteVector v1, ByteVector v2) { Vector<Integer> acc = IMAX.zero(); VectorMask<Integer> fourLanes = VectorMask.fromLong(IMAX, 0b1111); for (int slice = 0; slice < v1.length()/16; slice++) { int[] dp = new int[4]; for (int part = 0; part < 4; part++) { var i1 = v1.castShape(I128, part + slice * 4).reinterpretAsInts(); var i2 = v2.castShape(I128, part + slice * 4).reinterpretAsInts(); var mul = i1.lanewise(VectorOperators.MUL, i2).reinterpretAsInts(); dp[part] = mul.reduceLanes(VectorOperators.ADD); } var result = (IntVector) I128.fromArray(dp, 0); var padded = (IntVector) result.castShape(IMAX, 0); acc = padded.unslice(slice * 4, acc, 0, fourLanes); } This is a more complex pattern to match to emit a DOT instruction for., and I suspect this would be a bit more difficult to recognise. ------------- PR Comment: https://git.openjdk.org/jdk/pull/32359#issuecomment-6128837929
