On Tue, 13 Oct 2026 22:02:09 GMT, Vladimir Ivanov <[email protected]> wrote:
>> Adds support for vectorized dot product on aarch64 (`sdot` and `udot`) >> through the Vector API >> >> Dedicated dot product instructions have been shown to provide up to 10x >> throughput for lucene >> ([results](https://github.com/apache/lucene/pull/13572)) compared to >> vectorized multiply and add. This PR adds two new methods to `ByteVector` to >> leverage the aarch dot product instructions `sdot` and `udot` respectively. >> - `IntVector dot(Vector<Byte> v, Vector<Integer> acc)` >> - `IntVector dotUnsigned(Vector<Byte> v, Vector<Integer> acc)` >> >> Each int lane of the result holds the dot product of the corresponding group >> of four bytes from the two operands, added to the matching accumulator lane. >> For example: >> >> a = [a1, a2, a3, a4, ..., ..., a13, a14, a15, a16] >> b = [b1, b2, b3, b4, ..., ..., b13, b14, b15, b16] >> acc = [acc1, ..., ..., acc4] >> >> a.dot(b, acc) -> [ >> acc1 + a1 * b1 + a2 * b2 + a3 * b3 + a4 * b4, >> ..., >> ..., >> acc4 + a13 * b13 + a14 * b14 + a15 * b15 + a16 * b16 >> ] >> >> >> The equivalent instructions on x86 are `VPDPBSSD` and `VPDPBUUD` which >> perform the same operations and match the proposed new functions however >> this PR only includes aarch64. >> >> Graviton 2 >> >> Benchmark Mode Cnt Score Error Units >> VectorDotBenchmark.dotScalar thrpt 25 2412.719 ± 0.014 ops/ms >> VectorDotBenchmark.dotMulAdd thrpt 25 1497.953 ± 2.734 ops/ms >> VectorDotBenchmark.dotVector thrpt 25 23243.016 ± 141.708 ops/ms >> VectorDotBenchmark.dotUnsignedScalar thrpt 25 2402.966 ± 0.157 ops/ms >> VectorDotBenchmark.dotUnsignedMulAdd thrpt 25 1498.898 ± 1.442 ops/ms >> VectorDotBenchmark.dotUnsignedVector thrpt 25 23344.116 ± 222.150 ops/ms >> >> >> Graviton 3 >> >> Benchmark Mode Cnt Score Error Units >> VectorDotBenchmark.dotScalar thrpt 25 7946.950 ± 2.564 ops/ms >> VectorDotBenchmark.dotMulAdd thrpt 25 3257.311 ± 5.308 ops/ms >> VectorDotBenchmark.dotVector thrpt 25 41430.996 ± 688.936 ops/ms >> VectorDotBenchmark.dotUnsignedScalar thrpt 25 2536.268 ± 0.438 ops/ms >> VectorDotBenchmark.dotUnsignedMulAdd thrpt 25 3259.853 ± 3.511 ops/ms >> VectorDotBenchmark.dotUnsignedVector thrpt 25 41263.821 ± 478.856 ops/ms >> >> >> --------- >> - [x] I confirm that I make this contribution in accordance with the >> [OpenJDK Interim AI Policy](https://openjdk.org/legal... > > I'd still recommend to separate implementation and API work. Even though > matching vector operation shapes is brittle compared to dedicated API point, > it still exposes the desired functionality for potential users to try it out. > > > Unfortunately, I don't see a clear path to expose SVE/AVX512-VNNI > instructions through Vector API as it is shaped now. > There's no precedent vector operation shape (yet?) to match them well. There > are multiple reasons why they can be considered "snowflakes": applicable only > for subword element types (8-/16-bit in size), mixture of signed & unsigned > operands, mixture of subword operands and word-sized result (ByteVector x > ByteVector -> IntVector operation). Even though both AArch64 and AVX512 > feature similar instructions, it doesn't make the operations themselves less > adhoc. > > So, as the stop-the-gap solution, matching enables access to desired > functionality while giving more time to explore possible approaches on API > side. > >> I made an attempt at rewriting the routine from @iwanowww , but with the >> actual dot product routine behaviour. The SVE *DOT instructions calculate >> the dot product of 4 groups of 4 bytes from two vectors and store the >> results into 4x 32-bit lanes. For vectors longer that 128-bits, this is >> repeated over however many 128-bit slices there are in a vector: > > FTR I deliberately avoided exposing accumulator as a vector. Proposed version > gives enough freedom for JIT-compiler to generate efficient code (reordering > associative operations, plus transforming sum of reductions to reduction on a > vector accumulator may be used to match even naive implementation to the > particular machine instruction). Thanks for the explanation @iwanowww , I get the "snowflake" analogy now. They are numerous, and each have a unique shape. re: the example code. The code as you originally wrote would only handle the first 128-bit part of the source vectors. My example would just need to be changed to accumulate a single 32-bit value across the 32-bit value that is summed from the the 4x 4xbyte multiplications. As I'm not an end user of this, I have to wonder if overflow is a danger with very long vectors. ------------- PR Comment: https://git.openjdk.org/jdk/pull/32359#issuecomment-6170185447
