On Thu, 5 Nov 2026 14:13:28 GMT, Ivan Zhechev <[email protected]> wrote:
>> NEON non-strictly ordered floating-point reduction instruction rules are >> updated to avoid materialising a redundant `fadd`, and additionally now use >> an immediate operand instead of a register. >> >> Vector API and auto-vectorisation share the AddReductionV node which has to >> support the needs of both - it models a scalar value plus a vector operand. >> Since `reduceLanes()` has no scalar source the Vector API intrinsic >> generates an identity scalar value which is passed as the first input to >> AddReductionVF/VD. Because addition with the identity value is unnecessary >> when implementing a Vector API reduction, NEON non-strict floating-point >> reductions no longer generate an extra instruction. Furthermore, the >> matcher has been updated to use zero-constrained immediate operands to >> guarantee that the scalar value is zero. >> >> Update the predicate of `reduce_non_strict_order_add2D_neon` to restrict it >> to 128-bit vectors, ensuring vectors wider than 128 bits select >> `reduce_addD_sve` for correct code generation on all SVE-capable systems. >> >> Before: >> >> faddp v26.4s, v18.4s, v18.4s >> faddp s19, v26.2s >> fadd s19, s19, s17 // accum(s19) += s17(value 0.0) >> >> >> After: >> >> faddp v26.4s, v18.4s, v18.4s >> faddp s19, v26.2s >> >> >> Throughput benchmarks are split into 2 different types, based on the >> reduction benchmarks in >> `test/micro/org/openjdk/bench/vm/compiler/VectorAlgorithmsImpl.java` >> >> 1. A naive version which performs lane reductions inside the loop, e.g. >> `vectorDoubleAddReduce_naive` >> 2. 'reduction_after_loop' version which performs element-wise addition >> inside the loop, followed by a single `reduceLanes` after the loop, e.g. >> `vectorDoubleAddReduce_reduction_after_loop` >> >> Benchmarking has been conducted on Arm Neoverse N1, Neoverse V2, as well as >> Arm Cortex-A725, Cortex-X925 CPUs. >> >> Mode: thrpt >> Units: ops/us >> ObjectAlignmentInBytes: 64 >> >> Number is speed-up, higher number is better >> >> | Benchmark | Type | Count | A725 | X925 | N1 | V2 | >> | naive | Double | 128 | 1.0 | 1.0 | 1.0 | 1.0 | >> | naive | Double | 512 | 1.0 | 1.0 | 1.0 | 1.0 | >> | naive | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 | >> | naive | Double | 4096 | 1.0 | 1.01 | 1.0 | 1.0 | >> | reduction_after_loop | Double | 128 | 1.02 | 1.02 | 1.03 | 1.01 | >> | reduction_after_loop | Double | 512 | 1.0 | 0.99 | 1.0 | 1.0 | >> | reduction_after_loop | Double | 2048 | 1.0 | ... > > Ivan Zhechev has updated the pull request with a new target base due to a > merge or a rebase. The incremental webrev excludes the unrelated changes > brought in by the merge/rebase. The pull request contains three additional > commits since the last revision: > > - Add test > - Merge branch 'master' into 8392077-ivan > - 8392077: AArch64: Remove redundant FADD in floating-point add reductions > on NEON > > NEON non-strictly ordered floating-point reduction instruction rules are > updated to avoid materialising a redundant `fadd`, and additionally now > use an immediate operand instead of a register. > > Vector API and auto-vectorisation share the AddReductionV node which has > to support the needs of both - it models a scalar value plus a vector > operand. Since `reduceLanes()` has no scalar source the Vector API > intrinsic generates an identity scalar value which is passed as the > first input to AddReductionVF/VD. Because addition with the identity > value is unnecessary when implementing a Vector API reduction, NEON > non-strict floating-point reductions no longer generate an extra > instruction. Furthermore, the matcher has been updated to use > zero-constrained immediate operands to guarantee that the scalar value > is zero. > > Update the predicate of `reduce_non_strict_order_add2D_neon` to restrict > it to 128-bit vectors, ensuring vectors wider than 128 bits select > `reduce_addD_sve` for correct code generation on all SVE-capable > systems. > > Before: > > faddp v26.4s, v18.4s, v18.4s > faddp s19, v26.2s > fadd s19, s19, s17 // accum(s19) += s17(value 0.0) > > After: > > faddp v26.4s, v18.4s, v18.4s > faddp s19, v26.2s > > Throughput benchmarks are split into 2 different types, based on the > reduction benchmarks in > test/micro/org/openjdk/bench/vm/compiler/VectorAlgorithmsImpl.java > > 1. A naive version which performs lane reductions inside the loop, e.g. > vectorDoubleAddReduce_naive > 2. 'reduction_after_loop' version which performs element-wise addition > inside the loop, followed by a single reduceLanes after the loop, > e.g. vectorDoubleAddReduce_reduction_after_loop > > Benchmarking has been conducted on Arm Neoverse N1, Neoverse V2, as well > as Arm Cortex-A725, Cortex-X925 CPUs. > > Mode: thrpt > Units: ops/us > ObjectAlignmentInBytes: 64 > > Number is speed-up, higher number is better > > | Benchmark | Type | Count | A725 | X925 | N1 | V2 | > ... Does it have the same opportunity for other reduction operations for integer types, or maybe other ops like min/max and the logic ones? BTW, do you think it's better to separate the reduction IRs for Vector API and auto-vectorization. Looks like the last scalar operation for each reduction from Vector API is not needed for all architectures, is it? ------------- PR Comment: https://git.openjdk.org/jdk/pull/33319#issuecomment-6905203101
