> NEON non-strictly ordered floating-point reduction instruction rules are
> updated to avoid materialising a redundant `fadd`, and additionally now use
> an immediate operand instead of a register.
>
> Vector API and auto-vectorisation share the AddReductionV node which has to
> support the needs of both - it models a scalar value plus a vector operand.
> Since `reduceLanes()` has no scalar source the Vector API intrinsic generates
> an identity scalar value which is passed as the first input to
> AddReductionVF/VD. Because addition with the identity value is unnecessary
> when implementing a Vector API reduction, NEON non-strict floating-point
> reductions no longer generate an extra instruction. Furthermore, the matcher
> has been updated to use zero-constrained immediate operands to guarantee that
> the scalar value is zero.
>
> Update the predicate of `reduce_non_strict_order_add2D_neon` to restrict it
> to 128-bit vectors, ensuring vectors wider than 128 bits select
> `reduce_addD_sve` for correct code generation on all SVE-capable systems.
>
> Before:
>
> faddp v26.4s, v18.4s, v18.4s
> faddp s19, v26.2s
> fadd s19, s19, s17 // accum(s19) += s17(value 0.0)
>
>
> After:
>
> faddp v26.4s, v18.4s, v18.4s
> faddp s19, v26.2s
>
>
> Throughput benchmarks are split into 2 different types, based on the
> reduction benchmarks in
> `test/micro/org/openjdk/bench/vm/compiler/VectorAlgorithmsImpl.java`
>
> 1. A naive version which performs lane reductions inside the loop, e.g.
> `vectorDoubleAddReduce_naive`
> 2. 'reduction_after_loop' version which performs element-wise addition inside
> the loop, followed by a single `reduceLanes` after the loop, e.g.
> `vectorDoubleAddReduce_reduction_after_loop`
>
> Benchmarking has been conducted on Arm Neoverse N1, Neoverse V2, as well as
> Arm Cortex-A725, Cortex-X925 CPUs.
>
> Mode: thrpt
> Units: ops/us
> ObjectAlignmentInBytes: 64
>
> Number is speed-up, higher number is better
>
> | Benchmark | Type | Count | A725 | X925 | N1 | V2 |
> | naive | Double | 128 | 1.0 | 1.0 | 1.0 | 1.0 |
> | naive | Double | 512 | 1.0 | 1.0 | 1.0 | 1.0 |
> | naive | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 |
> | naive | Double | 4096 | 1.0 | 1.01 | 1.0 | 1.0 |
> | reduction_after_loop | Double | 128 | 1.02 | 1.02 | 1.03 | 1.01 |
> | reduction_after_loop | Double | 512 | 1.0 | 0.99 | 1.0 | 1.0 |
> | reduction_after_loop | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 |
> | reduction_after_loop | Double | 4096 | 1.0 | 1.0 | 1.0 ...
Ivan Zhechev has updated the pull request with a new target base due to a merge
or a rebase. The incremental webrev excludes the unrelated changes brought in
by the merge/rebase. The pull request contains three additional commits since
the last revision:
- Add test
- Merge branch 'master' into 8392077-ivan
- 8392077: AArch64: Remove redundant FADD in floating-point add reductions on
NEON
NEON non-strictly ordered floating-point reduction instruction rules are
updated to avoid materialising a redundant `fadd`, and additionally now
use an immediate operand instead of a register.
Vector API and auto-vectorisation share the AddReductionV node which has
to support the needs of both - it models a scalar value plus a vector
operand. Since `reduceLanes()` has no scalar source the Vector API
intrinsic generates an identity scalar value which is passed as the
first input to AddReductionVF/VD. Because addition with the identity
value is unnecessary when implementing a Vector API reduction, NEON
non-strict floating-point reductions no longer generate an extra
instruction. Furthermore, the matcher has been updated to use
zero-constrained immediate operands to guarantee that the scalar value
is zero.
Update the predicate of `reduce_non_strict_order_add2D_neon` to restrict
it to 128-bit vectors, ensuring vectors wider than 128 bits select
`reduce_addD_sve` for correct code generation on all SVE-capable
systems.
Before:
faddp v26.4s, v18.4s, v18.4s
faddp s19, v26.2s
fadd s19, s19, s17 // accum(s19) += s17(value 0.0)
After:
faddp v26.4s, v18.4s, v18.4s
faddp s19, v26.2s
Throughput benchmarks are split into 2 different types, based on the
reduction benchmarks in
test/micro/org/openjdk/bench/vm/compiler/VectorAlgorithmsImpl.java
1. A naive version which performs lane reductions inside the loop, e.g.
vectorDoubleAddReduce_naive
2. 'reduction_after_loop' version which performs element-wise addition
inside the loop, followed by a single reduceLanes after the loop,
e.g. vectorDoubleAddReduce_reduction_after_loop
Benchmarking has been conducted on Arm Neoverse N1, Neoverse V2, as well
as Arm Cortex-A725, Cortex-X925 CPUs.
Mode: thrpt
Units: ops/us
ObjectAlignmentInBytes: 64
Number is speed-up, higher number is better
| Benchmark | Type | Count | A725 | X925 | N1 | V2 |
| naive | Double | 128 | 1.0 | 1.0 | 1.0 | 1.0 |
| naive | Double | 512 | 1.0 | 1.0 | 1.0 | 1.0 |
| naive | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 |
| naive | Double | 4096 | 1.0 | 1.01 | 1.0 | 1.0 |
| reduction_after_loop | Double | 128 | 1.02 | 1.02 | 1.03 | 1.01 |
| reduction_after_loop | Double | 512 | 1.0 | 0.99 | 1.0 | 1.0 |
| reduction_after_loop | Double | 2048 | 1.0 | 1.0 | 1.0 | 1.0 |
| reduction_after_loop | Double | 4096 | 1.0 | 1.0 | 1.0 | 1.0 |
| naive | Float | 128 | 1.02 | 1.01 | 1.21 | 1.0 |
| naive | Float | 512 | 1.02 | 1.0 | 1.16 | 1.0 |
| naive | Float | 2048 | 1.02 | 1.0 | 1.14 | 1.0 |
| naive | Float | 4096 | 1.06 | 1.01 | 1.14 | 1.0 |
| reduction_after_loop | Float | 128 | 1.02 | 1.02 | 1.03 | 1.01 |
| reduction_after_loop | Float | 512 | 1.01 | 1.01 | 1.0 | 1.01 |
| reduction_after_loop | Float | 2048 | 1.0 | 1.0 | 1.0 | 1.0 |
| reduction_after_loop | Float | 4096 | 1.0 | 1.0 | 1.0 | 1.0 |
Speed-up is observed on Neoverse N1 for the naive Float benchmark, as
well as a slight general improvement for low trip counts of
reduction_after_loop.
As well as, a latency benchmark which uses reduction results in
subsequent reductions, which forms a latency-sensitive instruction
sequence.
Mode: avgt
Units: ns/op
ObjectAlignmentInBytes: 64
Number is reduction in time, lower number is better
| Count | Type | A725 | X925 | N1 | V2 |
| 128 | Double | 0.73 | 0.64 | 0.74 | 0.73 |
| 512 | Double | 0.75 | 0.73 | 0.75 | 0.75 |
| 2048 | Double | 0.75 | 0.74 | 0.75 | 0.75 |
| 4096 | Double | 0.75 | 0.75 | 0.75 | 0.75 |
| 128 | Float | 0.8 | 0.76 | 0.81 | 0.8 |
| 512 | Float | 0.82 | 0.8 | 0.82 | 0.82 |
| 2048 | Float | 0.82 | 0.82 | 0.82 | 0.82 |
| 4096 | Float | 0.82 | 0.82 | 0.82 | 0.82 |
-------------
Changes:
- all: https://git.openjdk.org/jdk/pull/33319/files
- new: https://git.openjdk.org/jdk/pull/33319/files/1281953c..23d7a5e6
Webrevs:
- full: https://webrevs.openjdk.org/?repo=jdk&pr=33319&range=01
- incr: https://webrevs.openjdk.org/?repo=jdk&pr=33319&range=00-01
Stats: 133943 lines in 2856 files changed: 82550 ins; 36205 del; 15188 mod
Patch: https://git.openjdk.org/jdk/pull/33319.diff
Fetch: git fetch https://git.openjdk.org/jdk.git pull/33319/head:pull/33319
PR: https://git.openjdk.org/jdk/pull/33319
