On Tue, 30 Jun 2026 09:42:06 GMT, Jatin Bhateja <[email protected]> wrote:
> Accumulation is naturally carried in the FMA addend, which is why the > existing `FmaVF/FmaVD` x86 AD rules tie the destination to the addend and > emit the 231 form (dst = m1*m2 + dst), leaving both multiplicands intact. The > Float16 rules `vector_fma_HF_reg/vector_fma_HF_mem` in x86.ad instead tied > the destination to a multiplicand and emitted `vfmadd132ph` (dst = dst*src + > addend). > > In a register-blocked reduction (the canonical FP16 GEMM/dot-product > micro-kernel) each multiplicand is reused across several accumulators within > a k-step, so the 132 form cannot keep an accumulator live in its own register > and forces C2 to emit vmovdqu register-register copies per k-step for a tile > of T accumulators, increasing register pressure and throttling throughput. > The added pressure also makes the allocator overflow the ZMM register file > sooner, so at larger tiles these copies become actual stack spills/reloads > rather than register-register moves — and unlike register-register moves > (which several microarchitectures eliminate at rename), stack traffic cannot > be elided and costs real load/store bandwidth. > > This change reworks the two FmaVHF rules to match the addend as the > destination and emit vfmadd231ph, matching the FmaVF/FmaVD behaviour. > > Following are the performance number of include micro benchmark on x86 target > with AVX512-FP16 feature (Granite Rapids at 2.5GHz fixed frequency) > > > Baseline:- > Benchmark (K) Mode Cnt Score Error > Units > Float16VectorFMAAccumBenchmark.fmaAccum24 256 thrpt 2 401.999 > ops/ms > Float16VectorFMAAccumBenchmark.fmaAccum24 1024 thrpt 2 92.296 > ops/ms > > Withopt:- > Benchmark (K) Mode Cnt Score Error > Units > Float16VectorFMAAccumBenchmark.fmaAccum24 256 thrpt 2 550.271 > ops/ms > Float16VectorFMAAccumBenchmark.fmaAccum24 1024 thrpt 2 139.446 > ops/ms > > > Kindly review and share your feedback. > > Best Regards, > Jatin > > --------- > - [x] I confirm that I make this contribution in accordance with the [OpenJDK > Interim AI Policy](https://openjdk.org/legal/ai). src/hotspot/cpu/x86/x86.ad line 24954: > 24952: int vlen_enc = vector_length_encoding(this); > 24953: __ evfmadd231ph($dst$$XMMRegister, $src2$$XMMRegister, > $src1$$Address, vlen_enc); > 24954: %} We should add the predicate: predicate(Matcher::vector_length_in_bytes(n->in(1)) > 8); Also a nit: It reads weird to have the order as src2, src1 instead of src1, src2. Let us have the match rule as: match(Set dst (FmaVHF dst (Binary src1 (VectorReinterpret (LoadVector src2))))); and the corresponding changes below. ------------- PR Review Comment: https://git.openjdk.org/jdk/pull/31724#discussion_r3624222517
