> ### Summary
> This PR adds a non-temporal store path to the x86_64 AVX2 forward primitive 
> arraycopy stubs, extending the existing AVX-512 streaming-store approach to 
> the AVX2 implementation and improving throughput for large streaming copies.
> 
> ### Problem
> The existing AVX2 arraycopy implementation suffers from two major performance 
> issues:
> 
> - Unnecessary write-allocate traffic: temporal destination stores for 
> streaming copies introduce write-allocate traffic and cache pollution
> - Missing streaming path in AVX2: the AVX-512 implementation already provides 
> a streaming store path for large copies, but the AVX2 implementation does not
> 
> ### Changes
> Added a non-temporal store path for large primitive array copies in the AVX2 
> forward stubs
> 
> The new loop aligns the destination to 32 bytes and copies 64 bytes per 
> iteration using two 256-bit loads and two VMOVNTDQ stores, followed by an 
> SFENCE. Alignment and trailing elements use temporal stores, with each source 
> element copied once to preserve correctness for forward overlapping copies.
> 
> `ArrayCopyNonTemporalThreshold` specifies the selection threshold in bytes. 
> When unspecified, it is initialized from the detected L3 cache size: standard 
> CPUID leaf 4 on Intel and other applicable x86 processors, and extended leaf 
> 0x80000006 on AMD/Hygon. An explicit nonzero value overrides detection; 
> explicit zero disables the new path; failed detection leaves the default at 
> zero. The threshold is included in AOT code-cache configuration checks to 
> prevent reuse of stubs generated with a different setting.
> 
> The path requires AVX2 and `UseUnalignedLoadStores`. Reference array copies, 
> backward copies, and the existing AVX-512 paths retain their current 
> implementations.
> 
> ### Testing
> JMH benchmarks
> 
> Measurements were taken on a two-socket Intel Xeon Platinum 8558 (Sapphire 
> Rapids) system with a reported 260 MiB L3 cache per socket, running CentOS 8. 
> The benchmark repeatedly copies a source byte array into two alternating 
> destination arrays and measures throughput in operations per second at 0.5x, 
> 1x, 2x, and 4x the reported per-socket L3 size.
> 
> The machine supports AVX-512 and defaults to `UseAVX=3`; these runs 
> explicitly use `-XX:UseAVX=2` to exercise the AVX2 implementation, with 
> `ArrayCopyNonTemporalThreshold` automatically detected at 272629760 bytes 
> (260 MiB).
> 
> | Copy size | Baseline, ops/s | Patched, ops/s | Speedup |
> |-----------|----------------:|---------------:|--------:|
> | 130 MiB (0.5x L3) | 47.8 | 53.8 | 1.13x |
> | 260 MiB (1...

Larry has updated the pull request incrementally with one additional commit 
since the last revision:

  Update test cases

-------------

Changes:
  - all: https://git.openjdk.org/jdk/pull/33437/files
  - new: https://git.openjdk.org/jdk/pull/33437/files/05d27449..c0868a67

Webrevs:
 - full: https://webrevs.openjdk.org/?repo=jdk&pr=33437&range=03
 - incr: https://webrevs.openjdk.org/?repo=jdk&pr=33437&range=02-03

  Stats: 27 lines in 1 file changed: 21 ins; 0 del; 6 mod
  Patch: https://git.openjdk.org/jdk/pull/33437.diff
  Fetch: git fetch https://git.openjdk.org/jdk.git pull/33437/head:pull/33437

PR: https://git.openjdk.org/jdk/pull/33437

Reply via email to