On Tue, Aug 11, 2026 at 7:15 PM H.J. Lu <[email protected]> wrote: > > On Tue, Aug 11, 2026 at 11:32 AM Hongtao Liu <[email protected]> wrote: > > > > On Wed, Jul 22, 2026 at 2:24 PM H.J. Lu <[email protected]> wrote: > > > > > > Changes in v2: > > > > > > 1. Add TARGET_128BIT_ATOMIC_ENABLED. > > > 2. Remove __atomic Builtins reference from -mcx16. > > > 3. Update -m128bit-atomic documentation. > > > 4. Issue an error for -m128bit-atomic with -m32. > > > > > > -- > > > H.J. > > > -- > > > With the silicon vendor guarantees from Intel, AMD, Hygon and Zhaoxin in: > > > > > > https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 > > > > > > many software developers would happily use inline 128-bit atomic loads > > > and stores in their programs because they only target compatible CPUs. > > > Add -m128bit-atomic to generate 128-bit atomic loads and stores to avoid > > > the overhead of calling into libatomic. Enable -m128bit-atomic in 64-bit > > > mode by default if supported by the targeting processor, which is one of > > > x86-64-v3 capable processors as well as AVX capable processors from Intel, > > > AMD, Hygon and Zhaoxin. > > > > > > gcc/ > > > > > > PR target/94649 > > > PR target/126293 > > > * common/config/i386/i386-cpuinfo.h (ix86_decode_cpu_info): New > > > function. > > > * config/i386/i386-options.cc > > > (ix86_option_override_internal): Issue an error for -m128bit-atomic > > > in 32-bit mode. Turn on -mcx16 if -m128bit-atomic is enabled. > > > Enable -m128bit-atomic in 64-bit mode by default if supported by > > > the targeting processor. > > > * config/i386/i386.h (TARGET_128BIT_ATOMIC_ENABLED): New. > > > * config/i386/i386.opt (ix86_flags): New Variable. > > > (m128bit-atomic): New option. > > > * config/i386/i386.opt.urls: Regenerated. > > > * config/i386/sync.md (atomic_loadti): New pattern. > > > (atomic_loadti_sse): Likewise. > > > (atomic_storeti): Likewise. > > > (atomic_storeti_sse): Likewise. > > > * doc/invoke.texi: Remove __atomic Builtins reference from -mcx16. > > > Document -m128bit-atomic. > > > > > > > + || (subtype >= INTEL_COREI7_GRANITERAPIDS > > + && subtype <= INTEL_COREI7_PANTHERLAKE) > > + || subtype == INTEL_COREI7_DIAMONDRAPIDS > > + || subtype <= INTEL_COREI7_NOVALAKE) > > + { > > + vendor = VENDOR_INTEL; > > + type = INTEL_COREI7; > > + } > > || substype <= INTEL_COREI7_NOVALAKE classifies AMD/Zhaoxin CPUs as > > Intel, is it a typo of *subtype == NTEL_COREI7_NOVALAKE* > > Fixed. > > > > > + if (TARGET_128BIT_ATOMIC_P (opts->x_ix86_flags)) > > + { > > + if (!TARGET_64BIT_P (opts->x_ix86_isa_flags)) > > + error ("%<-m128bit-atomic%> not supported for 32-bit code"); > > + else if (!TARGET_CX16_P (opts_set->x_ix86_isa_flags2)) > > + { > > + /* Enable CMPXCHG16B when -m128bit-atomic is enabled. */ > > + opts->x_ix86_isa_flags2 |= OPTION_MASK_ISA2_CX16; > > + } > > + } > > > > With global -march=skylake, a function marked target("arch=x86-64") > > still emits movdqa atomic loads. The non-explicit bit needs clearing > > before deriving the default for each target option set. > > Fixed. > > > +Variable > > +int ix86_flags = 0 > > > > Can we reuse ix86_target_flags, m128bit-atomic only need one bit. > > Fixed. > > > +(define_expand "atomic_loadti" > > + [(set (match_operand:TI 0 "nonimmediate_operand") > > + (unspec:TI [(match_operand:TI 1 "memory_operand") > > + (match_operand:SI 2 "const_int_operand")] > > + UNSPEC_LDA))] > > + "TARGET_128BIT_ATOMIC_ENABLED" > > +{ > > + emit_insn (gen_atomic_loadti_sse (operands[0], operands[1])); > > + DONE; > > +}) > > > > change operands[0] from nonimmediate_operand to register_operand? > > It is done on purpose the same way as > > (define_expand "atomic_load<mode>" > [(set (match_operand:ATOMIC 0 "nonimmediate_operand") > (unspec:ATOMIC [(match_operand:ATOMIC 1 "memory_operand") > (match_operand:SI 2 "const_int_operand")] > UNSPEC_LDA))] > "" > ... > > > > > +(define_expand "atomic_storeti" > > + [(set (match_operand:TI 0 "memory_operand") > > + (unspec:TI [(match_operand:TI 1 "nonimmediate_operand") > > + (match_operand:SI 2 "const_int_operand")] > > + UNSPEC_STA))] > > + "TARGET_128BIT_ATOMIC_ENABLED" > > +{ > > + /* Use V1TImode to force vector register for atomic store. */ > > + rtx src = gen_reg_rtx (V1TImode); > > + rtx op1 = gen_lowpart (V1TImode, operands[1]); > > + emit_move_insn (src, op1); > > + emit_insn (gen_atomic_storeti_sse (operands[0], src)); > > + DONE; > > +}) > > + > > Other atomic store handles sequentially consistent stores with a fence. .i.e > > > > /* ... followed by an MFENCE, if required. */ > > if (is_mm_seq_cst (model)) > > emit_insn (gen_mem_thread_fence (operands[2])); > > DONE; > > > > Do we need a similar handle for atomic_storeti? > > > > Fixed. > > I am testing in the v3 patch and will submit when it is > done. >
Here is the v3 patch. It passed GCC tests on Linux/x86-64. -- H.J. ---With the silicon vendor guarantees from Intel, AMD, Hygon and Zhaoxin in: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 many software developers would happily use inline 128-bit atomic loads and stores in their programs because they only target compatible CPUs. Add -m128bit-atomic to generate 128-bit atomic loads and stores to avoid the overhead of calling into libatomic. Enable -m128bit-atomic in 64-bit mode by default if supported by the targeting processor, which is one of x86-64-v3 capable processors as well as AVX capable processors from Intel, AMD, Hygon and Zhaoxin. gcc/ PR target/94649 PR target/126293 * common/config/i386/i386-cpuinfo.h (ix86_decode_cpu_info): New function. * config/i386/i386-options.cc (ix86_option_override_internal): Issue an error for -m128bit-atomic in 32-bit mode. Turn on -mcx16 if -m128bit-atomic is enabled. Enable -m128bit-atomic in 64-bit mode by default if supported by the targeting processor. * config/i386/i386.h (TARGET_128BIT_ATOMIC_ENABLED): New. * config/i386/i386.opt (m128bit-atomic): New option. * config/i386/i386.opt.urls: Regenerated. * config/i386/sync.md (atomic_loadti): New pattern. (atomic_loadti_sse): Likewise. (atomic_storeti): Likewise. (atomic_storeti_sse): Likewise. * doc/invoke.texi: Remove __atomic Builtins reference from -mcx16. Document -m128bit-atomic. gcc/testsuite/ PR target/94649 PR target/126293 * g++.target/i386/pr94649-1.C: New test. * gcc.target/i386/pr126293-1a.c: Likewise. * gcc.target/i386/pr126293-1b.c: Likewise. * gcc.target/i386/pr126293-1c.c: Likewise. * gcc.target/i386/pr126293-1d.c: Likewise. * gcc.target/i386/pr126293-2a.c: Likewise. * gcc.target/i386/pr126293-2b.c: Likewise. * gcc.target/i386/pr126293-3a.c: Likewise. * gcc.target/i386/pr126293-3b.c: Likewise. * gcc.target/i386/pr126293-4a.c: Likewise. * gcc.target/i386/pr126293-4b.c: Likewise. * gcc.target/i386/pr126293-4c.c: Likewise. * gcc.target/i386/pr126293-4d.c: Likewise. * gcc.target/i386/pr126293-4e.c: Likewise. * gcc.target/i386/pr126293-4f.c: Likewise. * gcc.target/i386/pr126293-4g.c: Likewise. * gcc.target/i386/pr126293-4h.c: Likewise. * gcc.target/i386/pr126293-4i.c: Likewise. * gcc.target/i386/pr126293-4j.c: Likewise. * gcc.target/i386/pr126293-4k.c: Likewise. * gcc.target/i386/pr126293-4l.c: Likewise. * gcc.target/i386/pr126293-4m.c: Likewise. * gcc.target/i386/pr126293-4n.c: Likewise. * gcc.target/i386/pr126293-4o.c: Likewise. * gcc.target/i386/pr126293-4p.c: Likewise. * gcc.target/i386/pr126293-4q.c: Likewise. * gcc.target/i386/pr126293-4r.c: Likewise. * gcc.target/i386/pr126293-4s.c: Likewise. * gcc.target/i386/pr126293-5.c: Likewise. * gcc.target/i386/pr126293-6.c: Likewise. * gcc.target/i386/pr126293-7.c: Likewise. * gcc.target/i386/pr126293-8.c: Likewise. * gcc.target/i386/pr126293-9.c: Likewise. * gcc.target/i386/pr126293-10.c: Likewise. * gcc.target/i386/pr126293-11.c: Likewise. * gcc.target/i386/pr126293-12.c: Likewise. * gcc.target/i386/pr126293-13.c: Likewise. * gcc.target/i386/pr126293-14.c: Likewise. * gcc.target/i386/pr126293-15.c: Likewise. * gcc.target/i386/pr126293-16.c: Likewise. * gcc.target/i386/pr126293-17.c: Likewise.
From 621fa35fa60d687d0645bf2ccf99cac1ab8aab93 Mon Sep 17 00:00:00 2001 From: "H.J. Lu" <[email protected]> Date: Fri, 17 Jul 2026 08:35:34 +0800 Subject: [PATCH v3] x86-64: Add -m128bit-atomic With the silicon vendor guarantees from Intel, AMD, Hygon and Zhaoxin in: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=104688 many software developers would happily use inline 128-bit atomic loads and stores in their programs because they only target compatible CPUs. Add -m128bit-atomic to generate 128-bit atomic loads and stores to avoid the overhead of calling into libatomic. Enable -m128bit-atomic in 64-bit mode by default if supported by the targeting processor, which is one of x86-64-v3 capable processors as well as AVX capable processors from Intel, AMD, Hygon and Zhaoxin. gcc/ PR target/94649 PR target/126293 * common/config/i386/i386-cpuinfo.h (ix86_decode_cpu_info): New function. * config/i386/i386-options.cc (ix86_option_override_internal): Issue an error for -m128bit-atomic in 32-bit mode. Turn on -mcx16 if -m128bit-atomic is enabled. Enable -m128bit-atomic in 64-bit mode by default if supported by the targeting processor. * config/i386/i386.h (TARGET_128BIT_ATOMIC_ENABLED): New. * config/i386/i386.opt (m128bit-atomic): New option. * config/i386/i386.opt.urls: Regenerated. * config/i386/sync.md (atomic_loadti): New pattern. (atomic_loadti_sse): Likewise. (atomic_storeti): Likewise. (atomic_storeti_sse): Likewise. * doc/invoke.texi: Remove __atomic Builtins reference from -mcx16. Document -m128bit-atomic. gcc/testsuite/ PR target/94649 PR target/126293 * g++.target/i386/pr94649-1.C: New test. * gcc.target/i386/pr126293-1a.c: Likewise. * gcc.target/i386/pr126293-1b.c: Likewise. * gcc.target/i386/pr126293-1c.c: Likewise. * gcc.target/i386/pr126293-1d.c: Likewise. * gcc.target/i386/pr126293-2a.c: Likewise. * gcc.target/i386/pr126293-2b.c: Likewise. * gcc.target/i386/pr126293-3a.c: Likewise. * gcc.target/i386/pr126293-3b.c: Likewise. * gcc.target/i386/pr126293-4a.c: Likewise. * gcc.target/i386/pr126293-4b.c: Likewise. * gcc.target/i386/pr126293-4c.c: Likewise. * gcc.target/i386/pr126293-4d.c: Likewise. * gcc.target/i386/pr126293-4e.c: Likewise. * gcc.target/i386/pr126293-4f.c: Likewise. * gcc.target/i386/pr126293-4g.c: Likewise. * gcc.target/i386/pr126293-4h.c: Likewise. * gcc.target/i386/pr126293-4i.c: Likewise. * gcc.target/i386/pr126293-4j.c: Likewise. * gcc.target/i386/pr126293-4k.c: Likewise. * gcc.target/i386/pr126293-4l.c: Likewise. * gcc.target/i386/pr126293-4m.c: Likewise. * gcc.target/i386/pr126293-4n.c: Likewise. * gcc.target/i386/pr126293-4o.c: Likewise. * gcc.target/i386/pr126293-4p.c: Likewise. * gcc.target/i386/pr126293-4q.c: Likewise. * gcc.target/i386/pr126293-4r.c: Likewise. * gcc.target/i386/pr126293-4s.c: Likewise. * gcc.target/i386/pr126293-5.c: Likewise. * gcc.target/i386/pr126293-6.c: Likewise. * gcc.target/i386/pr126293-7.c: Likewise. * gcc.target/i386/pr126293-8.c: Likewise. * gcc.target/i386/pr126293-9.c: Likewise. * gcc.target/i386/pr126293-10.c: Likewise. * gcc.target/i386/pr126293-11.c: Likewise. * gcc.target/i386/pr126293-12.c: Likewise. * gcc.target/i386/pr126293-13.c: Likewise. * gcc.target/i386/pr126293-14.c: Likewise. * gcc.target/i386/pr126293-15.c: Likewise. * gcc.target/i386/pr126293-16.c: Likewise. * gcc.target/i386/pr126293-17.c: Likewise. Signed-off-by: H.J. Lu <[email protected]> --- gcc/common/config/i386/i386-cpuinfo.h | 126 ++++++++++++++++++++ gcc/config/i386/i386-options.cc | 50 ++++++++ gcc/config/i386/i386.h | 9 ++ gcc/config/i386/i386.opt | 4 + gcc/config/i386/i386.opt.urls | 3 + gcc/config/i386/sync.md | 55 +++++++++ gcc/doc/invoke.texi | 24 +++- gcc/testsuite/g++.target/i386/pr94649-1.C | 17 +++ gcc/testsuite/gcc.target/i386/pr126293-10.c | 26 ++++ gcc/testsuite/gcc.target/i386/pr126293-11.c | 24 ++++ gcc/testsuite/gcc.target/i386/pr126293-12.c | 29 +++++ gcc/testsuite/gcc.target/i386/pr126293-13.c | 28 +++++ gcc/testsuite/gcc.target/i386/pr126293-14.c | 31 +++++ gcc/testsuite/gcc.target/i386/pr126293-15.c | 30 +++++ gcc/testsuite/gcc.target/i386/pr126293-16.c | 3 + gcc/testsuite/gcc.target/i386/pr126293-17.c | 4 + gcc/testsuite/gcc.target/i386/pr126293-1a.c | 22 ++++ gcc/testsuite/gcc.target/i386/pr126293-1b.c | 6 + gcc/testsuite/gcc.target/i386/pr126293-1c.c | 6 + gcc/testsuite/gcc.target/i386/pr126293-1d.c | 6 + gcc/testsuite/gcc.target/i386/pr126293-2a.c | 23 ++++ gcc/testsuite/gcc.target/i386/pr126293-2b.c | 55 +++++++++ gcc/testsuite/gcc.target/i386/pr126293-3a.c | 21 ++++ gcc/testsuite/gcc.target/i386/pr126293-3b.c | 45 +++++++ gcc/testsuite/gcc.target/i386/pr126293-4a.c | 31 +++++ gcc/testsuite/gcc.target/i386/pr126293-4b.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4c.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4d.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4e.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4f.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4g.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4h.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4i.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4j.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4k.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4l.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4m.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4n.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4o.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4p.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4q.c | 19 +++ gcc/testsuite/gcc.target/i386/pr126293-4r.c | 20 ++++ gcc/testsuite/gcc.target/i386/pr126293-4s.c | 20 ++++ gcc/testsuite/gcc.target/i386/pr126293-5.c | 25 ++++ gcc/testsuite/gcc.target/i386/pr126293-6.c | 24 ++++ gcc/testsuite/gcc.target/i386/pr126293-7.c | 23 ++++ gcc/testsuite/gcc.target/i386/pr126293-8.c | 24 ++++ gcc/testsuite/gcc.target/i386/pr126293-9.c | 23 ++++ 48 files changed, 1138 insertions(+), 3 deletions(-) create mode 100644 gcc/testsuite/g++.target/i386/pr94649-1.C create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-10.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-11.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-12.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-13.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-14.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-15.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-16.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-17.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-1a.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-1b.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-1c.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-1d.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-2a.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-2b.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-3a.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-3b.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4a.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4b.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4c.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4d.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4e.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4f.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4g.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4h.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4i.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4j.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4k.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4l.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4m.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4n.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4o.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4p.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4q.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4r.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-4s.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-5.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-6.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-7.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-8.c create mode 100644 gcc/testsuite/gcc.target/i386/pr126293-9.c diff --git a/gcc/common/config/i386/i386-cpuinfo.h b/gcc/common/config/i386/i386-cpuinfo.h index 5b9bf5db68b..dd3b4e46d8b 100644 --- a/gcc/common/config/i386/i386-cpuinfo.h +++ b/gcc/common/config/i386/i386-cpuinfo.h @@ -300,3 +300,129 @@ enum processor_features #define M_VENDOR(a) (a) #define M_CPU_TYPE(a) (M_CPU_TYPE_START + int (a)) #define M_CPU_SUBTYPE(a) (M_CPU_SUBTYPE_START + a) + +#ifdef __cplusplus +/* Extract processor VENDOR, TYPE and SUBTYPE from MODEL. */ + +static inline void +ix86_decode_cpu_info (int model, processor_vendor &vendor, + processor_types &type, + processor_subtypes &subtype) +{ + if (model > M_CPU_SUBTYPE_START) + { + subtype = (processor_subtypes) (model - M_CPU_SUBTYPE_START); + if (subtype <= INTEL_COREI7_SANDYBRIDGE + || (subtype >= INTEL_COREI7_IVYBRIDGE + && subtype <= INTEL_COREI7_ICELAKE_SERVER) + || (subtype >= INTEL_COREI7_CASCADELAKE + && subtype <= INTEL_COREI7_ALDERLAKE) + || subtype == INTEL_COREI7_ROCKETLAKE + || (subtype >= INTEL_COREI7_GRANITERAPIDS + && subtype <= INTEL_COREI7_PANTHERLAKE) + || subtype == INTEL_COREI7_DIAMONDRAPIDS + || subtype == INTEL_COREI7_NOVALAKE) + { + vendor = VENDOR_INTEL; + type = INTEL_COREI7; + } + else if (subtype >= AMDFAM10H_BARCELONA + && subtype <= AMDFAM10H_ISTANBUL) + { + vendor = VENDOR_AMD; + type = AMDFAM10H; + } + else if (subtype >= AMDFAM15H_BDVER1 + && subtype <= AMDFAM15H_BDVER4) + { + vendor = VENDOR_AMD; + type = AMDFAM15H; + } + else if (subtype == AMDFAM17H_ZNVER1 + || subtype == AMDFAM17H_ZNVER2) + { + vendor = VENDOR_AMD; + type = AMDFAM17H; + } + else if (subtype == AMDFAM19H_ZNVER3 + || subtype == AMDFAM19H_ZNVER4) + { + vendor = VENDOR_AMD; + type = AMDFAM19H; + } + else if (subtype == AMDFAM1AH_ZNVER5 + || subtype == AMDFAM1AH_ZNVER6) + { + vendor = VENDOR_AMD; + type = AMDFAM1AH; + } + else if (subtype == ZHAOXIN_FAM7H_LUJIAZUI + || subtype == ZHAOXIN_FAM7H_YONGFENG + || subtype == ZHAOXIN_FAM7H_SHIJIDADAO) + { + vendor = VENDOR_ZHAOXIN; + type = ZHAOXIN_FAM7H; + } + else if (subtype >= HYGONFAM18H_C86_4G_M4 + && subtype <= HYGONFAM18H_C86_4G_M8) + { + vendor = VENDOR_HYGON; + type = HYGONFAM18H; + } + else + { + /* Set VENDOR to VENDOR_OTHER and TYPE to CPU_TYPE_MAX for + unsupported SUBTYPE. */ + vendor = VENDOR_OTHER; + type = CPU_TYPE_MAX; + } + } + else if (model > M_CPU_TYPE_START) + { + type = (processor_types) (model - M_CPU_TYPE_START); + if ((type >= INTEL_BONNELL && type <= INTEL_COREI7) + || type == INTEL_SILVERMONT + || (type >= INTEL_GOLDMONT && type <= INTEL_TREMONT) + || (type >= INTEL_SIERRAFOREST + && type <= INTEL_CLEARWATERFOREST)) + { + vendor = VENDOR_INTEL; + subtype = CPU_SUBTYPE_MAX; + } + else if (type == AMDFAM10H + || type == AMDFAM15H + || (type >= AMD_BTVER1 && type <= AMDFAM17H) + || type == AMDFAM19H + || type == AMDFAM1AH) + { + vendor = VENDOR_AMD; + subtype = CPU_SUBTYPE_MAX; + } + else if (type == ZHAOXIN_FAM7H) + { + vendor = VENDOR_ZHAOXIN; + subtype = CPU_SUBTYPE_MAX; + } + else if (type == HYGONFAM18H) + { + vendor = VENDOR_HYGON; + subtype = CPU_SUBTYPE_MAX; + } + else + { + /* Set VENDOR to VENDOR_OTHER and SUBTYPE to CPU_SUBTYPE_MAX + for unsupported TYPE. */ + vendor = VENDOR_OTHER; + subtype = CPU_SUBTYPE_MAX; + } + } + else + { + vendor = (processor_vendor) model; + /* Set TYPE to CPU_TYPE_MAX and SUBTYPE to CPU_SUBTYPE_MAX for + unknown TYPE and SUBTYPE. */ + type = CPU_TYPE_MAX; + subtype = CPU_SUBTYPE_MAX; + } +} +#endif diff --git a/gcc/config/i386/i386-options.cc b/gcc/config/i386/i386-options.cc index afbd96b79c3..6bb5e1a08d9 100644 --- a/gcc/config/i386/i386-options.cc +++ b/gcc/config/i386/i386-options.cc @@ -2305,6 +2305,11 @@ ix86_option_override_internal (bool main_args_p, sorry ("%i-bit mode not compiled in", (opts->x_ix86_isa_flags & OPTION_MASK_ISA_64BIT) ? 64 : 32); + processor_vendor ix86_vendor = VENDOR_OTHER; + processor_types ix86_type = CPU_TYPE_MAX; + processor_subtypes ix86_subtype = CPU_SUBTYPE_MAX; + wide_int_bitmask ix86_pta = 0; + /* Last processor_alias_table must point to "generic" entry. */ gcc_checking_assert (strcmp (processor_alias_table[pta_size - 1].name, "generic") == 0); @@ -2340,6 +2345,10 @@ ix86_option_override_internal (bool main_args_p, ix86_schedule = processor_alias_table[i].schedule; ix86_arch = processor_alias_table[i].processor; + ix86_pta = processor_alias_table[i].flags; + + ix86_decode_cpu_info (processor_alias_table[i].model, + ix86_vendor, ix86_type, ix86_subtype); /* Default cpu tuning to the architecture, unless the table entry requests not to do this. Used by the x86-64 psABI @@ -2528,6 +2537,47 @@ ix86_option_override_internal (bool main_args_p, XDELETEVEC (s); } + /* Disable -m128bit-atomic if not set on command-line. */ + if (!TARGET_128BIT_ATOMIC_P (opts_set->x_ix86_target_flags)) + opts->x_ix86_target_flags &= ~OPTION_MASK_128BIT_ATOMIC; + + if (TARGET_128BIT_ATOMIC_P (opts->x_ix86_target_flags)) + { + if (!TARGET_64BIT_P (opts->x_ix86_isa_flags)) + error ("%<-m128bit-atomic%> not supported for 32-bit code"); + else if (!TARGET_CX16_P (opts_set->x_ix86_isa_flags2)) + { + /* Enable CMPXCHG16B when -m128bit-atomic is enabled. */ + opts->x_ix86_isa_flags2 |= OPTION_MASK_ISA2_CX16; + } + } + else if (!TARGET_128BIT_ATOMIC_P (opts_set->x_ix86_target_flags) + && TARGET_64BIT_P (opts->x_ix86_isa_flags) + && ((TARGET_AVX_P (opts->x_ix86_isa_flags) + && (ix86_vendor == VENDOR_INTEL + || ix86_vendor == VENDOR_AMD + || ix86_vendor == VENDOR_HYGON + || ix86_vendor == VENDOR_ZHAOXIN)) + || (ix86_pta & PTA_X86_64_V3) == PTA_X86_64_V3 + || ix86_type == AMDFAM17H + || ix86_type == AMDFAM19H + || ix86_type == AMDFAM1AH + || ix86_type == HYGONFAM18H + || (ix86_type >= INTEL_SIERRAFOREST + && ix86_type <= INTEL_CLEARWATERFOREST) + || ix86_subtype == INTEL_COREI7_SANDYBRIDGE + || (ix86_subtype >= AMDFAM17H_ZNVER1 + && ix86_subtype <= INTEL_COREI7_ROCKETLAKE) + || (ix86_subtype >= AMDFAM19H_ZNVER4 + && ix86_subtype <= HYGONFAM18H_C86_4G_M8))) + { + /* Turn on -m128bit-atomic in 64-bit mode by default if supported + by the targeting processor, which is one of x86-64-v3 capable + processors as well as AVX capable processors from Intel, AMD, + Hygon and Zhaoxin. */ + opts->x_ix86_target_flags |= OPTION_MASK_128BIT_ATOMIC; + } + set_ix86_tune_features (opts, ix86_tune, opts->x_ix86_dump_tunes); ix86_recompute_optlev_based_flags (opts, opts_set); diff --git a/gcc/config/i386/i386.h b/gcc/config/i386/i386.h index d4082a84059..26c3b73a50f 100644 --- a/gcc/config/i386/i386.h +++ b/gcc/config/i386/i386.h @@ -3085,6 +3085,15 @@ extern void debug_dispatch_window (int); #define TARGET_RECIP_VEC_DIV ((recip_mask & RECIP_MASK_VEC_DIV) != 0) #define TARGET_RECIP_VEC_SQRT ((recip_mask & RECIP_MASK_VEC_SQRT) != 0) +/* -m128bit-atomic requires CMPXCHG16B and SSE2. + + Note: Pre-Tiger Lake (Desktop/Mobile): Generations including Kaby + Lake, Coffee Lake, and Comet Lake (e.g., Pentium Gold G5400, Celeron + G5900, N4020) do not support AVX. However, 128-bit aligned SSE loads + and stores are atomic on these processors. Should AVX be required? */ +#define TARGET_128BIT_ATOMIC_ENABLED \ + (TARGET_CX16 && TARGET_SSE2 && TARGET_128BIT_ATOMIC) + /* Use 128-bit AVX instructions in the auto-vectorizer. */ #define TARGET_PREFER_AVX128 (prefer_vector_width_type == PVW_AVX128) /* Use 256-bit AVX instructions in the auto-vectorizer. */ diff --git a/gcc/config/i386/i386.opt b/gcc/config/i386/i386.opt index 2cdc71a2324..57ba3fbfa8f 100644 --- a/gcc/config/i386/i386.opt +++ b/gcc/config/i386/i386.opt @@ -1043,6 +1043,10 @@ mgeneral-regs-only Target RejectNegative Mask(GENERAL_REGS_ONLY) Var(ix86_target_flags) Save Generate code which uses only the general registers. +m128bit-atomic +Target Mask(128BIT_ATOMIC) Var(ix86_target_flags) Save +Generate cmpxchg16b, 128-bit atomic vector loads and stores for the x86-64 architecture. + mshstk Target Mask(ISA_SHSTK) Var(ix86_isa_flags) Save Enable shadow stack built-in functions from Control-flow Enforcement diff --git a/gcc/config/i386/i386.opt.urls b/gcc/config/i386/i386.opt.urls index d0c58ca3526..d52e6de8014 100644 --- a/gcc/config/i386/i386.opt.urls +++ b/gcc/config/i386/i386.opt.urls @@ -464,6 +464,9 @@ UrlSuffix(gcc/x86-Options.html#index-mstack-protector-guard-symbol) mgeneral-regs-only UrlSuffix(gcc/x86-Options.html#index-mgeneral-regs-only-2) +m128bit-atomic +UrlSuffix(gcc/x86-Options.html#index-m128bit-atomic) + mshstk UrlSuffix(gcc/x86-Options.html#index-mshstk) diff --git a/gcc/config/i386/sync.md b/gcc/config/i386/sync.md index d9f822f4821..a5d71587b0d 100644 --- a/gcc/config/i386/sync.md +++ b/gcc/config/i386/sync.md @@ -388,6 +388,61 @@ (define_insn "storedi_via_sse" [(set_attr "type" "ssemov") (set_attr "mode" "DI")]) +(define_expand "atomic_loadti" + [(set (match_operand:TI 0 "nonimmediate_operand") + (unspec:TI [(match_operand:TI 1 "memory_operand") + (match_operand:SI 2 "const_int_operand")] + UNSPEC_LDA))] + "TARGET_128BIT_ATOMIC_ENABLED" +{ + emit_insn (gen_atomic_loadti_sse (operands[0], operands[1])); + DONE; +}) + +(define_insn_and_split "atomic_loadti_sse" + [(set (match_operand:TI 0 "register_operand" "=v") + (unspec:TI [(match_operand:TI 1 "memory_operand" "m")] + UNSPEC_LDX_ATOMIC))] + "TARGET_128BIT_ATOMIC_ENABLED" + "#" + "&& reload_completed" + [(set (match_dup 0) (match_dup 1))]) + +(define_expand "atomic_storeti" + [(set (match_operand:TI 0 "memory_operand") + (unspec:TI [(match_operand:TI 1 "nonimmediate_operand") + (match_operand:SI 2 "const_int_operand")] + UNSPEC_STA))] + "TARGET_128BIT_ATOMIC_ENABLED" +{ + enum memmodel model = memmodel_from_int (INTVAL (operands[2])); + + /* Use V1TImode to force vector register for atomic store. */ + rtx src = gen_reg_rtx (V1TImode); + rtx op1 = gen_lowpart (V1TImode, operands[1]); + emit_move_insn (src, op1); + emit_insn (gen_atomic_storeti_sse (operands[0], src)); + + /* ... followed by an MFENCE, if required. */ + if (is_mm_seq_cst (model)) + emit_insn (gen_mem_thread_fence (operands[2])); + DONE; +}) + +(define_insn_and_split "atomic_storeti_sse" + [(set (match_operand:TI 0 "memory_operand" "=m") + (unspec:TI [(match_operand:V1TI 1 "nonimmediate_operand" "v")] + UNSPEC_STX_ATOMIC))] + "TARGET_128BIT_ATOMIC_ENABLED" + "#" + "&& reload_completed" + [(const_int 0)] +{ + rtx src = gen_lowpart (TImode, operands[1]); + emit_move_insn (operands[0], src); + DONE; +}) + (define_expand "atomic_compare_and_swap<mode>" [(match_operand:QI 0 "register_operand") ;; bool success output (match_operand:SWI124 1 "register_operand") ;; oldval output diff --git a/gcc/doc/invoke.texi b/gcc/doc/invoke.texi index 7c65c924271..2e53a8c8a7e 100644 --- a/gcc/doc/invoke.texi +++ b/gcc/doc/invoke.texi @@ -1576,7 +1576,7 @@ See RS/6000 and PowerPC Options. -mstack-protector-guard-offset=@var{offset} -mstack-protector-guard-symbol=@var{symbol} -mgeneral-regs-only -mcall-ms2sysv-xlogues -mtls-dialect=@var{type} --mrelax-cmpxchg-loop +-m128bit-atomic -mrelax-cmpxchg-loop -mindirect-branch=@var{choice} -mfunction-return=@var{choice} -mindirect-branch-register -mharden-sls=@var{choice} -mindirect-branch-cs-prefix -mapx-inline-asm-use-gpr32 @@ -36726,8 +36726,7 @@ This option enables GCC to generate @code{CMPXCHG16B} instructions in 64-bit code to implement compare-and-exchange operations on 16-byte aligned 128-bit objects. This is useful for atomic updates of data structures exceeding one machine word in size. The compiler uses this instruction to implement -@ref{__sync Builtins}. However, for @ref{__atomic Builtins} operating on -128-bit integers, a library call is always used. +@ref{__sync Builtins}. @opindex msahf @opindex mno-sahf @@ -37292,6 +37291,25 @@ Generate code that uses only the general-purpose registers. This prevents the compiler from using floating-point, vector, mask and bound registers. +@opindex m128bit-atomic +@opindex mno-128bit-atomic +@item -m128bit-atomic +Generate cmpxchg16b, 128-bit atomic vector load and store instructions +for the x86-64 architecture. This is safe to use only on x86-64-v3 +capable processors as well as AVX capable processors from Intel, AMD, +Hygon and Zhaoxin, which guarantee that 128-bit aligned vector loads +and stores are atomic. This option requires SSE2 and CMPXCHG16B. It +implies @option{-mcx16} which enables CMPXCHG16B. This option is +enabled in 64-bit mode by default if supported by the targeting +processor. For @ref{__atomic Builtins} operating on 128-bit integers, +this option generates atomic instructions directly, instead of calling +a library function in the @file{libatomic} runtime library. + +Note that when SSE2 is disabled, i.e., by @option{-mgeneral-regs-only}, +128-bit atomic operations are unsupported and @file{libatomic} shouldn't +be used since it may use SSE2 or AVX instructions to implement 128-bit +atomic operations. + @opindex mrelax-cmpxchg-loop @opindex mno-relax-cmpxchg-loop @item -mrelax-cmpxchg-loop diff --git a/gcc/testsuite/g++.target/i386/pr94649-1.C b/gcc/testsuite/g++.target/i386/pr94649-1.C new file mode 100644 index 00000000000..7871e319134 --- /dev/null +++ b/gcc/testsuite/g++.target/i386/pr94649-1.C @@ -0,0 +1,17 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -std=c++11 -march=x86-64 -m128bit-atomic" } */ +/* { dg-final { scan-assembler-times "lock;?\[ \\t\]+cmpxchg16b" 1 } } */ + +#include <atomic> + +struct alignas(16) a +{ + long x; + long y; +}; + +bool +cmpxchg(std::atomic<a>& data, a expected, a newval) +{ + return std::atomic_compare_exchange_weak(&data, &expected, newval); +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-10.c b/gcc/testsuite/gcc.target/i386/pr126293-10.c new file mode 100644 index 00000000000..462042c29bb --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-10.c @@ -0,0 +1,26 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** fldt 8\(%rsp\) +** fstpt -24\(%rsp\) +** movdqa -24\(%rsp\), %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +extern _Atomic long double store; + +void +func (long double i) +{ + store = i; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-11.c b/gcc/testsuite/gcc.target/i386/pr126293-11.c new file mode 100644 index 00000000000..aa65acda009 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-11.c @@ -0,0 +1,24 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movdqa load\(%rip\), %xmm0 +** movaps %xmm0, -24\(%rsp\) +** fldt -24\(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +extern _Atomic long double load; + +long double +func (void) +{ + return load; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-12.c b/gcc/testsuite/gcc.target/i386/pr126293-12.c new file mode 100644 index 00000000000..065b779c409 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-12.c @@ -0,0 +1,29 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +typedef float vector __attribute__((vector_size (16))); +typedef struct +{ + vector v; +} v; + +extern _Atomic v store; + +void +func (v i) +{ + store = i; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-13.c b/gcc/testsuite/gcc.target/i386/pr126293-13.c new file mode 100644 index 00000000000..a46eb736347 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-13.c @@ -0,0 +1,28 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movdqa load\(%rip\), %xmm0 +** ret +** .cfi_endproc +**... +*/ + +typedef float vector __attribute__((vector_size (16))); +typedef struct +{ + vector v; +} v; + +extern _Atomic v load; + +v +func (void) +{ + return load; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-14.c b/gcc/testsuite/gcc.target/i386/pr126293-14.c new file mode 100644 index 00000000000..14d9822b476 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-14.c @@ -0,0 +1,31 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rdi, %xmm0 +** movq %rsi, %xmm1 +** punpcklqdq %xmm1, %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +typedef struct +{ + int a[4]; +} s; + +extern _Atomic s store; + +void +func (s i) +{ + store = i; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-15.c b/gcc/testsuite/gcc.target/i386/pr126293-15.c new file mode 100644 index 00000000000..058e432058d --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-15.c @@ -0,0 +1,30 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movdqa load\(%rip\), %xmm0 +** movaps %xmm0, -24\(%rsp\) +** movq -24\(%rsp\), %rax +** movq -16\(%rsp\), %rdx +** ret +** .cfi_endproc +**... +*/ + +typedef struct +{ + int a[4]; +} s; + +_Atomic s load; + +s +func (void) +{ + return load; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-16.c b/gcc/testsuite/gcc.target/i386/pr126293-16.c new file mode 100644 index 00000000000..4c4d486bc30 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-16.c @@ -0,0 +1,3 @@ +/* { dg-do compile { target ia32 } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic" } */ +/* { dg-error "'-m128bit-atomic' not supported for 32-bit code" "" { target *-*-* } 0 } */ diff --git a/gcc/testsuite/gcc.target/i386/pr126293-17.c b/gcc/testsuite/gcc.target/i386/pr126293-17.c new file mode 100644 index 00000000000..aafee4797fb --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-17.c @@ -0,0 +1,4 @@ +/* { dg-do compile { target ia32 } } */ +/* { dg-additional-options "-O2 -march=x86-64-v3" } */ + +#include <x86intrin.h> diff --git a/gcc/testsuite/gcc.target/i386/pr126293-1a.c b/gcc/testsuite/gcc.target/i386/pr126293-1a.c new file mode 100644 index 00000000000..cd34926a377 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-1a.c @@ -0,0 +1,22 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic" } */ +/* { dg-final { scan-assembler-times "lock;?\[ \\t\]+cmpxchg16b" 8 } } */ + +#include <stdint.h> + +#define FUNC_ATOMIC(TYPE, OP) \ +__attribute__ ((noinline, noclone)) \ +TYPE f_##TYPE##_##OP##_fetch (TYPE *a, TYPE b) \ +{ \ + return __atomic_##OP##_fetch (a, b, __ATOMIC_RELAXED); \ +} \ +__attribute__ ((noinline, noclone)) \ +TYPE f_##TYPE##_fetch_##OP (TYPE *a, TYPE b) \ +{ \ + return __atomic_fetch_##OP (a, b, __ATOMIC_RELAXED); \ +} + +FUNC_ATOMIC (__int128_t, and) +FUNC_ATOMIC (__int128_t, nand) +FUNC_ATOMIC (__int128_t, or) +FUNC_ATOMIC (__int128_t, xor) diff --git a/gcc/testsuite/gcc.target/i386/pr126293-1b.c b/gcc/testsuite/gcc.target/i386/pr126293-1b.c new file mode 100644 index 00000000000..bbd74e4c73a --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-1b.c @@ -0,0 +1,6 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -mno-cx16 -m128bit-atomic" } */ +/* { dg-final { scan-assembler-times "call\[ \\t\]+__atomic_fetch_" 4 } } */ +/* { dg-final { scan-assembler-times "jmp\[ \\t\]+__atomic_fetch_" 4 } } */ + +#include "pr126293-1a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-1c.c b/gcc/testsuite/gcc.target/i386/pr126293-1c.c new file mode 100644 index 00000000000..85fe25cae8b --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-1c.c @@ -0,0 +1,6 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic -mno-cx16" } */ +/* { dg-final { scan-assembler-times "call\[ \\t\]+__atomic_fetch_" 4 } } */ +/* { dg-final { scan-assembler-times "jmp\[ \\t\]+__atomic_fetch_" 4 } } */ + +#include "pr126293-1a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-1d.c b/gcc/testsuite/gcc.target/i386/pr126293-1d.c new file mode 100644 index 00000000000..6ca19cb1060 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-1d.c @@ -0,0 +1,6 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic" } */ +/* { dg-final { scan-assembler-times "lock;?\[ \\t\]+cmpxchg16b" 8 } } */ + +#include <x86intrin.h> +#include "pr126293-1a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-2a.c b/gcc/testsuite/gcc.target/i386/pr126293-2a.c new file mode 100644 index 00000000000..d2a11579d2f --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-2a.c @@ -0,0 +1,23 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic -mrelax-cmpxchg-loop" } */ +/* { dg-final { scan-assembler-times "rep;?\[ \\t\]+nop" 8 } } */ +/* { dg-final { scan-assembler-times "lock;?\[ \\t\]+cmpxchg16b" 8 } } */ + +#include <stdint.h> + +#define FUNC_ATOMIC(TYPE, OP) \ +__attribute__ ((noinline, noclone)) \ +TYPE f_##TYPE##_##OP##_fetch (TYPE *a, TYPE b) \ +{ \ + return __atomic_##OP##_fetch (a, b, __ATOMIC_RELAXED); \ +} \ +__attribute__ ((noinline, noclone)) \ +TYPE f_##TYPE##_fetch_##OP (TYPE *a, TYPE b) \ +{ \ + return __atomic_fetch_##OP (a, b, __ATOMIC_RELAXED); \ +} + +FUNC_ATOMIC (__int128_t, and) +FUNC_ATOMIC (__int128_t, nand) +FUNC_ATOMIC (__int128_t, or) +FUNC_ATOMIC (__int128_t, xor) diff --git a/gcc/testsuite/gcc.target/i386/pr126293-2b.c b/gcc/testsuite/gcc.target/i386/pr126293-2b.c new file mode 100644 index 00000000000..b74579c7cb0 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-2b.c @@ -0,0 +1,55 @@ +/* { dg-do run { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic" } */ + +#include <stdlib.h> +#include "pr126293-2a.c" + +#define FUNC_ATOMIC_RELAX(TYPE, OP) \ +__attribute__ ((noinline, noclone, target ("relax-cmpxchg-loop"))) \ +TYPE relax_##TYPE##_##OP##_fetch (TYPE *a, TYPE b) \ +{ \ + return __atomic_##OP##_fetch (a, b, __ATOMIC_RELAXED); \ +} \ +__attribute__ ((noinline, noclone, target ("relax-cmpxchg-loop"))) \ +TYPE relax_##TYPE##_fetch_##OP (TYPE *a, TYPE b) \ +{ \ + return __atomic_fetch_##OP (a, b, __ATOMIC_RELAXED); \ +} + +FUNC_ATOMIC_RELAX (__int128_t, and) +FUNC_ATOMIC_RELAX (__int128_t, nand) +FUNC_ATOMIC_RELAX (__int128_t, or) +FUNC_ATOMIC_RELAX (__int128_t, xor) + +#define TEST_ATOMIC_FETCH_LOGIC(TYPE, OP) \ +{ \ + TYPE a = 11, b = 101, res, exp; \ + TYPE c = 11, d = 101; \ + res = relax_##TYPE##_##OP##_fetch (&a, b); \ + exp = f_##TYPE##_##OP##_fetch (&c, d); \ + if (res != exp || a != c) \ + abort (); \ + a = c = 21, b = d = 92; \ + res = relax_##TYPE##_fetch_##OP (&a, b); \ + exp = f_##TYPE##_fetch_##OP (&c, d); \ + if (res != exp || a != c) \ + abort (); \ +} + +__attribute__((noinline)) +static void +do_test (void) +{ + TEST_ATOMIC_FETCH_LOGIC (__int128_t, and) + TEST_ATOMIC_FETCH_LOGIC (__int128_t, nand) + TEST_ATOMIC_FETCH_LOGIC (__int128_t, or) + TEST_ATOMIC_FETCH_LOGIC (__int128_t, xor) +} + +int +main (void) +{ + if (__builtin_cpu_supports ("cmpxchg16b")) + do_test (); + return 0; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-3a.c b/gcc/testsuite/gcc.target/i386/pr126293-3a.c new file mode 100644 index 00000000000..48630f5f6b6 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-3a.c @@ -0,0 +1,21 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic -mrelax-cmpxchg-loop" } */ +/* { dg-final { scan-assembler-times "lock;?\[ \\t\]+cmpxchg16b" 1 } } */ + +#include <stdint.h> + +#define FUNC_CMPXCHG(TYPE) \ +__attribute__ ((noinline, noclone)) \ +TYPE f_##TYPE##_cmpxchg (TYPE *lock, TYPE newval, TYPE oldval) \ +{ \ + do \ + { \ + newval = oldval | 1; \ + } while (! __atomic_compare_exchange_n (lock, &oldval, newval, \ + 0, __ATOMIC_RELEASE, \ + __ATOMIC_RELAXED)); \ + return *lock; \ +} + + +FUNC_CMPXCHG (__int128_t) diff --git a/gcc/testsuite/gcc.target/i386/pr126293-3b.c b/gcc/testsuite/gcc.target/i386/pr126293-3b.c new file mode 100644 index 00000000000..1e9d1872f8e --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-3b.c @@ -0,0 +1,45 @@ +/* { dg-do run { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -march=x86-64 -m128bit-atomic" } */ + +#include <stdlib.h> +#include "pr126293-3a.c" + +#define FUNC_CMPXCHG_RELAX(TYPE) \ +__attribute__ ((noinline, noclone, target ("relax-cmpxchg-loop"))) \ +TYPE relax_##TYPE##_cmpxchg (TYPE *lock, TYPE newval, TYPE oldval) \ +{ \ + do \ + { \ + newval = oldval | 1; \ + } while (! __atomic_compare_exchange_n (lock, &oldval, newval, \ + 0, __ATOMIC_RELEASE, \ + __ATOMIC_RELAXED)); \ + return *lock; \ +} + +FUNC_CMPXCHG_RELAX (__int128_t) + +#define TEST_CMPXCHG_LOOP(TYPE) \ +{ \ + TYPE a = 11, b = 20, c = 11, res, exp; \ + TYPE d = 11, e = 20, f = 11; \ + res = relax_##TYPE##_cmpxchg (&a, b, c); \ + exp = f_##TYPE##_cmpxchg (&d, e, f); \ + if (res != exp || a != d) \ + abort (); \ +} + +__attribute__((noinline)) +static void +do_test (void) +{ + TEST_CMPXCHG_LOOP (__int128_t) +} + +int +main (void) +{ + if (__builtin_cpu_supports ("cmpxchg16b")) + do_test (); + return 0; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4a.c b/gcc/testsuite/gcc.target/i386/pr126293-4a.c new file mode 100644 index 00000000000..547f5fc9469 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4a.c @@ -0,0 +1,31 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rdi, %xmm0 +** movq %rsi, %xmm1 +** punpcklqdq %xmm1, %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#ifndef ATTRIBUTE +#define ATTRIBUTE +#endif + +extern _Atomic __uint128_t store; + +ATTRIBUTE +void +func (__uint128_t i) +{ + store = i; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4b.c b/gcc/testsuite/gcc.target/i386/pr126293-4b.c new file mode 100644 index 00000000000..8a3ec685db8 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4b.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=diamondrapids" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4c.c b/gcc/testsuite/gcc.target/i386/pr126293-4c.c new file mode 100644 index 00000000000..f1848cdb543 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4c.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=nehalem -mavx" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4d.c b/gcc/testsuite/gcc.target/i386/pr126293-4d.c new file mode 100644 index 00000000000..39caeee001b --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4d.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=znver1" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4e.c b/gcc/testsuite/gcc.target/i386/pr126293-4e.c new file mode 100644 index 00000000000..9981ae138b5 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4e.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=yongfeng" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4f.c b/gcc/testsuite/gcc.target/i386/pr126293-4f.c new file mode 100644 index 00000000000..3931a3e8bcc --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4f.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=diamondrapids -mno-avx" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rdi, %xmm0 +** pinsrq \$1, %rsi, %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4g.c b/gcc/testsuite/gcc.target/i386/pr126293-4g.c new file mode 100644 index 00000000000..77ff16507ef --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4g.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=sandybridge -mno-avx" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rdi, %xmm0 +** pinsrq \$1, %rsi, %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4h.c b/gcc/testsuite/gcc.target/i386/pr126293-4h.c new file mode 100644 index 00000000000..6327e4185fe --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4h.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=ivybridge -mno-avx" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rdi, %xmm0 +** pinsrq \$1, %rsi, %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4i.c b/gcc/testsuite/gcc.target/i386/pr126293-4i.c new file mode 100644 index 00000000000..db9f8b49ec3 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4i.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=znver1 -mno-avx" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rdi, %xmm0 +** pinsrq \$1, %rsi, %xmm0 +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4j.c b/gcc/testsuite/gcc.target/i386/pr126293-4j.c new file mode 100644 index 00000000000..1628319b5b5 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4j.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=lujiazui" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rsi, %rdx +** movl \$5, %ecx +** movq %rdi, %rsi +** movl \$store, %edi +** jmp __atomic_store_16 +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4k.c b/gcc/testsuite/gcc.target/i386/pr126293-4k.c new file mode 100644 index 00000000000..76a37927575 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4k.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=skylake -mno-cx16" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rsi, %rdx +** movl \$5, %ecx +** movq %rdi, %rsi +** movl \$store, %edi +** jmp __atomic_store_16 +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4l.c b/gcc/testsuite/gcc.target/i386/pr126293-4l.c new file mode 100644 index 00000000000..4a80ed40efa --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4l.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=skylake -mno-sse" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rsi, %rdx +** movl \$5, %ecx +** movq %rdi, %rsi +** movl \$store, %edi +** jmp __atomic_store_16 +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4m.c b/gcc/testsuite/gcc.target/i386/pr126293-4m.c new file mode 100644 index 00000000000..4ea11e00ff3 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4m.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=grandridge" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4n.c b/gcc/testsuite/gcc.target/i386/pr126293-4n.c new file mode 100644 index 00000000000..f0f92b429ce --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4n.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=skylake -mno-128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rsi, %rdx +** movl \$5, %ecx +** movq %rdi, %rsi +** movl \$store, %edi +** jmp __atomic_store_16 +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4o.c b/gcc/testsuite/gcc.target/i386/pr126293-4o.c new file mode 100644 index 00000000000..6b94a7d59a4 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4o.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=c86-4g-m8" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4p.c b/gcc/testsuite/gcc.target/i386/pr126293-4p.c new file mode 100644 index 00000000000..9b2de450c5d --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4p.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64-v3" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4q.c b/gcc/testsuite/gcc.target/i386/pr126293-4q.c new file mode 100644 index 00000000000..4268a297c2c --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4q.c @@ -0,0 +1,19 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64-v4" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4r.c b/gcc/testsuite/gcc.target/i386/pr126293-4r.c new file mode 100644 index 00000000000..45e8bb7e55c --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4r.c @@ -0,0 +1,20 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** vmovq %rdi, %xmm1 +** vpinsrq \$1, %rsi, %xmm1, %xmm0 +** vmovdqa %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +#define ATTRIBUTE __attribute__ ((target("arch=skylake"))) +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-4s.c b/gcc/testsuite/gcc.target/i386/pr126293-4s.c new file mode 100644 index 00000000000..38eb8bdac28 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-4s.c @@ -0,0 +1,20 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=skylake" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movq %rsi, %rdx +** movl \$5, %ecx +** movq %rdi, %rsi +** movl \$store, %edi +** jmp __atomic_store_16 +** .cfi_endproc +**... +*/ + +#define ATTRIBUTE __attribute__ ((target("arch=x86-64"))) +#include "pr126293-4a.c" diff --git a/gcc/testsuite/gcc.target/i386/pr126293-5.c b/gcc/testsuite/gcc.target/i386/pr126293-5.c new file mode 100644 index 00000000000..2f0794132c9 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-5.c @@ -0,0 +1,25 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movdqa load\(%rip\), %xmm0 +** movaps %xmm0, -24\(%rsp\) +** movq -24\(%rsp\), %rax +** movq -16\(%rsp\), %rdx +** ret +** .cfi_endproc +**... +*/ + +_Atomic __uint128_t load; + +__uint128_t +func (void) +{ + return load; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-6.c b/gcc/testsuite/gcc.target/i386/pr126293-6.c new file mode 100644 index 00000000000..5ad0b4b6a7e --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-6.c @@ -0,0 +1,24 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +typedef int vector __attribute__((vector_size (16))); +extern _Atomic vector store; + +void +func (vector i) +{ + store = i; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-7.c b/gcc/testsuite/gcc.target/i386/pr126293-7.c new file mode 100644 index 00000000000..0f5073ea985 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-7.c @@ -0,0 +1,23 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movdqa load\(%rip\), %xmm0 +** ret +** .cfi_endproc +**... +*/ + +typedef int vector __attribute__((vector_size (16))); +extern _Atomic vector load; + +vector +func (void) +{ + return load; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-8.c b/gcc/testsuite/gcc.target/i386/pr126293-8.c new file mode 100644 index 00000000000..5c9abe02dd3 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-8.c @@ -0,0 +1,24 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movaps %xmm0, store\(%rip\) +** lock orq \$0, \(%rsp\) +** ret +** .cfi_endproc +**... +*/ + +typedef float vector __attribute__((vector_size (16))); +extern _Atomic vector store; + +void +func (vector i) +{ + store = i; +} diff --git a/gcc/testsuite/gcc.target/i386/pr126293-9.c b/gcc/testsuite/gcc.target/i386/pr126293-9.c new file mode 100644 index 00000000000..623498711f7 --- /dev/null +++ b/gcc/testsuite/gcc.target/i386/pr126293-9.c @@ -0,0 +1,23 @@ +/* { dg-do compile { target { ! ia32 } } } */ +/* { dg-additional-options "-O2 -fno-pic -std=c23 -march=x86-64 -m128bit-atomic" } */ +/* Keep labels and directives ('.cfi_startproc', '.cfi_endproc'). */ +/* { dg-final { check-function-bodies "**" "" "" { target *-*-linux* } {^\t?\.} } } */ + +/* +**func: +**.LFB0: +** .cfi_startproc +** movdqa load\(%rip\), %xmm0 +** ret +** .cfi_endproc +**... +*/ + +typedef float vector __attribute__((vector_size (16))); +extern _Atomic vector load; + +vector +func (void) +{ + return load; +} -- 2.55.0
