llvmorg-github-actions[bot] wrote:
<!--LLVM PR SUMMARY COMMENT-->
@llvm/pr-subscribers-clang-codegen
Author: Akash Manna (akash-manna-sky)
<details>
<summary>Changes</summary>
[Clang] Accept vector types in the atomic builtins
Fixes #<!-- -->213237
`atomicrmw` operates elementwise on vector operands, but Sema rejected every
vector value type, so none of the atomic builtins could be used with one:
```c
typedef _Float16 half2 __attribute__((ext_vector_type(2)));
half2 f(half2 *p, half2 v) {
return __scoped_atomic_fetch_add(p, v, __ATOMIC_RELAXED,
__MEMORY_SCOPE_DEVICE);
// error: address argument to atomic operation must be a pointer to
// integer, pointer or supported floating point type
}
```
The same failure occurred for the `__atomic`, `__c11_atomic`, `__hip_atomic`
and `__opencl_atomic` spellings.
## Sema
`BuildAtomicExpr` tested `isIntegerType()`, `isPointerType()` and
`isFloatingType()` directly on the value type, and all three are false for a
`VectorType`. This checks the element type instead, which matches how the LLVM
verifier states the rule for `atomicrmw` (`isFPOrFPVectorTy()` /
`isIntOrIntVectorTy()`). Vectors of `_Bool` remain rejected, since their
elements are not individually addressable.
The arithmetic operations are the one family with no libcall fallback --
`EmitAtomicExpr()` reaches an `llvm_unreachable` on the libcall path -- so Sema
also rejects a vector that cannot be emitted inline, rather than crashing on
`float3` (not a power-of-two size) or `float8` (larger than the 16 bytes
CodeGen emits without a libcall). This is reported by a new diagnostic,
`err_atomic_op_needs_supported_vector`.
## CodeGen
Two matching assumptions had to move with it:
* the `atomicrmw` opcode was selected from
`getValueType()->isFloatingType()`,
which would have picked the integer `Add` for a vector of floats;
* `shouldCastToInt()` bitcast any non-scalar operand to `iN`. That is required
for `cmpxchg`, which has no vector form, and harmless for load and store, but
wrong for arithmetic: an `atomicrmw add` on an `i32` propagates carries across
the lanes of a `<2 x i16>`. A vector whose size is not a power of two
is still
accessed as an integer, as it is not a valid atomic operand, so existing
codegen for e.g. `_Atomic(float3)` is unchanged.
The same `shouldCastToInt()` rule is mirrored in `CIRGenAtomic.cpp`, so
`-fclangir` reports its existing "unsupported type" error instead of silently
emitting an integer operation.
## Tests
* `clang/test/Sema/atomic-ops-vector.c` (new) -- accepted and rejected vector
types across the GNU, C11 and scoped forms, on `x86_64` and `amdgcn`.
* `clang/test/CodeGen/atomic-ops-vector.c` (new) -- verifies the emitted
`atomicrmw` / `load atomic` / `store atomic` uses the vector type directly,
including the non-power-of-two integer fallback and the AMDGPU `syncscope`.
Fixes #<!-- -->213237
---
Patch is 21.73 KiB, truncated to 20.00 KiB below, full version:
https://github.com/llvm/llvm-project/pull/214653.diff
7 Files Affected:
- (modified) clang/docs/ReleaseNotes.md (+6)
- (modified) clang/include/clang/Basic/DiagnosticSemaKinds.td (+4)
- (modified) clang/lib/CIR/CodeGen/CIRGenAtomic.cpp (+5)
- (modified) clang/lib/CodeGen/CGAtomic.cpp (+43-29)
- (modified) clang/lib/Sema/SemaChecking.cpp (+33-1)
- (added) clang/test/CodeGen/atomic-ops-vector.c (+95)
- (added) clang/test/Sema/atomic-ops-vector.c (+65)
``````````diff
diff --git a/clang/docs/ReleaseNotes.md b/clang/docs/ReleaseNotes.md
index 7976b82b63f6e..7764ec68e8e91 100644
--- a/clang/docs/ReleaseNotes.md
+++ b/clang/docs/ReleaseNotes.md
@@ -169,6 +169,12 @@ features cannot lower the translation-unit ABI level;
### Non-comprehensive list of changes in this release
+- The `__atomic`, `__c11_atomic`, `__hip_atomic`, `__opencl_atomic` and
+ `__scoped_atomic` builtins now accept vector types. The operation is
performed
+ elementwise by a single atomic instruction. The arithmetic operations require
+ the vector to have a power-of-two size of at most 16 bytes, as they cannot be
+ lowered to a libcall.
+
### New Compiler Flags
- New option `-fdefined-pointer-subtraction` added to preserve stable semantics
diff --git a/clang/include/clang/Basic/DiagnosticSemaKinds.td
b/clang/include/clang/Basic/DiagnosticSemaKinds.td
index b57ff6c197d1d..4da92e5170d45 100644
--- a/clang/include/clang/Basic/DiagnosticSemaKinds.td
+++ b/clang/include/clang/Basic/DiagnosticSemaKinds.td
@@ -9704,6 +9704,10 @@ def err_atomic_op_needs_atomic_int : Error<
def err_atomic_op_needs_atomic_fp
: Error<"address argument to atomic operation must be a pointer to "
"%select{|atomic }0floating point type (%1 invalid)">;
+def err_atomic_op_needs_supported_vector
+ : Error<"address argument to atomic operation must be a pointer to a "
+ "vector with a power-of-two size of at most %0 bytes "
+ "(%1 invalid)">;
def warn_atomic_op_has_invalid_memory_order : Warning<
"%select{|success |failure }0memory order argument to atomic operation is
invalid">,
InGroup<DiagGroup<"atomic-memory-ordering">>;
diff --git a/clang/lib/CIR/CodeGen/CIRGenAtomic.cpp
b/clang/lib/CIR/CodeGen/CIRGenAtomic.cpp
index 4a028e0be34f7..21b145a9cbee4 100644
--- a/clang/lib/CIR/CodeGen/CIRGenAtomic.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenAtomic.cpp
@@ -326,6 +326,11 @@ bool AtomicInfo::emitMemSetZeroIfNecessary() const {
/// floating point operands. TODO: Allow compare-and-exchange and FP - see
/// comment in CIRGenAtomicExpandPass.cpp.
static bool shouldCastToInt(mlir::Type valueTy, bool cmpxchg) {
+ // The atomic operations act on a vector directly, which is required for the
+ // elementwise arithmetic operations. cmpxchg has no vector form, so it is
+ // still handled as an integer of the same size.
+ if (isa<cir::VectorType>(valueTy))
+ return cmpxchg;
if (cir::isAnyFloatingPointType(valueTy))
return isa<cir::FP80Type>(valueTy) || cmpxchg;
return !isa<cir::IntType>(valueTy) && !isa<cir::PointerType>(valueTy);
diff --git a/clang/lib/CodeGen/CGAtomic.cpp b/clang/lib/CodeGen/CGAtomic.cpp
index cd7b278eab15d..2760cd9b03eb6 100644
--- a/clang/lib/CodeGen/CGAtomic.cpp
+++ b/clang/lib/CodeGen/CGAtomic.cpp
@@ -21,6 +21,7 @@
#include "llvm/ADT/DenseMap.h"
#include "llvm/IR/DataLayout.h"
#include "llvm/IR/Intrinsics.h"
+#include "llvm/Support/MathExtras.h"
using namespace clang;
using namespace CodeGen;
@@ -529,7 +530,7 @@ static llvm::Value *EmitPostAtomicMinMax(CGBuilderTy
&Builder,
bool IsSigned,
llvm::Value *OldVal,
llvm::Value *RHS) {
- const bool IsFP = OldVal->getType()->isFloatingPointTy();
+ const bool IsFP = OldVal->getType()->isFPOrFPVectorTy();
if (IsFP) {
llvm::Intrinsic::ID IID = (Op == AtomicExpr::AO__atomic_max_fetch ||
@@ -567,6 +568,13 @@ static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr
*E, Address Dest,
bool PostOpMinMax = false;
unsigned PostOp = 0;
+ // Atomic operations on a vector are performed elementwise, so it is the
+ // element type which selects between the integer and the floating point
+ // form of an operation.
+ QualType ElemTy = E->getValueType();
+ if (const auto *VecTy = ElemTy->getAs<VectorType>())
+ ElemTy = VecTy->getElementType();
+
switch (E->getOp()) {
case AtomicExpr::AO__c11_atomic_init:
case AtomicExpr::AO__opencl_atomic_init:
@@ -666,30 +674,30 @@ static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr
*E, Address Dest,
case AtomicExpr::AO__atomic_add_fetch:
case AtomicExpr::AO__scoped_atomic_add_fetch:
- PostOp = E->getValueType()->isFloatingType() ? llvm::Instruction::FAdd
- : llvm::Instruction::Add;
+ PostOp = ElemTy->isFloatingType() ? llvm::Instruction::FAdd
+ : llvm::Instruction::Add;
[[fallthrough]];
case AtomicExpr::AO__c11_atomic_fetch_add:
case AtomicExpr::AO__hip_atomic_fetch_add:
case AtomicExpr::AO__opencl_atomic_fetch_add:
case AtomicExpr::AO__atomic_fetch_add:
case AtomicExpr::AO__scoped_atomic_fetch_add:
- Op = E->getValueType()->isFloatingType() ? llvm::AtomicRMWInst::FAdd
- : llvm::AtomicRMWInst::Add;
+ Op = ElemTy->isFloatingType() ? llvm::AtomicRMWInst::FAdd
+ : llvm::AtomicRMWInst::Add;
break;
case AtomicExpr::AO__atomic_sub_fetch:
case AtomicExpr::AO__scoped_atomic_sub_fetch:
- PostOp = E->getValueType()->isFloatingType() ? llvm::Instruction::FSub
- : llvm::Instruction::Sub;
+ PostOp = ElemTy->isFloatingType() ? llvm::Instruction::FSub
+ : llvm::Instruction::Sub;
[[fallthrough]];
case AtomicExpr::AO__c11_atomic_fetch_sub:
case AtomicExpr::AO__hip_atomic_fetch_sub:
case AtomicExpr::AO__opencl_atomic_fetch_sub:
case AtomicExpr::AO__atomic_fetch_sub:
case AtomicExpr::AO__scoped_atomic_fetch_sub:
- Op = E->getValueType()->isFloatingType() ? llvm::AtomicRMWInst::FSub
- : llvm::AtomicRMWInst::Sub;
+ Op = ElemTy->isFloatingType() ? llvm::AtomicRMWInst::FSub
+ : llvm::AtomicRMWInst::Sub;
break;
case AtomicExpr::AO__atomic_min_fetch:
@@ -701,23 +709,22 @@ static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr
*E, Address Dest,
case AtomicExpr::AO__opencl_atomic_fetch_min:
case AtomicExpr::AO__atomic_fetch_min:
case AtomicExpr::AO__scoped_atomic_fetch_min:
- Op = E->getValueType()->isFloatingType()
- ? llvm::AtomicRMWInst::FMin
- : (E->getValueType()->isSignedIntegerType()
- ? llvm::AtomicRMWInst::Min
- : llvm::AtomicRMWInst::UMin);
+ Op = ElemTy->isFloatingType() ? llvm::AtomicRMWInst::FMin
+ : (ElemTy->isSignedIntegerType()
+ ? llvm::AtomicRMWInst::Min
+ : llvm::AtomicRMWInst::UMin);
break;
case AtomicExpr::AO__atomic_fetch_fminimum:
case AtomicExpr::AO__scoped_atomic_fetch_fminimum:
- assert(E->getValueType()->isFloatingType() &&
+ assert(ElemTy->isFloatingType() &&
"fminimum operations only support floating-point types");
Op = llvm::AtomicRMWInst::FMinimum;
break;
case AtomicExpr::AO__atomic_fetch_fminimum_num:
case AtomicExpr::AO__scoped_atomic_fetch_fminimum_num:
- assert(E->getValueType()->isFloatingType() &&
+ assert(ElemTy->isFloatingType() &&
"fminimum_num operations only support floating-point types");
Op = llvm::AtomicRMWInst::FMinimumNum;
break;
@@ -731,23 +738,22 @@ static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr
*E, Address Dest,
case AtomicExpr::AO__opencl_atomic_fetch_max:
case AtomicExpr::AO__atomic_fetch_max:
case AtomicExpr::AO__scoped_atomic_fetch_max:
- Op = E->getValueType()->isFloatingType()
- ? llvm::AtomicRMWInst::FMax
- : (E->getValueType()->isSignedIntegerType()
- ? llvm::AtomicRMWInst::Max
- : llvm::AtomicRMWInst::UMax);
+ Op = ElemTy->isFloatingType() ? llvm::AtomicRMWInst::FMax
+ : (ElemTy->isSignedIntegerType()
+ ? llvm::AtomicRMWInst::Max
+ : llvm::AtomicRMWInst::UMax);
break;
case AtomicExpr::AO__atomic_fetch_fmaximum:
case AtomicExpr::AO__scoped_atomic_fetch_fmaximum:
- assert(E->getValueType()->isFloatingType() &&
+ assert(ElemTy->isFloatingType() &&
"fmaximum operations only support floating-point types");
Op = llvm::AtomicRMWInst::FMaximum;
break;
case AtomicExpr::AO__atomic_fetch_fmaximum_num:
case AtomicExpr::AO__scoped_atomic_fetch_fmaximum_num:
- assert(E->getValueType()->isFloatingType() &&
+ assert(ElemTy->isFloatingType() &&
"fmaximum_num operations only support floating-point types");
Op = llvm::AtomicRMWInst::FMaximumNum;
break;
@@ -840,8 +846,8 @@ static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr
*E, Address Dest,
llvm::Value *Result = RMWI;
if (PostOpMinMax)
Result = EmitPostAtomicMinMax(CGF.Builder, E->getOp(),
- E->getValueType()->isSignedIntegerType(),
- RMWI, LoadVal1);
+ ElemTy->isSignedIntegerType(), RMWI,
+ LoadVal1);
else if (PostOp)
Result = CGF.Builder.CreateBinOp((llvm::Instruction::BinaryOps)PostOp,
RMWI,
LoadVal1);
@@ -868,6 +874,13 @@ EmitValToTemp(CodeGenFunction &CGF, Expr *E) {
/// floating point operands. TODO: Allow compare-and-exchange and FP - see
/// comment in AtomicExpandPass.cpp.
static bool shouldCastToInt(llvm::Type *ValTy, bool CmpXchg) {
+ // The atomic instructions operate on a vector directly, which is required
for
+ // the elementwise arithmetic operations. cmpxchg has no vector form, and a
+ // vector whose size is not a power of two is not a valid atomic operand, so
+ // both are still handled as an integer of the enclosing atomic type's size.
+ if (auto *VecTy = dyn_cast<llvm::FixedVectorType>(ValTy))
+ return CmpXchg ||
+
!llvm::isPowerOf2_64(VecTy->getPrimitiveSizeInBits().getFixedValue());
if (ValTy->isFloatingPointTy())
return ValTy->isX86_FP80Ty() || CmpXchg;
return !ValTy->isIntegerTy() && !ValTy->isPointerTy();
@@ -1571,10 +1584,11 @@ RValue AtomicInfo::ConvertToValueOrAtomic(llvm::Value
*Val,
SourceLocation Loc, bool AsValue,
bool CmpXchg) const {
// Try not to in some easy cases.
- assert((Val->getType()->isIntegerTy() || Val->getType()->isPointerTy() ||
- Val->getType()->isIEEELikeFPTy()) &&
- "Expected integer, pointer or floating point value when converting "
- "result.");
+ llvm::Type *ValScalarTy = Val->getType()->getScalarType();
+ assert((ValScalarTy->isIntegerTy() || ValScalarTy->isPointerTy() ||
+ ValScalarTy->isIEEELikeFPTy()) &&
+ "Expected integer, pointer or floating point value, or a vector "
+ "thereof, when converting result.");
if (getEvaluationKind() == TEK_Scalar &&
(((!LVal.isBitField() ||
LVal.getBitFieldInfo().Size == ValueSizeInBits) &&
diff --git a/clang/lib/Sema/SemaChecking.cpp b/clang/lib/Sema/SemaChecking.cpp
index 0db040ed90e3f..bd9252f409f3d 100644
--- a/clang/lib/Sema/SemaChecking.cpp
+++ b/clang/lib/Sema/SemaChecking.cpp
@@ -5335,6 +5335,15 @@ ExprResult Sema::BuildAtomicExpr(SourceRange CallRange,
SourceRange ExprRange,
// trivial type errors.
auto IsAllowedValueType = [&](QualType ValType,
unsigned AllowedType) -> bool {
+ // Atomic operations on a vector are performed elementwise, so it is the
+ // element type which decides whether the operation is well-formed. A
+ // vector of _Bool is rejected outright, as its elements are not
+ // individually addressable.
+ if (const auto *VecTy = ValType->getAs<VectorType>()) {
+ if (VecTy->getElementType()->isBooleanType())
+ return false;
+ ValType = VecTy->getElementType();
+ }
bool IsX87LongDouble =
ValType->isSpecificBuiltinType(BuiltinType::LongDouble) &&
&Context.getTargetInfo().getLongDoubleFormat() ==
@@ -5367,6 +5376,26 @@ ExprResult Sema::BuildAtomicExpr(SourceRange CallRange,
SourceRange ExprRange,
<< IsC11 << Ptr->getType() << Ptr->getSourceRange();
return ExprError();
}
+ // An arithmetic operation on a vector is always emitted as a single
+ // atomicrmw instruction; unlike the other forms there is no libcall to
fall
+ // back on for a size which cannot be handled inline. The size of the
vector
+ // itself is checked here, as the type holding it is padded out when the
+ // element count is not a power of two.
+ if (Form == Arithmetic) {
+ if (const auto *VecTy = ValType->getAs<VectorType>()) {
+ // The largest size EmitAtomicExpr() emits without a libcall.
+ constexpr unsigned MaxVectorSizeInBytes = 16;
+ uint64_t VecSizeInBits = Context.getTypeSize(VecTy->getElementType()) *
+ VecTy->getNumElements();
+ if (!llvm::isPowerOf2_64(VecSizeInBits) ||
+ VecSizeInBits > MaxVectorSizeInBytes * Context.getCharWidth()) {
+ Diag(ExprRange.getBegin(),
diag::err_atomic_op_needs_supported_vector)
+ << MaxVectorSizeInBytes << Ptr->getType()
+ << Ptr->getSourceRange();
+ return ExprError();
+ }
+ }
+ }
if (IsC11 && ValType->isPointerType() &&
RequireCompleteType(Ptr->getBeginLoc(), ValType->getPointeeType(),
diag::err_incomplete_type)) {
@@ -5635,7 +5664,10 @@ ExprResult Sema::BuildAtomicExpr(SourceRange CallRange,
SourceRange ExprRange,
? 0
: 1);
- if (ValType->isBitIntType()) {
+ QualType BitIntCandidateTy = ValType;
+ if (const auto *VecTy = ValType->getAs<VectorType>())
+ BitIntCandidateTy = VecTy->getElementType();
+ if (BitIntCandidateTy->isBitIntType()) {
Diag(Ptr->getExprLoc(), diag::err_atomic_builtin_bit_int_prohibit);
return ExprError();
}
diff --git a/clang/test/CodeGen/atomic-ops-vector.c
b/clang/test/CodeGen/atomic-ops-vector.c
new file mode 100644
index 0000000000000..a6bb8d13fef39
--- /dev/null
+++ b/clang/test/CodeGen/atomic-ops-vector.c
@@ -0,0 +1,95 @@
+// RUN: %clang_cc1 %s -emit-llvm -o - -triple=x86_64-unknown-linux-gnu \
+// RUN: -target-feature +cx16 -Wno-atomic-alignment \
+// RUN: | FileCheck --check-prefixes=CHECK,X64 %s
+// RUN: %clang_cc1 %s -emit-llvm -o - -triple=amdgcn-amd-amdhsa \
+// RUN: -Wno-atomic-alignment | FileCheck --check-prefixes=CHECK,AMDGCN %s
+
+// Atomic operations on a vector are performed elementwise by a single atomic
+// instruction, rather than being bitcast to an integer of the same size.
+
+typedef _Float16 half2 __attribute__((ext_vector_type(2)));
+typedef __bf16 bfloat2 __attribute__((ext_vector_type(2)));
+typedef float float2 __attribute__((ext_vector_type(2)));
+typedef float float3 __attribute__((ext_vector_type(3)));
+typedef int int2 __attribute__((ext_vector_type(2)));
+
+// CHECK-LABEL: @test_load(
+half2 test_load(half2 *p) {
+ // CHECK: load atomic <2 x half>, ptr {{.*}} monotonic
+ return __atomic_load_n(p, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_store(
+void test_store(half2 *p, half2 v) {
+ // CHECK: store atomic <2 x half> {{.*}}, ptr {{.*}} monotonic
+ __atomic_store_n(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_exchange(
+half2 test_exchange(half2 *p, half2 v) {
+ // CHECK: atomicrmw xchg ptr {{.*}}, <2 x half> {{.*}} monotonic
+ return __atomic_exchange_n(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_fetch_add_half2(
+half2 test_fetch_add_half2(half2 *p, half2 v) {
+ // CHECK: atomicrmw fadd ptr {{.*}}, <2 x half> {{.*}} monotonic
+ return __atomic_fetch_add(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_fetch_add_bfloat2(
+bfloat2 test_fetch_add_bfloat2(bfloat2 *p, bfloat2 v) {
+ // CHECK: atomicrmw fadd ptr {{.*}}, <2 x bfloat> {{.*}} monotonic
+ return __atomic_fetch_add(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_add_fetch_float2(
+float2 test_add_fetch_float2(float2 *p, float2 v) {
+ // CHECK: atomicrmw fadd ptr {{.*}}, <2 x float> {{.*}} monotonic
+ // CHECK: fadd <2 x float>
+ return __atomic_add_fetch(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_fetch_fmaximum_float2(
+float2 test_fetch_fmaximum_float2(float2 *p, float2 v) {
+ // CHECK: atomicrmw fmaximum ptr {{.*}}, <2 x float> {{.*}} monotonic
+ return __atomic_fetch_fmaximum(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_max_fetch_int2(
+int2 test_max_fetch_int2(int2 *p, int2 v) {
+ // CHECK: atomicrmw max ptr {{.*}}, <2 x i32> {{.*}} monotonic
+ // CHECK: icmp sgt <2 x i32>
+ // CHECK: select <2 x i1>
+ return __atomic_max_fetch(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_nand_fetch_int2(
+int2 test_nand_fetch_int2(int2 *p, int2 v) {
+ // CHECK: atomicrmw nand ptr {{.*}}, <2 x i32> {{.*}} monotonic
+ // CHECK: and <2 x i32>
+ // CHECK: xor <2 x i32>
+ return __atomic_nand_fetch(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_c11_fetch_add(
+half2 test_c11_fetch_add(_Atomic(half2) *p, half2 v) {
+ // CHECK: atomicrmw fadd ptr {{.*}}, <2 x half> {{.*}} monotonic
+ return __c11_atomic_fetch_add(p, v, __ATOMIC_RELAXED);
+}
+
+// CHECK-LABEL: @test_scoped_fetch_add(
+half2 test_scoped_fetch_add(half2 *p, half2 v) {
+ // X64: atomicrmw fadd ptr {{.*}}, <2 x half> {{.*}} monotonic
+ // AMDGCN: atomicrmw fadd ptr {{.*}}, <2 x half> {{.*}} syncscope("agent")
monotonic
+ return __scoped_atomic_fetch_add(p, v, __ATOMIC_RELAXED,
+ __MEMORY_SCOPE_DEVICE);
+}
+
+// A vector whose size is not a power of two is not a valid atomic operand, so
+// it is still accessed as an integer of the size of the type holding it.
+// CHECK-LABEL: @test_load_float3(
+float3 test_load_float3(float3 *p) {
+ // CHECK: load atomic i128, ptr {{.*}} monotonic
+ return __atomic_load_n(p, __ATOMIC_RELAXED);
+}
diff --git a/clang/test/Sema/atomic-ops-vector.c
b/clang/test/Sema/atomic-ops-vector.c
new file mode 100644
index 0000000000000..752bb4a5bb933
--- /dev/null
+++ b/clang/test/Sema/atomic-ops-vector.c
@@ -0,0 +1,65 @@
+// RUN: %clang_cc1 %s -verify -fsyntax-only -triple=x86_64-unknown-linux-gnu
-std=c11
+// RUN: %clang_cc1 %s -verify -fsyntax-only -triple=amdgcn-amd-amdhsa -std=c11
+
+// Atomic builtins accept vector types; the operation is performed elementwise
+// by a single atomicrmw instruction.
+
+typedef _Float16 half2 __attribute__((ext_vector_type(2)));
+typedef __bf16 bfloat2 __attribute__((ext_vector_type(2)));
+typedef float float2 __attribute__((ext_vector_type(2)));
+typedef float float3 __attribute__((ext_vector_type(3)));
+typedef float float8 __attribute__((ext_vector_type(8)));
+typedef int int2 __attribute__((ext_vector_type(2)));
+typedef unsigned int uint4 __attribute__((ext_vector_type(4)));
+typedef _Bool bool8 __attribute__((ext_vector_type(8)));
+typedef _BitInt(16) bitint2 __attribute__((ext_vector_type(2)));
+
+void test_gnu(half2 *h2, bfloat2 *b2, float2 *f2, int2 *i2, uint4 *u4) {
+ (void)__atomic_load_n(h2, __ATOMIC_RELAXED);
+ __atomic_store_n(h2, *h2, __ATOMIC_RELAXED);
+ (void)__atomic_exchange_n(h2, *h2, __ATOMIC_RELAXED);
+
+ (void)__atomic_fetch_add(h2, *h2, __ATOMIC_RELAXED);
+ (void)__atomic_fetch_add(b2, *b2, __ATOMIC_RELAXED);
+ (void)__atomic_fetch_sub(f2, *f2, __ATOMIC_RELAXED);
+ (void)__atomic_add_fetch(f2, *f2, __ATOMIC_RELAXED);
+ (void)__atomic_fetch_min(f2, *f2, __ATOMIC_RELAXED);
+ (void)__atomic_max_fetch(i2, *i2, __ATOMIC_RELAXED);
+ (void)__atomic_fetch_fmaximum(f2, *f2, __ATOMIC_RELAXED);
+ (void)__atomic_fetch_and(i2, *i2, __ATOMIC_RELAXED);
+ (void)__atomic_or_fetch(u4, *u4, __ATOMIC_RELAXED);
+ (void)__atomic_nand_fetch(i2, *i2, __ATOMIC_RELAXED);
+
+ // The integer operations still reject a vector of floating point elements.
+ (void)__atomic_fetch_and(f2, *f2, __ATOMIC_RELAXED); // expected-error
{{must be a pointer to integer}}
+ // The f-prefixed operations still reject a vector of integer elements.
+ (void)__atomic_fetch_fminimum(i2, *i2, __ATOMIC_RELAXED); // expected-error
{{must be a pointer to floating point ...
[truncated]
``````````
</details>
https://github.com/llvm/llvm-project/pull/214653
_______________________________________________
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
