https://github.com/xroche created https://github.com/llvm/llvm-project/pull/204815
This is a proof-of-concept demo. It is not asking anyone to merge it, review it, or endorse the approach. It is one data point for the ongoing bit-precise-integer discussion linked below. Clang rejects `_BitInt(N)` in every atomic context today: the `_Atomic(...)` type specifier, the `__c11_atomic_*` and `__atomic_*` builtins, and so transitively `std::atomic` and `std::atomic_ref`. Two blanket `isBitIntType()` checks disable it, dating to the type's introduction (the `__atomic` builtin half is D84049). `__int128`, the closest analogue, is allowed at both sites. This patch removes the two checks and adds the one piece of codegen the removal exposes. With the checks gone, `_BitInt` flows through the normal integer path. Load, store, exchange, compare-exchange, and bitwise read-modify-write are correct at every width through the existing canonicalizing store and the libcall fallback that `_Atomic` structs already use. Arithmetic read-modify-write is the part that needs new code. A `_BitInt(N)` has a value width N and a larger byte-aligned memory width (`_BitInt(37)` lives in an `i64`). A single `atomicrmw add` on that memory integer carries out of bit N-1 into the padding, leaving a non-canonical value that breaks the next compare-exchange and gives the wrong answer for signed min/max. A wide arithmetic fetch (`_BitInt(256)`) is worse: it hit an `llvm_unreachable` in the libcall path, a compiler crash, because no arbitrary-width `__atomic_fetch_*` libcall exists. Both are handled by emitting a compare-exchange loop that computes at width N and writes back a canonical representation, reusing the existing `EmitAtomicCompareExchange` helper, which already selects the inline `cmpxchg` or the `__atomic_compare_exchange` libcall by size. No-padding inline widths (64, 128) keep the direct `atomicrmw` fast path. The change is small: two deletions in Sema and about 250 lines in `CGAtomic.cpp`, most of it a mechanical op classifier that mirrors the existing atomic-op switch. It was checked against gcc-14, which accepts `_Atomic(_BitInt(N))` and agrees on size, alignment, and the padding representation (cross-compiler compare-exchange interop works both directions). A multi-threaded contention suite (lost-update, torn read/write, CAS, across widths, under ThreadSanitizer) also passes locally. Nothing here is a standard feature or proposed for the standard. Like the other prototypes, it is one input to the bit-precise-integer discussion: https://discourse.llvm.org/t/implementing-p3666r4-bit-precise-integers-in-libc/91070 Out of scope: `_BitInt` wider than 128 bits is x86-only, which is a backend limitation this patch does not touch. Assisted-by: Claude (Anthropic) >From f569e15a314ad34fd4bc12defde59f09fbde137d Mon Sep 17 00:00:00 2001 From: Xavier Roche <[email protected]> Date: Fri, 19 Jun 2026 13:59:45 +0200 Subject: [PATCH] [Clang] Support atomic operations on _BitInt(N) _BitInt(N) was rejected by every atomic path: the _Atomic(...) type specifier, the __c11_atomic_*/__atomic_* builtins, and transitively std::atomic. Two blanket isBitIntType() checks disabled it, dating to the type's introduction (the __atomic builtin half is D84049). __int128, the closest analogue, was allowed at both sites. Lift both rejections so _BitInt flows through the normal integer path. load, store, exchange, compare-exchange, and bitwise read-modify-write are then correct at every width through the existing canonicalizing store and the libcall fallback. Arithmetic read-modify-write needs more care. A single atomicrmw on the padded memory integer of a non-byte-aligned width (e.g. _BitInt(37) in an i64) carries into the padding bits, leaving a non-canonical value that breaks a later compare-exchange and gives wrong signed min/max. A wide arithmetic fetch (e.g. _BitInt(256)) hit an llvm_unreachable in the libcall path, a compiler crash. Both are fixed by emitting a compare-exchange loop that computes at width N and writes back a canonical representation, reusing the existing EmitAtomicCompareExchange helper, which also selects the inline cmpxchg or the __atomic_compare_exchange libcall by size. No-padding inline widths (64, 128) keep the direct atomicrmw fast path. Verified against gcc-14: identical size and alignment for all widths, and cross-compiler compare-exchange interop in both directions, confirming the padding canonicalization matches. Assisted-by: Claude (Anthropic) Co-Authored-By: Claude Opus 4.6 <[email protected]> --- clang/docs/LanguageExtensions.rst | 9 + clang/docs/ReleaseNotes.rst | 6 + clang/lib/CodeGen/CGAtomic.cpp | 267 +++++++++++++++++++++++++++++ clang/lib/Sema/SemaChecking.cpp | 5 - clang/lib/Sema/SemaType.cpp | 2 - clang/test/CodeGen/atomic-bitint.c | 90 ++++++++++ clang/test/Sema/builtins.c | 4 +- clang/test/SemaCXX/ext-int.cpp | 10 +- 8 files changed, 378 insertions(+), 15 deletions(-) create mode 100644 clang/test/CodeGen/atomic-bitint.c diff --git a/clang/docs/LanguageExtensions.rst b/clang/docs/LanguageExtensions.rst index d79d82a175c68..5ff076d3e48ad 100644 --- a/clang/docs/LanguageExtensions.rst +++ b/clang/docs/LanguageExtensions.rst @@ -451,6 +451,15 @@ favor of the standard type. Note: the ABI for ``_BitInt(N)`` is still in the process of being stabilized, so this type should not yet be used in interfaces that require ABI stability. +``_BitInt(N)`` may be used as an atomic type: ``_Atomic(_BitInt(N))``, the +``__c11_atomic_*`` and ``__atomic_*`` builtins, and ``std::atomic`` all accept +it for any width. Widths the target cannot operate on inline are lowered to the +``__atomic_*`` libcalls. For a width whose representation has padding bits (``N`` +not a multiple of the type's alignment, e.g. ``_BitInt(37)``), arithmetic +read-modify-write operations are emitted as a compare-exchange loop that computes +at width ``N``, so the result wraps modulo ``2**N`` and the padding bits stay +canonical. + C keywords supported in all language modes ------------------------------------------ diff --git a/clang/docs/ReleaseNotes.rst b/clang/docs/ReleaseNotes.rst index 7f056abfbbe24..8692da8830dff 100644 --- a/clang/docs/ReleaseNotes.rst +++ b/clang/docs/ReleaseNotes.rst @@ -265,6 +265,12 @@ Non-comprehensive list of changes in this release - Added support for floating point and pointer values in most ``__atomic_`` builtins. +- Atomic operations on ``_BitInt(N)`` are now supported, including + ``_Atomic(_BitInt(N))``, the ``__c11_atomic_*`` / ``__atomic_*`` builtins, and + ``std::atomic``. Widths the target cannot operate on inline use the + ``__atomic_*`` libcalls; arithmetic read-modify-write on a width with padding + bits is emitted as a compare-exchange loop computing at the value width. + - Added ``__builtin_stdc_rotate_left`` and ``__builtin_stdc_rotate_right`` for bit rotation of unsigned integers including ``_BitInt`` types. Rotation counts are normalized modulo the bit-width and support negative values. diff --git a/clang/lib/CodeGen/CGAtomic.cpp b/clang/lib/CodeGen/CGAtomic.cpp index 270965b109943..1b32c913157e7 100644 --- a/clang/lib/CodeGen/CGAtomic.cpp +++ b/clang/lib/CodeGen/CGAtomic.cpp @@ -558,6 +558,253 @@ static llvm::Value *EmitPostAtomicMinMax(CGBuilderTy &Builder, return Builder.CreateSelect(Cmp, OldVal, RHS, "newval"); } +namespace { +/// The arithmetic/bitwise kind of an atomic read-modify-write, for the _BitInt +/// compare-exchange loop. Exchange is excluded: it never needs the loop. +enum class BitIntRMWKind { + Add, + Sub, + And, + Or, + Xor, + Nand, + Min, + Max, + UIncWrap, + UDecWrap +}; +} // namespace + +/// Classify an atomic op as an arithmetic/bitwise read-modify-write (one that +/// normally lowers to a single `atomicrmw`). Reports the kind and whether the +/// builtin returns the new value (`<op>_fetch`) rather than the old value +/// (`fetch_<op>`). Returns false for exchange/load/store/compare-exchange and +/// any non-RMW op, none of which need the _BitInt loop. +static bool classifyBitIntRMW(AtomicExpr::AtomicOp Op, BitIntRMWKind &Kind, + bool &ReturnsNew) { + switch (Op) { + case AtomicExpr::AO__c11_atomic_fetch_add: + case AtomicExpr::AO__hip_atomic_fetch_add: + case AtomicExpr::AO__opencl_atomic_fetch_add: + case AtomicExpr::AO__atomic_fetch_add: + case AtomicExpr::AO__scoped_atomic_fetch_add: + Kind = BitIntRMWKind::Add, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_add_fetch: + case AtomicExpr::AO__scoped_atomic_add_fetch: + Kind = BitIntRMWKind::Add, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_sub: + case AtomicExpr::AO__hip_atomic_fetch_sub: + case AtomicExpr::AO__opencl_atomic_fetch_sub: + case AtomicExpr::AO__atomic_fetch_sub: + case AtomicExpr::AO__scoped_atomic_fetch_sub: + Kind = BitIntRMWKind::Sub, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_sub_fetch: + case AtomicExpr::AO__scoped_atomic_sub_fetch: + Kind = BitIntRMWKind::Sub, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_and: + case AtomicExpr::AO__hip_atomic_fetch_and: + case AtomicExpr::AO__opencl_atomic_fetch_and: + case AtomicExpr::AO__atomic_fetch_and: + case AtomicExpr::AO__scoped_atomic_fetch_and: + Kind = BitIntRMWKind::And, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_and_fetch: + case AtomicExpr::AO__scoped_atomic_and_fetch: + Kind = BitIntRMWKind::And, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_or: + case AtomicExpr::AO__hip_atomic_fetch_or: + case AtomicExpr::AO__opencl_atomic_fetch_or: + case AtomicExpr::AO__atomic_fetch_or: + case AtomicExpr::AO__scoped_atomic_fetch_or: + Kind = BitIntRMWKind::Or, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_or_fetch: + case AtomicExpr::AO__scoped_atomic_or_fetch: + Kind = BitIntRMWKind::Or, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_xor: + case AtomicExpr::AO__hip_atomic_fetch_xor: + case AtomicExpr::AO__opencl_atomic_fetch_xor: + case AtomicExpr::AO__atomic_fetch_xor: + case AtomicExpr::AO__scoped_atomic_fetch_xor: + Kind = BitIntRMWKind::Xor, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_xor_fetch: + case AtomicExpr::AO__scoped_atomic_xor_fetch: + Kind = BitIntRMWKind::Xor, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_nand: + case AtomicExpr::AO__atomic_fetch_nand: + case AtomicExpr::AO__scoped_atomic_fetch_nand: + Kind = BitIntRMWKind::Nand, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_nand_fetch: + case AtomicExpr::AO__scoped_atomic_nand_fetch: + Kind = BitIntRMWKind::Nand, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_min: + case AtomicExpr::AO__hip_atomic_fetch_min: + case AtomicExpr::AO__opencl_atomic_fetch_min: + case AtomicExpr::AO__atomic_fetch_min: + case AtomicExpr::AO__scoped_atomic_fetch_min: + Kind = BitIntRMWKind::Min, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_min_fetch: + case AtomicExpr::AO__scoped_atomic_min_fetch: + Kind = BitIntRMWKind::Min, ReturnsNew = true; + return true; + case AtomicExpr::AO__c11_atomic_fetch_max: + case AtomicExpr::AO__hip_atomic_fetch_max: + case AtomicExpr::AO__opencl_atomic_fetch_max: + case AtomicExpr::AO__atomic_fetch_max: + case AtomicExpr::AO__scoped_atomic_fetch_max: + Kind = BitIntRMWKind::Max, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_max_fetch: + case AtomicExpr::AO__scoped_atomic_max_fetch: + Kind = BitIntRMWKind::Max, ReturnsNew = true; + return true; + case AtomicExpr::AO__atomic_fetch_uinc: + case AtomicExpr::AO__scoped_atomic_fetch_uinc: + Kind = BitIntRMWKind::UIncWrap, ReturnsNew = false; + return true; + case AtomicExpr::AO__atomic_fetch_udec: + case AtomicExpr::AO__scoped_atomic_fetch_udec: + Kind = BitIntRMWKind::UDecWrap, ReturnsNew = false; + return true; + default: + return false; + } +} + +/// True for a `_BitInt(N)` whose value width N differs from its in-memory width +/// (e.g. `_BitInt(37)` occupies 64 bits), so the high bits are padding. +static bool hasBitIntPadding(QualType T, const ASTContext &C) { + if (const auto *BIT = T->getAs<BitIntType>()) + return BIT->getNumBits() != C.getTypeSize(T); + return false; +} + +/// Compute the new value of a _BitInt atomic RMW at the value width, so the +/// arithmetic wraps mod 2^N and never carries into the padding bits. +static llvm::Value *emitBitIntRMWNewValue(CodeGenFunction &CGF, + BitIntRMWKind Kind, llvm::Value *Old, + llvm::Value *RHS, bool IsSigned) { + CGBuilderTy &B = CGF.Builder; + switch (Kind) { + case BitIntRMWKind::Add: + return B.CreateAdd(Old, RHS); + case BitIntRMWKind::Sub: + return B.CreateSub(Old, RHS); + case BitIntRMWKind::And: + return B.CreateAnd(Old, RHS); + case BitIntRMWKind::Or: + return B.CreateOr(Old, RHS); + case BitIntRMWKind::Xor: + return B.CreateXor(Old, RHS); + case BitIntRMWKind::Nand: + return B.CreateNot(B.CreateAnd(Old, RHS)); + case BitIntRMWKind::Min: { + auto P = IsSigned ? llvm::ICmpInst::ICMP_SLE : llvm::ICmpInst::ICMP_ULE; + return B.CreateSelect(B.CreateICmp(P, Old, RHS), Old, RHS); + } + case BitIntRMWKind::Max: { + auto P = IsSigned ? llvm::ICmpInst::ICMP_SGE : llvm::ICmpInst::ICMP_UGE; + return B.CreateSelect(B.CreateICmp(P, Old, RHS), Old, RHS); + } + case BitIntRMWKind::UIncWrap: { + // Matches atomicrmw uinc_wrap: (old u>= val) ? 0 : old + 1. + llvm::Value *Zero = llvm::ConstantInt::get(Old->getType(), 0); + llvm::Value *One = llvm::ConstantInt::get(Old->getType(), 1); + return B.CreateSelect(B.CreateICmpUGE(Old, RHS), Zero, + B.CreateAdd(Old, One)); + } + case BitIntRMWKind::UDecWrap: { + // Matches atomicrmw udec_wrap: (old == 0 || old u> val) ? val : old - 1. + llvm::Value *Zero = llvm::ConstantInt::get(Old->getType(), 0); + llvm::Value *One = llvm::ConstantInt::get(Old->getType(), 1); + llvm::Value *Wrap = + B.CreateOr(B.CreateICmpEQ(Old, Zero), B.CreateICmpUGT(Old, RHS)); + return B.CreateSelect(Wrap, RHS, B.CreateSub(Old, One)); + } + } + llvm_unreachable("unknown BitIntRMWKind"); +} + +/// Map a constant C ABI memory order to an llvm ordering. A non-constant order +/// is handled conservatively with the strongest ordering. +static llvm::AtomicOrdering atomicOrderOrSeqCst(llvm::Value *Order) { + auto *C = dyn_cast<llvm::ConstantInt>(Order); + if (!C || !llvm::isValidAtomicOrderingCABI(C->getZExtValue())) + return llvm::AtomicOrdering::SequentiallyConsistent; + switch ((llvm::AtomicOrderingCABI)C->getZExtValue()) { + case llvm::AtomicOrderingCABI::relaxed: + return llvm::AtomicOrdering::Monotonic; + case llvm::AtomicOrderingCABI::consume: + case llvm::AtomicOrderingCABI::acquire: + return llvm::AtomicOrdering::Acquire; + case llvm::AtomicOrderingCABI::release: + return llvm::AtomicOrdering::Release; + case llvm::AtomicOrderingCABI::acq_rel: + return llvm::AtomicOrdering::AcquireRelease; + case llvm::AtomicOrderingCABI::seq_cst: + return llvm::AtomicOrdering::SequentiallyConsistent; + } + llvm_unreachable("invalid CABI ordering"); +} + +/// Emit a `_BitInt(N)` atomic read-modify-write as a compare-exchange loop. A +/// single `atomicrmw` on the padded memory integer would carry into / compare +/// the padding bits, and no arbitrary-width `__atomic_fetch_*` libcall exists +/// for wide widths. The loop computes the new value at width N and writes back +/// a canonical (extended) representation via the existing cmpxchg helper, which +/// also picks the inline-vs-libcall form by size. +static RValue emitBitIntAtomicRMWLoop(CodeGenFunction &CGF, AtomicExpr *E, + Address Ptr, Address Val1, + QualType AtomicTy, BitIntRMWKind Kind, + bool ReturnsNew, llvm::Value *Order) { + QualType ValTy = E->getValueType(); + bool IsSigned = ValTy->isSignedIntegerType(); + llvm::AtomicOrdering AO = atomicOrderOrSeqCst(Order); + llvm::AtomicOrdering Failure = + llvm::AtomicCmpXchgInst::getStrongestFailureOrdering(AO); + + LValue AtomicLVal = CGF.MakeAddrLValue(Ptr, AtomicTy); + AtomicInfo Atomics(CGF, AtomicLVal); + + llvm::Value *RHS = + CGF.EmitLoadOfScalar(CGF.MakeAddrLValue(Val1, ValTy), E->getExprLoc()); + + RValue OldRV = Atomics.EmitAtomicLoad( + AggValueSlot::ignored(), E->getExprLoc(), + /*AsValue=*/true, llvm::AtomicOrdering::Monotonic, E->isVolatile()); + llvm::Value *Init = OldRV.getScalarVal(); + + llvm::BasicBlock *StartBB = CGF.Builder.GetInsertBlock(); + llvm::BasicBlock *LoopBB = CGF.createBasicBlock("atomicrmw.start", CGF.CurFn); + llvm::BasicBlock *EndBB = CGF.createBasicBlock("atomicrmw.end", CGF.CurFn); + CGF.Builder.CreateBr(LoopBB); + CGF.Builder.SetInsertPoint(LoopBB); + + llvm::PHINode *Old = CGF.Builder.CreatePHI(Init->getType(), 2); + Old->addIncoming(Init, StartBB); + + llvm::Value *New = emitBitIntRMWNewValue(CGF, Kind, Old, RHS, IsSigned); + + auto Res = Atomics.EmitAtomicCompareExchange( + RValue::get(Old), RValue::get(New), AO, Failure, /*IsWeak=*/true); + Old->addIncoming(Res.first.getScalarVal(), CGF.Builder.GetInsertBlock()); + CGF.Builder.CreateCondBr(Res.second, EndBB, LoopBB); + + CGF.Builder.SetInsertPoint(EndBB); + return RValue::get(ReturnsNew ? New : static_cast<llvm::Value *>(Old)); +} + static void EmitAtomicOp(CodeGenFunction &CGF, AtomicExpr *E, Address Dest, Address Ptr, Address Val1, Address Val2, Address ExpectedResult, llvm::Value *IsWeak, @@ -1109,6 +1356,26 @@ RValue CodeGenFunction::EmitAtomicExpr(AtomicExpr *E) { LValue AtomicVal = MakeAddrLValue(Ptr, AtomicTy); AtomicInfo Atomics(*this, AtomicVal); + // A `_BitInt(N)` read-modify-write whose value width has padding bits, or + // whose size forces a libcall, cannot use a single atomicrmw: the op would + // carry into / compare the padding bits, and no arbitrary-width + // __atomic_fetch_* libcall exists. Emit a compare-exchange loop instead. + // Bitwise and/or/xor are exact even with padding, so only the wide case needs + // the loop for them. load/store/exchange/compare_exchange keep their paths. + if (MemTy->isBitIntType()) { + BitIntRMWKind RMWKind; + bool RMWReturnsNew; + if (classifyBitIntRMW(E->getOp(), RMWKind, RMWReturnsNew)) { + bool WideOrNonPow2 = (Size & (Size - 1)) != 0 || Size > 16; + bool Bitwise = RMWKind == BitIntRMWKind::And || + RMWKind == BitIntRMWKind::Or || + RMWKind == BitIntRMWKind::Xor; + if (WideOrNonPow2 || (hasBitIntPadding(MemTy, getContext()) && !Bitwise)) + return emitBitIntAtomicRMWLoop(*this, E, Ptr, Val1, AtomicTy, RMWKind, + RMWReturnsNew, Order); + } + } + Address OriginalVal1 = Val1; if (ShouldCastToIntPtrTy) { Ptr = Atomics.castToAtomicIntPointer(Ptr); diff --git a/clang/lib/Sema/SemaChecking.cpp b/clang/lib/Sema/SemaChecking.cpp index b8a3f48a32f24..874ce2bf1ce3a 100644 --- a/clang/lib/Sema/SemaChecking.cpp +++ b/clang/lib/Sema/SemaChecking.cpp @@ -5460,11 +5460,6 @@ ExprResult Sema::BuildAtomicExpr(SourceRange CallRange, SourceRange ExprRange, ? 0 : 1); - if (ValType->isBitIntType()) { - Diag(Ptr->getExprLoc(), diag::err_atomic_builtin_bit_int_prohibit); - return ExprError(); - } - return AE; } diff --git a/clang/lib/Sema/SemaType.cpp b/clang/lib/Sema/SemaType.cpp index d2bb312feadc1..4a3506c281acf 100644 --- a/clang/lib/Sema/SemaType.cpp +++ b/clang/lib/Sema/SemaType.cpp @@ -10412,8 +10412,6 @@ QualType Sema::BuildAtomicType(QualType T, SourceLocation Loc) { else if (!T.isTriviallyCopyableType(Context) && getLangOpts().CPlusPlus) // Some other non-trivially-copyable type (probably a C++ class) DisallowedKind = 7; - else if (T->isBitIntType()) - DisallowedKind = 8; else if (getLangOpts().C23 && T->isUndeducedAutoType()) // _Atomic auto is prohibited in C23 DisallowedKind = 9; diff --git a/clang/test/CodeGen/atomic-bitint.c b/clang/test/CodeGen/atomic-bitint.c new file mode 100644 index 0000000000000..358b530e8a792 --- /dev/null +++ b/clang/test/CodeGen/atomic-bitint.c @@ -0,0 +1,90 @@ +// RUN: %clang_cc1 -std=c23 -triple x86_64-unknown-linux-gnu -emit-llvm %s -o - | FileCheck %s +// +// Atomic operations on _BitInt(N). load/store/exchange/compare-exchange and +// bitwise RMW lower directly; arithmetic RMW on a padded width and any RMW on a +// width too wide for an inline atomicrmw lower to a compare-exchange loop that +// computes at the value width. + +typedef _BitInt(37) S37; +typedef unsigned _BitInt(37) U37; +typedef _BitInt(64) S64; +typedef _BitInt(128) S128; +typedef _BitInt(256) S256; + +// CHECK-LABEL: @ld37( +// CHECK: load atomic i64 +S37 ld37(_Atomic(S37) *p) { return __c11_atomic_load(p, __ATOMIC_SEQ_CST); } + +// CHECK-LABEL: @st37( +// CHECK: store atomic i64 +void st37(_Atomic(S37) *p, S37 v) { __c11_atomic_store(p, v, __ATOMIC_SEQ_CST); } + +// CHECK-LABEL: @xchg37( +// CHECK: atomicrmw xchg ptr {{.*}} i64 +S37 xchg37(_Atomic(S37) *p, S37 v) { + return __c11_atomic_exchange(p, v, __ATOMIC_SEQ_CST); +} + +// CHECK-LABEL: @cas37( +// CHECK: cmpxchg ptr {{.*}} i64 +_Bool cas37(_Atomic(S37) *p, S37 *e, S37 d) { + return __c11_atomic_compare_exchange_strong(p, e, d, __ATOMIC_SEQ_CST, + __ATOMIC_SEQ_CST); +} + +// Bitwise RMW on a padded width keeps the direct atomicrmw: it is exact. +// CHECK-LABEL: @and37( +// CHECK: atomicrmw and ptr {{.*}} i64 +// CHECK-NOT: cmpxchg +S37 and37(_Atomic(S37) *p, S37 v) { + return __c11_atomic_fetch_and(p, v, __ATOMIC_SEQ_CST); +} + +// Arithmetic RMW on a padded width becomes a compare-exchange loop, not a bare +// atomicrmw that would carry into the padding bits. +// CHECK-LABEL: @add37( +// CHECK: atomicrmw.start: +// CHECK: cmpxchg weak ptr {{.*}} i64 +// CHECK-NOT: atomicrmw add +S37 add37(_Atomic(S37) *p, S37 v) { + return __c11_atomic_fetch_add(p, v, __ATOMIC_SEQ_CST); +} + +// Signed min is computed at the value width, so the sign bit is at bit N-1. +// CHECK-LABEL: @min37( +// CHECK: icmp sle i37 +// CHECK: select i1 +// CHECK: cmpxchg weak ptr {{.*}} i64 +U37 min37(_Atomic(S37) *p, S37 v) { + return __c11_atomic_fetch_min(p, v, __ATOMIC_SEQ_CST); +} + +// No padding: direct atomicrmw, no loop. +// CHECK-LABEL: @add64( +// CHECK: atomicrmw add ptr {{.*}} i64 +// CHECK-NOT: cmpxchg +S64 add64(_Atomic(S64) *p, S64 v) { + return __c11_atomic_fetch_add(p, v, __ATOMIC_SEQ_CST); +} + +// CHECK-LABEL: @add128( +// CHECK: atomicrmw add ptr {{.*}} i128 +S128 add128(_Atomic(S128) *p, S128 v) { + return __c11_atomic_fetch_add(p, v, __ATOMIC_SEQ_CST); +} + +// Wide: no inline atomicrmw and no arbitrary-width __atomic_fetch_add libcall, +// so the loop calls __atomic_compare_exchange. +// CHECK-LABEL: @add256( +// CHECK: call {{.*}}@__atomic_compare_exchange +// CHECK-NOT: cmpxchg +S256 add256(_Atomic(S256) *p, S256 v) { + return __c11_atomic_fetch_add(p, v, __ATOMIC_SEQ_CST); +} + +// Wide bitwise also needs the loop: the wide path has no inline atomicrmw. +// CHECK-LABEL: @or256( +// CHECK: call {{.*}}@__atomic_compare_exchange +S256 or256(_Atomic(S256) *p, S256 v) { + return __c11_atomic_fetch_or(p, v, __ATOMIC_SEQ_CST); +} diff --git a/clang/test/Sema/builtins.c b/clang/test/Sema/builtins.c index b669ee68cdd95..57e0eefdb772b 100644 --- a/clang/test/Sema/builtins.c +++ b/clang/test/Sema/builtins.c @@ -281,7 +281,7 @@ void test_ei_i42i(_BitInt(42) *ptr, int value) { // expected-warning@+1 {{the semantics of this intrinsic changed with GCC version 4.4 - the newer semantics are provided here}} __sync_nand_and_fetch(ptr, value); // expected-error {{atomic memory operand must have a power-of-two size}} - __atomic_fetch_add(ptr, 1, 0); // expected-error {{argument to atomic builtin of type '_BitInt' is not supported}} + __atomic_fetch_add(ptr, 1, 0); // expect success: the GNU atomic builtins support _BitInt } void test_ei_i64i(_BitInt(64) *ptr, int value) { @@ -289,7 +289,7 @@ void test_ei_i64i(_BitInt(64) *ptr, int value) { // expected-warning@+1 {{the semantics of this intrinsic changed with GCC version 4.4 - the newer semantics are provided here}} __sync_nand_and_fetch(ptr, value); // expect success - __atomic_fetch_add(ptr, 1, 0); // expected-error {{argument to atomic builtin of type '_BitInt' is not supported}} + __atomic_fetch_add(ptr, 1, 0); // expect success } void test_ei_ii42(int *ptr, _BitInt(42) value) { diff --git a/clang/test/SemaCXX/ext-int.cpp b/clang/test/SemaCXX/ext-int.cpp index 281ae3d3c1779..f62a07a84200e 100644 --- a/clang/test/SemaCXX/ext-int.cpp +++ b/clang/test/SemaCXX/ext-int.cpp @@ -121,13 +121,11 @@ _Complex _BitInt(3) Cmplx; // expected-error@+1{{'_Complex _BitInt' is invalid}} typedef _Complex _BitInt(3) Cmp; -// Reject cases of _Atomic: -// expected-error@+1{{_Atomic cannot be applied to integer type '_BitInt(4)'}} -_Atomic _BitInt(4) TooSmallAtomic; -// expected-error@+1{{_Atomic cannot be applied to integer type '_BitInt(9)'}} +// _Atomic accepts any _BitInt width: small and non-power-of-2 included. +// Sizes the target cannot lower inline use the __atomic_* libcalls. +_Atomic _BitInt(4) SmallAtomic; _Atomic _BitInt(9) NotPow2Atomic; -// expected-error@+1{{_Atomic cannot be applied to integer type '_BitInt(128)'}} -_Atomic _BitInt(128) JustRightAtomic; +_Atomic _BitInt(128) WideAtomic; // Test result types of Unary/Bitwise/Binary Operations: void Ops() { _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
