https://github.com/adams381 created https://github.com/llvm/llvm-project/pull/215117
The CallConvLowering bridge accepted only `float` and `double`, so a function taking a `_Complex`, or a float in any other format, failed the pass instead of being classified. An all-float aggregate failed for a different reason: its SSE eightbyte coerces to a vector, and the bridge had no way to represent one, so it reported the coercion NYI rather than emitting a wrong signature. Mapping every CIR floating-point type through `FPTypeInterface` covers all of them at once. A `_Complex` maps to the library's complex type and a vector coercion now converts back to a CIR vector. Accepting a `long double` also makes a union holding one classifiable. That exposes the ABI-compatibility flags, which the pass left at the library defaults. They now come from the triple and the compatibility version, which is what lets a `long double` union reach registers on Darwin instead of memory. `updateArgAttrs` appended argument attributes instead of setting them, so a name already present landed in the dictionary twice. CIRGen marks a `_Complex long double` parameter `llvm.noundef`, and the ABI then passes it byval, which wants `llvm.noundef` too. An integer coercion lost its bit-precise flag coming back from the classifier. A struct holding a `_BitInt(128)` then took `__int128`'s 16-byte alignment for its coerce slot instead of 8. Assisted-by: Cursor / claude-opus-5 >From 1932bd2c2faf725f6e0d5905855d25a0063383a9 Mon Sep 17 00:00:00 2001 From: Adam Smith <[email protected]> Date: Sat, 8 Aug 2026 14:36:51 -0700 Subject: [PATCH] [CIR] Accept _Complex and all float formats in x86_64 callconv lowering The CallConvLowering bridge accepted only float and double, so a function taking a _Complex, or a float in any other format, failed the pass instead of being classified. An all-float aggregate failed for a different reason: its SSE eightbyte coerces to a vector, and the bridge had no way to represent one, so it reported the coercion NYI rather than emitting a wrong signature. Mapping every CIR floating-point type through FPTypeInterface covers all of them at once. A _Complex maps to the library's complex type and a vector coercion now converts back to a CIR vector. Accepting a long double also makes a union holding one classifiable. That exposes the ABI-compatibility flags, which the pass left at the library defaults. They now come from the triple and the compatibility version, which is what lets a long double union reach registers on Darwin instead of memory. updateArgAttrs appended argument attributes instead of setting them, so a name already present landed in the dictionary twice. CIRGen marks a _Complex long double parameter llvm.noundef, and the ABI then passes it byval, which wants llvm.noundef too. An integer coercion lost its bit-precise flag coming back from the classifier. A struct holding a _BitInt(128) then took __int128's 16-byte alignment for its coerce slot instead of 8. Assisted-by: Cursor / claude-opus-5 --- clang/include/clang/CIR/Dialect/Passes.h | 3 +- .../Transforms/CallConvLoweringPass.cpp | 99 ++++++---- .../TargetLowering/CIRABIRewriteContext.cpp | 28 +-- clang/lib/CIR/Lowering/CIRPasses.cpp | 32 ++- .../call-conv-lowering-x86_64-abi-compat.c | 20 ++ .../CIR/CodeGen/call-conv-lowering-x86_64.c | 187 ++++++++++++++++++ .../abi-lowering/x86_64-aggregate-nyi.cir | 51 ----- .../Transforms/abi-lowering/x86_64-bitint.cir | 12 +- .../abi-lowering/x86_64-complex.cir | 75 +++++++ .../abi-lowering/x86_64-variadic-call.cir | 15 ++ .../abi-lowering/x86_64-variadic-nyi.cir | 29 --- .../Transforms/abi-lowering/x86_64-vector.cir | 62 ++++++ .../abi-lowering/x86_64-wide-floats.cir | 75 +++++++ 13 files changed, 553 insertions(+), 135 deletions(-) create mode 100644 clang/test/CIR/CodeGen/call-conv-lowering-x86_64-abi-compat.c create mode 100644 clang/test/CIR/Transforms/abi-lowering/x86_64-complex.cir create mode 100644 clang/test/CIR/Transforms/abi-lowering/x86_64-vector.cir create mode 100644 clang/test/CIR/Transforms/abi-lowering/x86_64-wide-floats.cir diff --git a/clang/include/clang/CIR/Dialect/Passes.h b/clang/include/clang/CIR/Dialect/Passes.h index 0b8142fc394bd..888e7b833b1cf 100644 --- a/clang/include/clang/CIR/Dialect/Passes.h +++ b/clang/include/clang/CIR/Dialect/Passes.h @@ -38,7 +38,8 @@ std::unique_ptr<Pass> createTargetLoweringPass(); std::unique_ptr<Pass> createCallConvLoweringPass(); std::unique_ptr<Pass> createCallConvLoweringPass(cir::CallConvTarget target, - llvm::abi::X86AVXABILevel x86AvxAbiLevel); + llvm::abi::X86AVXABILevel x86AvxAbiLevel, + const llvm::abi::ABICompatInfo &x86AbiCompat); std::unique_ptr<Pass> createHoistAllocasPass(); std::unique_ptr<Pass> createLoweringPreparePass(); std::unique_ptr<Pass> createLoweringPreparePass(clang::ASTContext *astCtx); diff --git a/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp b/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp index 193c2b6f4a9dc..10ed3d81f8d56 100644 --- a/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp +++ b/clang/lib/CIR/Dialect/Transforms/CallConvLoweringPass.cpp @@ -62,17 +62,16 @@ namespace mlir { namespace { //===----------------------------------------------------------------------===// -// x86_64 System V classifier bridge (scalar and struct/array types) +// x86_64 System V classifier bridge // // Maps CIR types to llvm::abi::Type, runs the LLVM ABI Lowering Library's // SysV x86_64 classifier, and converts the result back into the // dialect-agnostic mlir::abi::FunctionClassification that CIRABIRewriteContext // consumes. Integer (including `_BitInt` up to 128 bits) / pointer / bool / -// f32 / f64 scalars and struct / union / array aggregates are handled. -// `_Complex`, vectors, wider floats, packed or padded records, and a union no -// member of which spans its declared size are reported NYI by -// classifyX86_64Function so an unsupported signature fails the pass instead of -// being misclassified. +// floating-point scalars are handled, as are struct / union / array aggregates +// and `_Complex`. Vectors, packed or padded records, and a union no member of +// which spans its declared size are reported NYI by classifyX86_64Function so +// an unsupported signature fails the pass instead of being misclassified. //===----------------------------------------------------------------------===// /// Whether a struct's declared argument-passing kind (from the module's @@ -101,10 +100,11 @@ static llvm::Align recordDeclaredAlign(ModuleOp modOp, cir::RecordType recTy, } /// The CIR types the x86_64 bridge handles. Scalars: an integer up to 128 -/// bits (including `_BitInt` and `__int128`), pointer, bool, void, f32, or f64. -/// Aggregates: a complete struct or union whose members are all themselves -/// supported, or an array of a supported element type. Everything else is -/// reported NYI at the reject() choke point in classifyX86_64Function. +/// bits (including `_BitInt` and `__int128`), pointer, bool, void, or any +/// floating-point type. Aggregates: a complete struct or union whose members +/// are all themselves supported, or an array of a supported element type. +/// Also a `_Complex` of a supported element type. Everything else is reported +/// NYI at the reject() choke point in classifyX86_64Function. static bool isSupportedType(mlir::Type ty, const DataLayout &dl) { // A pointer is only handled in the default address space (null) or an // already-lowered target address space. A LangAddressSpaceAttr must be @@ -112,7 +112,11 @@ static bool isSupportedType(mlir::Type ty, const DataLayout &dl) { if (auto ptrTy = dyn_cast<cir::PointerType>(ty)) return !ptrTy.getAddrSpace() || mlir::isa<cir::TargetAddressSpaceAttr>(ptrTy.getAddrSpace()); - if (isa<cir::VoidType, cir::BoolType, cir::SingleType, cir::DoubleType>(ty)) + if (isa<cir::VoidType, cir::BoolType>(ty)) + return true; + // Every CIR floating-point type carries the semantics the classifier + // switches on, so all of them are handled. + if (isa<cir::FPTypeInterface>(ty)) return true; if (auto intTy = dyn_cast<cir::IntType>(ty)) { // Integers up to 64 bits, __int128, and _BitInt up to 128 bits are @@ -129,6 +133,8 @@ static bool isSupportedType(mlir::Type ty, const DataLayout &dl) { return intTy.getWidth() <= 128; return intTy.getWidth() <= 64 || intTy.getWidth() == 128; } + if (auto complexTy = dyn_cast<cir::ComplexType>(ty)) + return isSupportedType(complexTy.getElementType(), dl); if (auto arrTy = dyn_cast<cir::ArrayType>(ty)) return isSupportedType(arrTy.getElementType(), dl); if (auto recTy = dyn_cast<cir::RecordType>(ty)) { @@ -177,7 +183,7 @@ static mlir::Type abiTypeToCIR(const llvm::abi::Type *ty, MLIRContext *ctx) { [&](const llvm::abi::VoidType *) { return cir::VoidType::get(ctx); }) .Case([&](const llvm::abi::IntegerType *intTy) { return cir::IntType::get(ctx, intTy->getSizeInBits().getFixedValue(), - intTy->isSigned()); + intTy->isSigned(), intTy->isBitInt()); }) .Case([&](const llvm::abi::FloatType *fltTy) { return cir::getFloatingPointType(*fltTy->getSemantics(), ctx); @@ -185,6 +191,13 @@ static mlir::Type abiTypeToCIR(const llvm::abi::Type *ty, MLIRContext *ctx) { .Case([&](const llvm::abi::PointerType *) { return cir::PointerType::get(cir::VoidType::get(ctx)); }) + .Case([&](const llvm::abi::VectorType *vecTy) -> mlir::Type { + mlir::Type elemCIR = abiTypeToCIR(vecTy->getElementType(), ctx); + if (!elemCIR) + return nullptr; + return cir::VectorType::get(elemCIR, + vecTy->getNumElements().getFixedValue()); + }) .Case([&](const llvm::abi::RecordType *recTy) -> mlir::Type { SmallVector<mlir::Type> fieldTypes; fieldTypes.reserve(recTy->getFields().size()); @@ -230,13 +243,16 @@ static const llvm::abi::Type *mapCIRType(mlir::Type type, /*Signed=*/false); }) .Case([&](cir::VoidType) { return tb.getVoidType(); }) - .Case([&](cir::SingleType) { - return tb.getFloatType(llvm::APFloat::IEEEsingle(), + .Case([&](cir::FPTypeInterface fpTy) { + // LongDoubleType reports its underlying format's semantics, so the + // classifier sees x87 or IEEE quad rather than the wrapper. + return tb.getFloatType(fpTy.getFloatSemantics(), llvm::Align(dl.getTypeABIAlignment(type))); }) - .Case([&](cir::DoubleType) { - return tb.getFloatType(llvm::APFloat::IEEEdouble(), - llvm::Align(dl.getTypeABIAlignment(type))); + .Case([&](cir::ComplexType complexTy) { + return tb.getComplexType( + mapCIRType(complexTy.getElementType(), typeMapper, dl, modOp), + llvm::Align(dl.getTypeABIAlignment(type))); }) .Case([&](cir::ArrayType arrTy) { const llvm::abi::Type *elemAbi = @@ -296,8 +312,8 @@ static const llvm::abi::Type *mapCIRType(mlir::Type type, /// eightbyte. getDirect keeps canFlatten set so the rewriter can split a /// multi-field coerced struct into individual wire arguments. Any other scalar /// passes in its natural CIR type, which a null coercion denotes. A coercion -/// this bridge cannot represent (an SSE <2 x float>, say) yields std::nullopt -/// so the caller reports NYI rather than silently passing the value unchanged. +/// this bridge cannot represent yields std::nullopt so the caller reports NYI +/// rather than silently passing the value unchanged. /// /// Extend: bool or a sub-register integer needs a signext/zeroext attribute. /// The x86_64 classifier (llvm/lib/ABI/Targets/X86.cpp) only returns Extend @@ -314,12 +330,12 @@ convertABIArgInfo(const llvm::abi::ArgInfo &info, MLIRContext *ctx, if (info.isDirect()) { // The classifier names a coerce type even where it matches the natural // type, so a non-null coerce does not by itself mean a rewrite is needed. - // Leaving a scalar alone also preserves its ABI alignment: abiTypeToCIR - // drops the bit-precise flag, so a _BitInt(128) routed through it would - // come back as !cir.int<s, 128> with __int128's 16-byte alignment instead - // of 8. const llvm::abi::Type *coerceAbi = info.getCoerceToType(); bool isAggregate = isa_and_present<cir::RecordType, cir::ArrayType>(origTy); + // For a _Complex the classifier's coerce is only sometimes the natural + // type, so it has to be read rather than assumed. + bool comparesAgainstCoerce = + coerceAbi && isa_and_present<cir::ComplexType>(origTy); bool coerceIsRegisterTuple = isa_and_present<llvm::abi::RecordType>(coerceAbi); // Compare widths rather than identity: a coerce no wider than the natural @@ -330,15 +346,19 @@ convertABIArgInfo(const llvm::abi::ArgInfo &info, MLIRContext *ctx, bool coerceWidensScalar = origInt && coerceInt && coerceInt->getSizeInBits().getFixedValue() > origInt.getWidth(); - if (!isAggregate && !coerceIsRegisterTuple && !coerceWidensScalar) + // Leaving the rest alone also avoids a lossy round trip: abiTypeToCIR + // drops the LongDoubleType wrapper and a pointer's pointee, so comparing a + // scalar against its own coerce would report a difference that is not one. + if (!isAggregate && !comparesAgainstCoerce && !coerceIsRegisterTuple && + !coerceWidensScalar) return ArgClassification::getDirect(nullptr); - // The coerce must be a type this bridge can represent. One it cannot map - // (an SSE vector, or a nested type it does not handle) yields a null type. - // Report that as NYI instead of leaving the value as an unchanged by-value - // record. mlir::Type coerced = abiTypeToCIR(coerceAbi, ctx); if (!coerced) return std::nullopt; + // Coercing a value to the type it already has would add a memory round + // trip for nothing. + if (comparesAgainstCoerce && coerced == origTy) + return ArgClassification::getDirect(nullptr); return ArgClassification::getDirect(coerced); } if (info.isExtend()) { @@ -412,9 +432,8 @@ static std::optional<FunctionClassification> classifyX86_64Signature( llvm::CallingConv::C, retAbi, argAbi, required); targetInfo.computeInfo(*fi); - // convertABIArgInfo returns nullopt when the classifier picks a coercion - // this bridge cannot represent (e.g. an SSE vector coerce for an all-float - // aggregate). Report it as NYI rather than emitting a wrong signature. + // convertABIArgInfo returns nullopt when the classifier picks a coercion this + // bridge cannot represent. auto nyiCoercion = [&](mlir::Type t) { emitError() << "x86_64 calling-convention lowering not yet " "implemented for the ABI coercion of type " @@ -499,7 +518,18 @@ static bool classifiesSamePrefix(const FunctionClassification &calleeFc, struct CallConvLoweringPass : public impl::CallConvLoweringBase<CallConvLoweringPass> { using CallConvLoweringBase::CallConvLoweringBase; + + CallConvLoweringPass(const CallConvLoweringOptions &options, + const llvm::abi::ABICompatInfo &x86AbiCompat) + : CallConvLoweringBase(options), x86AbiCompat(x86AbiCompat) {} + void runOnOperation() override; + + /// The x86_64 flags whose value depends on the target and the requested ABI + /// compatibility version. Carried outside the pass options because the + /// struct has no command-line parser, so a cir-opt run gets the library + /// defaults rather than a target's values. + llvm::abi::ABICompatInfo x86AbiCompat; }; /// Record on \p fc whether \p returnType is CIR's void. The x86_64 classifier @@ -610,7 +640,7 @@ void CallConvLoweringPass::runOnOperation() { x86TypeMapper.emplace(dl); x86Target = llvm::abi::createX86_64TargetInfo( x86TypeMapper->getTypeBuilder(), x86AvxAbiLevel.getValue(), - /*Has64BitPointers=*/true, llvm::abi::ABICompatInfo()); + /*Has64BitPointers=*/true, x86AbiCompat); } // Classify every cir.func up front. No IR mutation happens here, so @@ -827,9 +857,10 @@ std::unique_ptr<Pass> mlir::createCallConvLoweringPass() { std::unique_ptr<Pass> mlir::createCallConvLoweringPass(cir::CallConvTarget target, - llvm::abi::X86AVXABILevel x86AvxAbiLevel) { + llvm::abi::X86AVXABILevel x86AvxAbiLevel, + const llvm::abi::ABICompatInfo &x86AbiCompat) { CallConvLoweringOptions options; options.target = target; options.x86AvxAbiLevel = x86AvxAbiLevel; - return std::make_unique<CallConvLoweringPass>(options); + return std::make_unique<CallConvLoweringPass>(options, x86AbiCompat); } diff --git a/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp b/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp index a8b7f60b6a014..7c80aa300d642 100644 --- a/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp +++ b/clang/lib/CIR/Dialect/Transforms/TargetLowering/CIRABIRewriteContext.cpp @@ -179,6 +179,11 @@ mlir::Value createIgnoredValue(mlir::OpBuilder &builder, mlir::Location loc, /// llvm.align on Indirect args. Preserves any existing arg attributes on /// retained arg slots. \p origArgTypes provides the pre-rewrite type for /// each arg slot (needed to compute the llvm.byval pointee type). +/// +/// An attribute this function sets can already be present on the arg slot: +/// CIRGen marks a scalar parameter llvm.noundef, and the ABI can then pass that +/// parameter byval, which wants llvm.noundef too. So each name has to be set +/// rather than appended, or the dictionary carries it twice. mlir::ArrayAttr updateArgAttrs(mlir::MLIRContext *ctx, ArrayRef<mlir::Type> origArgTypes, mlir::ArrayAttr existingArgAttrs, @@ -204,9 +209,9 @@ mlir::ArrayAttr updateArgAttrs(mlir::MLIRContext *ctx, newArgAttrs.append(recTy.getNumElements(), builder.getDictionaryAttr({})); } else if (ac.kind == ArgKind::Extend) { StringRef attrName = ac.signExtend ? "llvm.signext" : "llvm.zeroext"; - SmallVector<mlir::NamedAttribute> attrs(existing.begin(), existing.end()); - attrs.push_back(builder.getNamedAttr(attrName, builder.getUnitAttr())); - newArgAttrs.push_back(builder.getDictionaryAttr(attrs)); + mlir::NamedAttrList attrs(existing); + attrs.set(attrName, builder.getUnitAttr()); + newArgAttrs.push_back(attrs.getDictionary(ctx)); } else if (ac.kind == ArgKind::Indirect) { // byval: caller-allocated copy; callee receives pointer to copy. // byref: callee receives pointer to the caller's original storage. @@ -226,18 +231,15 @@ mlir::ArrayAttr updateArgAttrs(mlir::MLIRContext *ctx, // produces a fresh alloca+store. mlir::Type pointeeTy = origArgTypes[oldIdx]; StringRef ownershipAttr = ac.byVal ? "llvm.byval" : "llvm.byref"; - SmallVector<mlir::NamedAttribute> attrs(existing.begin(), existing.end()); - attrs.push_back(builder.getNamedAttr( - "llvm.align", builder.getI64IntegerAttr(ac.indirectAlign.value()))); - attrs.push_back( - builder.getNamedAttr(ownershipAttr, mlir::TypeAttr::get(pointeeTy))); + mlir::NamedAttrList attrs(existing); + attrs.set("llvm.align", + builder.getI64IntegerAttr(ac.indirectAlign.value())); + attrs.set(ownershipAttr, mlir::TypeAttr::get(pointeeTy)); if (ac.byVal) { - attrs.push_back( - builder.getNamedAttr("llvm.noalias", builder.getUnitAttr())); - attrs.push_back( - builder.getNamedAttr("llvm.noundef", builder.getUnitAttr())); + attrs.set("llvm.noalias", builder.getUnitAttr()); + attrs.set("llvm.noundef", builder.getUnitAttr()); } - newArgAttrs.push_back(builder.getDictionaryAttr(attrs)); + newArgAttrs.push_back(attrs.getDictionary(ctx)); } else { newArgAttrs.push_back(existing); } diff --git a/clang/lib/CIR/Lowering/CIRPasses.cpp b/clang/lib/CIR/Lowering/CIRPasses.cpp index 0a683b8c8a498..dad17f2ef8659 100644 --- a/clang/lib/CIR/Lowering/CIRPasses.cpp +++ b/clang/lib/CIR/Lowering/CIRPasses.cpp @@ -13,6 +13,7 @@ #include "mlir/IR/BuiltinOps.h" #include "mlir/Pass/PassManager.h" #include "clang/AST/ASTContext.h" +#include "clang/Basic/LangOptions.h" #include "clang/Basic/TargetInfo.h" #include "clang/CIR/Dialect/Passes.h" #include "llvm/Support/TimeProfiler.h" @@ -28,6 +29,34 @@ static CallConvTarget getCallConvTarget(const llvm::Triple &triple) { return CallConvTarget::None; } +/// The x86_64 ABI-compatibility flags, derived from the target and the +/// requested compatibility version. Every flag defaults to true in the ABI +/// library, which is not what any target computes: Clang11Compat is false for a +/// modern Linux target, so leaving it at the default classifies a union larger +/// than an eightbyte as though every member spanned its size. Mirrors the +/// predicates in clang/lib/CodeGen/Targets/X86.cpp and the derivation in +/// CodeGenModule::getLLVMABITargetInfo, which computes the same five flags for +/// the classic path. +static llvm::abi::ABICompatInfo +getX86ABICompatInfo(const clang::ASTContext &astContext) { + const llvm::Triple &triple = astContext.getTargetInfo().getTriple(); + const clang::LangOptions &langOpts = astContext.getLangOpts(); + clang::LangOptions::ClangABI compat = langOpts.getClangABICompat(); + llvm::abi::ABICompatInfo abiCompat; + abiCompat.HonorsRevision98 = !triple.isOSDarwin(); + abiCompat.ClassifyIntegerMMXAsSSE = + compat > clang::LangOptions::ClangABI::Ver3_8 && !triple.isOSDarwin() && + !triple.isPS() && !triple.isOSFreeBSD(); + abiCompat.PassInt128VectorsInMem = + compat > clang::LangOptions::ClangABI::Ver9 && + (triple.isOSLinux() || triple.isOSNetBSD()); + abiCompat.ReturnCXXRecordGreaterThan128InMem = + compat > clang::LangOptions::ClangABI::Ver20 && !triple.isPS(); + abiCompat.Clang11Compat = + compat <= clang::LangOptions::ClangABI::Ver11 || triple.isPS(); + return abiCompat; +} + mlir::LogicalResult runCIRToCIRPasses(mlir::ModuleOp theModule, mlir::MLIRContext &mlirContext, clang::ASTContext &astContext, bool enableVerifier, @@ -70,7 +99,8 @@ runCIRToCIRPasses(mlir::ModuleOp theModule, mlir::MLIRContext &mlirContext, getCallConvTarget(astContext.getTargetInfo().getTriple()); if (target != CallConvTarget::None) pm.addPass(mlir::createCallConvLoweringPass( - target, llvm::abi::X86AVXABILevel::None)); + target, llvm::abi::X86AVXABILevel::None, + getX86ABICompatInfo(astContext))); } pm.addPass(mlir::createLoweringPreparePass(&astContext)); diff --git a/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-abi-compat.c b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-abi-compat.c new file mode 100644 index 0000000000000..6a1d9a3c4d3d2 --- /dev/null +++ b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64-abi-compat.c @@ -0,0 +1,20 @@ +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -fclangir -clangir-enable-call-conv-lowering -emit-llvm %s -o %t-cir.ll +// RUN: FileCheck --check-prefix=LINUX-CIR --input-file=%t-cir.ll %s +// RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm %s -o %t.ll +// RUN: FileCheck --check-prefix=LINUX-OGCG --input-file=%t.ll %s + +// RUN: %clang_cc1 -triple x86_64-apple-darwin -fclangir -clangir-enable-call-conv-lowering -emit-llvm %s -o %t-darwin-cir.ll +// RUN: FileCheck --check-prefix=DARWIN --input-file=%t-darwin-cir.ll %s +// RUN: %clang_cc1 -triple x86_64-apple-darwin -emit-llvm %s -o %t-darwin.ll +// RUN: FileCheck --check-prefix=DARWIN --input-file=%t-darwin.ll %s + +// The 0.98 ABI revision sends an eightbyte pair to memory when the high half is +// X87UP and the low half is not X87. Darwin exempts itself for binary +// compatibility with older GCC, so the same union passes in registers there. +// The int member is what makes the low half INTEGER rather than X87. +typedef union { long double l; int i; } ULongDouble; +void rev98(ULongDouble u) { (void)u; } + +// LINUX-CIR: define dso_local void @rev98(ptr noalias noundef byval(%union.ULongDouble) align 16 %{{[^,)]+}}) +// LINUX-OGCG: define dso_local void @rev98(ptr noundef byval(%union.ULongDouble) align 16 %{{[^,)]+}}) +// DARWIN: define void @rev98(i64 %{{[^,)]+}}, double %{{[^,)]+}}) diff --git a/clang/test/CIR/CodeGen/call-conv-lowering-x86_64.c b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64.c index 8382b15bb5d9e..ba85536ddf138 100644 --- a/clang/test/CIR/CodeGen/call-conv-lowering-x86_64.c +++ b/clang/test/CIR/CodeGen/call-conv-lowering-x86_64.c @@ -5,6 +5,13 @@ // RUN: %clang_cc1 -triple x86_64-unknown-linux-gnu -emit-llvm %s -o %t.ll // RUN: FileCheck --check-prefixes=LLVM,LLVM-OGCG --input-file=%t.ll %s +// Anonymous record aliases are numbered in the order they are printed, so +// capture each one rather than naming it. +// CIR-DAG: ![[X87PAIR:rec_anon_struct[0-9]*]] = !cir.struct<{!cir.f80, !cir.f80}> +// CIR-DAG: ![[I64PAIR:rec_anon_struct[0-9]*]] = !cir.struct<{!u64i, !u64i}> +// CIR-DAG: ![[F64PAIR:rec_anon_struct[0-9]*]] = !cir.struct<{!cir.double, !cir.double}> +// CIR-DAG: ![[F32X2PAIR:rec_anon_struct[0-9]*]] = !cir.struct<{!cir.vector<2 x !cir.float>, !cir.vector<2 x !cir.float>}> + typedef struct { int x; int y; } Pair2; typedef struct { long a; long b; } Pair16; typedef struct { long a, b, c, d; } Big; @@ -163,3 +170,183 @@ void take_struct_over_aligned(SOverAligned s) { (void)s; } // CIR: cir.func {{.*}}@take_struct_over_aligned(%arg0: !cir.ptr<!rec_SOverAligned> {{.*}}llvm.align = 32 : i64{{.*}}llvm.byval = !rec_SOverAligned{{.*}}) // LLVM-CIR: define dso_local void @take_struct_over_aligned(ptr noalias noundef byval(%struct.SOverAligned) align 32 %{{.+}}) // LLVM-OGCG: define dso_local void @take_struct_over_aligned(ptr noundef byval(%struct.SOverAligned) align 32 %{{.+}}) + +// A half occupies one SSE eightbyte. +_Float16 sse_half(_Float16 h) { return h; } + +// CIR: cir.func {{.*}}@sse_half(%arg0: !cir.f16 {{.*}}) -> !cir.f16 +// LLVM: define dso_local half @sse_half(half noundef %{{.+}}) + +// So does a bfloat. +__bf16 sse_bfloat(__bf16 b) { return b; } + +// CIR: cir.func {{.*}}@sse_bfloat(%arg0: !cir.bf16 {{.*}}) -> !cir.bf16 +// LLVM: define dso_local bfloat @sse_bfloat(bfloat noundef %{{.+}}) + +// __float128 spans an SSE/SSEUP pair, which is still one register pair. +__float128 sse_quad(__float128 q) { return q; } + +// CIR: cir.func {{.*}}@sse_quad(%arg0: !cir.f128 {{.*}}) -> !cir.f128 +// LLVM: define dso_local fp128 @sse_quad(fp128 noundef %{{.+}}) + +// x87 long double is the X87/X87UP pair, returned in st0. +long double x87_long_double(long double l) { return l; } + +// CIR: cir.func {{.*}}@x87_long_double(%arg0: !cir.long_double<!cir.f80> {{.*}}) -> !cir.long_double<!cir.f80> +// LLVM: define dso_local x86_fp80 @x87_long_double(x86_fp80 noundef %{{.+}}) + +// Wrapping the long double in a struct merges the eightbytes to MEMORY, so +// the argument becomes byval while the return still comes back in st0. +typedef struct { long double l; } SLongDouble; +SLongDouble ret_long_double_struct(SLongDouble s) { return s; } + +// CIR: cir.func {{.*}}@ret_long_double_struct(%arg0: !cir.ptr<!rec_SLongDouble> {{.*}}llvm.byval = !rec_SLongDouble{{.*}}) -> !cir.f80 +// LLVM-CIR: define dso_local x86_fp80 @ret_long_double_struct(ptr noalias noundef byval(%struct.SLongDouble) align 16 %{{.+}}) +// LLVM-OGCG: define dso_local x86_fp80 @ret_long_double_struct(ptr noundef byval(%struct.SLongDouble) align 16 %{{.+}}) + +// A union holding a long double is accepted because the long double spans the +// union's declared size. +typedef union { long double l; int i; } ULongDouble; +void take_union_long_double(ULongDouble u) { (void)u; } + +// CIR: cir.func {{.*}}@take_union_long_double(%arg0: !cir.ptr<!rec_ULongDouble> {{.*}}llvm.byval = !rec_ULongDouble{{.*}}) +// LLVM-CIR: define dso_local void @take_union_long_double(ptr noalias noundef byval(%union.ULongDouble) align 16 %{{.+}}) +// LLVM-OGCG: define dso_local void @take_union_long_double(ptr noundef byval(%union.ULongDouble) align 16 %{{.+}}) + +// A _Complex of quads exceeds two eightbytes and goes to memory both ways, so +// the sret and byval pointees here are a _Complex rather than a record. +_Complex __float128 complex_quad(_Complex __float128 z) { return z; } + +// CIR: cir.func {{.*}}@complex_quad(%arg0: !cir.ptr<!cir.complex<!cir.f128>> {{.*}}llvm.sret = !cir.complex<!cir.f128>{{.*}}, %arg1: !cir.ptr<!cir.complex<!cir.f128>> {{.*}}llvm.byval = !cir.complex<!cir.f128>{{.*}}) +// LLVM-CIR: define dso_local void @complex_quad(ptr dead_on_unwind noalias writable sret({ fp128, fp128 }) align 16 %{{[^,)]+}}, ptr noalias noundef byval({ fp128, fp128 }) align 16 %{{[^,)]+}}) +// LLVM-OGCG: define dso_local void @complex_quad(ptr dead_on_unwind noalias writable sret({ fp128, fp128 }) align 16 %{{[^,)]+}}, ptr noundef byval({ fp128, fp128 }) align 16 %{{[^,)]+}}) + +// Both halves of a _Complex float share one SSE eightbyte, so it coerces to +// the two-element vector that eightbyte holds. +_Complex float complex_float(_Complex float c) { return c; } + +// CIR: cir.func {{.*}}@complex_float(%arg0: !cir.vector<2 x !cir.float> {{.*}}) -> !cir.vector<2 x !cir.float> +// LLVM: define dso_local <2 x float> @complex_float(<2 x float> noundef %{{.+}}) + +// _Complex double needs two SSE eightbytes, so it flattens into a pair. +// Flattening drops the parameter's noundef, which classic keeps on each half. +_Complex double complex_double(_Complex double c) { return c; } + +// CIR: cir.func {{.*}}@complex_double(%arg0: !cir.double{{.*}}, %arg1: !cir.double{{.*}}) -> ![[F64PAIR]] +// LLVM-CIR: define dso_local { double, double } @complex_double(double %{{[^,)]+}}, double %{{[^,)]+}}) +// LLVM-OGCG: define dso_local { double, double } @complex_double(double noundef %{{[^,)]+}}, double noundef %{{[^,)]+}}) + +// A _Complex of integers packs both halves into one INTEGER eightbyte. +_Complex int complex_int(_Complex int c) { return c; } + +// CIR: cir.func {{.*}}@complex_int(%arg0: !u64i {{.*}}) -> !u64i +// LLVM: define dso_local i64 @complex_int(i64 noundef %{{.+}}) + +// COMPLEX_X87 passes in memory and returns as the st0/st1 pair. +_Complex long double complex_long_double(_Complex long double c) { return c; } + +// CIR: cir.func {{.*}}@complex_long_double(%arg0: !cir.ptr<!cir.complex<!cir.long_double<!cir.f80>>> {{.*}}llvm.byval = !cir.complex<!cir.long_double<!cir.f80>>{{.*}}) -> ![[X87PAIR]] +// LLVM-CIR: define dso_local { x86_fp80, x86_fp80 } @complex_long_double(ptr noalias noundef byval({ x86_fp80, x86_fp80 }) align 16 %{{.+}}) +// LLVM-OGCG: define dso_local { x86_fp80, x86_fp80 } @complex_long_double(ptr noundef byval({ x86_fp80, x86_fp80 }) align 16 %{{.+}}) + +// A _Complex of 16-bit floats fits one eightbyte, so it coerces to the +// two-element vector of that format. +_Complex _Float16 complex_half(_Complex _Float16 c) { return c; } + +// CIR: cir.func {{.*}}@complex_half(%arg0: !cir.vector<2 x !cir.f16> {{.*}}) -> !cir.vector<2 x !cir.f16> +// LLVM: define dso_local <2 x half> @complex_half(<2 x half> noundef %{{[^,)]+}}) + +// A _Complex of 64-bit integers spans two INTEGER eightbytes, so it flattens +// into a register pair instead of coercing to one value. +_Complex long long complex_longlong(_Complex long long c) { return c; } + +// CIR: cir.func {{.*}}@complex_longlong(%arg0: !u64i {{.*}}, %arg1: !u64i {{.*}}) -> ![[I64PAIR]] +// LLVM-CIR: define dso_local { i64, i64 } @complex_longlong(i64 %{{[^,)]+}}, i64 %{{[^,)]+}}) +// LLVM-OGCG: define dso_local { i64, i64 } @complex_longlong(i64 noundef %{{[^,)]+}}, i64 noundef %{{[^,)]+}}) + +// A _Complex reaches the classifier as a record member too, not just on its +// own, so these cover the field walk rather than the top-level mapping. +typedef struct { _Complex float c; } WrapComplexFloat; +void take_wrap_complex_float(WrapComplexFloat s) { (void)s; } + +// CIR: cir.func {{.*}}@take_wrap_complex_float(%arg0: !cir.vector<2 x !cir.float>{{.*}}) +// LLVM: define dso_local void @take_wrap_complex_float(<2 x float> %{{[^,)]+}}) + +typedef struct { _Complex double c; } WrapComplexDouble; +void take_wrap_complex_double(WrapComplexDouble s) { (void)s; } + +// CIR: cir.func {{.*}}@take_wrap_complex_double(%arg0: !cir.double{{.*}}, %arg1: !cir.double{{.*}}) +// LLVM: define dso_local void @take_wrap_complex_double(double %{{[^,)]+}}, double %{{[^,)]+}}) + +// An all-float aggregate's SSE eightbyte coerces to a vector. +typedef struct { float x, y; } TwoFloats; +TwoFloats two_floats(TwoFloats s) { return s; } + +// CIR: cir.func {{.*}}@two_floats(%arg0: !cir.vector<2 x !cir.float> {{.*}}) -> !cir.vector<2 x !cir.float> +// LLVM: define dso_local <2 x float> @two_floats(<2 x float> %{{[^,)]+}}) + +// The same holds for an array of floats inside a struct. +typedef struct { float a[2]; } FloatArray; +void take_float_array(FloatArray s) { (void)s; } + +// CIR: cir.func {{.*}}@take_float_array(%arg0: !cir.vector<2 x !cir.float>{{.*}}) +// LLVM: define dso_local void @take_float_array(<2 x float> %{{[^,)]+}}) + +// A 16-bit float pair coerces to a vector of that same format. +typedef struct { _Float16 a, b; } TwoHalves; +void take_two_halves(TwoHalves s) { (void)s; } + +// CIR: cir.func {{.*}}@take_two_halves(%arg0: !cir.vector<2 x !cir.f16>{{.*}}) +// LLVM: define dso_local void @take_two_halves(<2 x half> %{{[^,)]+}}) + +typedef struct { __bf16 a, b; } TwoBFloats; +void take_two_bfloats(TwoBFloats s) { (void)s; } + +// CIR: cir.func {{.*}}@take_two_bfloats(%arg0: !cir.vector<2 x !cir.bf16>{{.*}}) +// LLVM: define dso_local void @take_two_bfloats(<2 x bfloat> %{{[^,)]+}}) + +// A 16-bit float sharing its eightbyte with a wider float widens the vector to +// the eightbyte rather than to the members. The element format is always +// IEEE half here, so a bfloat pairing this way comes back as half too. +typedef struct { _Float16 h; float f; } HalfThenFloat; +void take_half_then_float(HalfThenFloat s) { (void)s; } + +// CIR: cir.func {{.*}}@take_half_then_float(%arg0: !cir.vector<4 x !cir.f16>{{.*}}) +// LLVM: define dso_local void @take_half_then_float(<4 x half> %{{[^,)]+}}) + +typedef struct { __bf16 b; float f; } BFloatThenFloat; +void take_bfloat_then_float(BFloatThenFloat s) { (void)s; } + +// CIR: cir.func {{.*}}@take_bfloat_then_float(%arg0: !cir.vector<4 x !cir.f16>{{.*}}) +// LLVM: define dso_local void @take_bfloat_then_float(<4 x half> %{{[^,)]+}}) + +// An IEEE quad reaches a register, where an x87 long double of the same width +// would go to memory. +typedef struct { __float128 q; } WrapQuad; +void take_wrap_quad(WrapQuad s) { (void)s; } + +// CIR: cir.func {{.*}}@take_wrap_quad(%arg0: !cir.f128{{.*}}) +// LLVM: define dso_local void @take_wrap_quad(fp128 %{{[^,)]+}}) + +// Three floats span two eightbytes: a vector for the first pair, a scalar for +// the remainder. +typedef struct { float x, y, z; } ThreeFloats; +void take_three_floats(ThreeFloats s) { (void)s; } + +// CIR: cir.func {{.*}}@take_three_floats(%arg0: !cir.vector<2 x !cir.float>{{.*}}, %arg1: !cir.float{{.*}}) +// LLVM: define dso_local void @take_three_floats(<2 x float> %{{[^,)]+}}, float %{{[^,)]+}}) + +// Four floats fill both eightbytes, so the coercion is the one record whose +// every field is a vector, in argument and in return position. +typedef struct { float a, b, c, d; } FourFloats; +FourFloats four_floats(FourFloats s) { return s; } + +// CIR: cir.func {{.*}}@four_floats(%arg0: !cir.vector<2 x !cir.float>{{.*}}, %arg1: !cir.vector<2 x !cir.float>{{.*}}) -> ![[F32X2PAIR]] +// LLVM: define dso_local { <2 x float>, <2 x float> } @four_floats(<2 x float> %{{[^,)]+}}, <2 x float> %{{[^,)]+}}) + +void call_complex_float(_Complex float c) { complex_float(c); } + +// CIR: cir.func {{.*}}@call_complex_float(%arg0: !cir.vector<2 x !cir.float> +// CIR: cir.call @complex_float(%{{.+}}) : (!cir.vector<2 x !cir.float> {llvm.noundef}) -> !cir.vector<2 x !cir.float> +// LLVM: define dso_local void @call_complex_float(<2 x float> noundef %{{.+}}) +// LLVM: call <2 x float> @complex_float(<2 x float> noundef %{{.+}}) diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-aggregate-nyi.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-aggregate-nyi.cir index 83d36ce22b970..4ac76bb2b2735 100644 --- a/clang/test/CIR/Transforms/abi-lowering/x86_64-aggregate-nyi.cir +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-aggregate-nyi.cir @@ -5,8 +5,6 @@ !s32i = !cir.int<s, 32> !u8i = !cir.int<u, 8> !rec_UPacked = !cir.union<"UPacked" packed {!s32i, !cir.array<!s8i x 5>}, padding = {!u8i}> -!rec_ULongDouble = !cir.union<"ULongDouble" {!cir.long_double<!cir.f80>, !s32i}> -!rec_UFloats = !cir.union<"UFloats" {!cir.array<!cir.float x 2>, !cir.array<!cir.float x 2>}> !rec_UOverAligned = !cir.union<"UOverAligned" {!s32i}, padding = {!cir.array<!u8i x 12>}> !rec_UShortStorage = !cir.union<"UShortStorage" {!s16i, !cir.array<!s8i x 3>}, padding = {!cir.array<!u8i x 2>}> !rec_UByteBlobs = !cir.union<"UByteBlobs" {!u8i, !u8i}, padding = {!cir.array<!u8i x 3>}> @@ -15,8 +13,6 @@ !rec_P = !cir.struct<"P" packed {!s8i, !s32i}> !rec_Ov = !cir.struct<"Ov" padded {!s32i, !cir.array<!u8i x 12>}> !rec_E = !cir.struct<"E" padded {!u8i}> -!rec_FF = !cir.struct<"FF" {!cir.float, !cir.float}> -!rec_RetFF = !cir.struct<"RetFF" {!cir.float, !cir.float}> module attributes { dlti.dl_spec = #dlti.dl_spec< @@ -34,22 +30,6 @@ module attributes { // CHECK: not yet implemented for type '!cir.union<"UPacked" packed - // A union member the bridge does not map keeps the whole union unsupported. - cir.func @take_union_long_double(%arg0: !rec_ULongDouble) { - cir.return - } - - // CHECK: not yet implemented for type '!cir.union<"ULongDouble" - - // A union whose highest-aligned member is an all-float array classifies to - // an SSE vector coerce this bridge does not represent, so it is reported NYI - // rather than passed unchanged. - cir.func @take_union_float_arrays(%arg0: !rec_UFloats) { - cir.return - } - - // CHECK: not yet implemented for the ABI coercion of type '!cir.union<"UFloats" - // No member of this union spans its 16-byte declared size, so the bytes past // the int cannot be told apart from the rest of a wider storage unit, and the // eightbyte the classifier would build from the union's size is a guess. @@ -121,35 +101,4 @@ module attributes { // CHECK: not yet implemented for type '!cir.struct<"E" padded - // An all-float struct classifies to an SSE vector coerce this bridge does - // not represent, so it is reported NYI rather than passed unchanged. - cir.func @take_ff(%arg0: !rec_FF) { - cir.return - } - - // CHECK: not yet implemented for the ABI coercion of type '!cir.struct<"FF" - - // The same holds for an all-float array. - cir.func @take_farr(%arg0: !cir.array<!cir.float x 2>) { - cir.return - } - - // CHECK: not yet implemented for the ABI coercion of type '!cir.array<!cir.float x 2> - - // A three-float struct coerces to a record with a vector field; the - // unmappable field propagates out as NYI too. - cir.func @take_f3(%arg0: !cir.struct<"F3" {!cir.float, !cir.float, !cir.float}>) { - cir.return - } - - // CHECK: not yet implemented for the ABI coercion of type '!cir.struct<"F3" - - // The unmappable-coercion check also covers the return value. - cir.func @ret_ff() -> !rec_RetFF { - %0 = cir.alloca "r" align(4) : !cir.ptr<!rec_RetFF> - %1 = cir.load %0 : !cir.ptr<!rec_RetFF>, !rec_RetFF - cir.return %1 : !rec_RetFF - } - - // CHECK: not yet implemented for the ABI coercion of type '!cir.struct<"RetFF" } diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-bitint.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-bitint.cir index 13c81882e9d0b..cfca3c9c9019e 100644 --- a/clang/test/CIR/Transforms/abi-lowering/x86_64-bitint.cir +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-bitint.cir @@ -740,18 +740,18 @@ module attributes { cir.return } - // CHECK-LABEL: cir.func{{.*}} @take_w(%arg0: !s128i) - // CHECK-NEXT: %[[SLOT:[0-9]+]] = cir.alloca "coerce" align(16) : !cir.ptr<!s128i> - // CHECK-NEXT: cir.store %arg0, %[[SLOT]] : !s128i, !cir.ptr<!s128i> + // CHECK-LABEL: cir.func{{.*}} @take_w(%arg0: !s128i_bitint) + // CHECK-NEXT: %[[SLOT:[0-9]+]] = cir.alloca "coerce" align(8) : !cir.ptr<!s128i_bitint> + // CHECK-NEXT: cir.store %arg0, %[[SLOT]] : !s128i_bitint, !cir.ptr<!s128i_bitint> // CHECK-NEXT: %[[VIEW:[0-9]+]] = cir.cast bitcast %[[SLOT]] - // CHECK-SAME: : !cir.ptr<!s128i> -> !cir.ptr<!rec_W> + // CHECK-SAME: : !cir.ptr<!s128i_bitint> -> !cir.ptr<!rec_W> // CHECK-NEXT: %{{[0-9]+}} = cir.load %[[VIEW]] : !cir.ptr<!rec_W>, !rec_W // CHECK-NEXT: cir.return // LLVM-LABEL: define void @take_w( // LLVM-SAME: i128 %[[ARG:[0-9]+]]) - // LLVM-NEXT: %[[SLOT:[0-9]+]] = alloca i128, i64 1, align 16 - // LLVM-NEXT: store i128 %[[ARG]], ptr %[[SLOT]], align 16 + // LLVM-NEXT: %[[SLOT:[0-9]+]] = alloca i128, i64 1, align 8 + // LLVM-NEXT: store i128 %[[ARG]], ptr %[[SLOT]], align 8 // LLVM-NEXT: %{{[0-9]+}} = load %struct.W, ptr %[[SLOT]] // LLVM-NEXT: ret void diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-complex.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-complex.cir new file mode 100644 index 0000000000000..79fb83a07ed29 --- /dev/null +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-complex.cir @@ -0,0 +1,75 @@ +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 | FileCheck %s +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 -cir-to-llvm -o - 2>/dev/null \ +// RUN: | mlir-translate -mlir-to-llvmir --allow-unregistered-dialect \ +// RUN: | FileCheck %s --check-prefix=LLVM + +!s16i = !cir.int<s, 16> +!s32i = !cir.int<s, 32> +!s64i = !cir.int<s, 64> + +// Anonymous record aliases are numbered in the order they are printed, so +// capture each pair rather than naming it. +// CHECK-DAG: ![[X87PAIR:rec_anon_struct[0-9]*]] = !cir.struct<{!cir.f80, !cir.f80}> +// CHECK-DAG: ![[F64PAIR:rec_anon_struct[0-9]*]] = !cir.struct<{!cir.double, !cir.double}> + +module attributes { + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i16, dense<16>: vector<2xi64>>, + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>, + #dlti.dl_entry<f32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<f64, dense<64>: vector<2xi64>>> +} { + + // Both halves of a _Complex float share one SSE eightbyte, so it coerces to + // a two-element vector rather than passing as a CIR complex. + cir.func @cfloat(%arg0: !cir.complex<!cir.float>) -> !cir.complex<!cir.float> { + cir.return %arg0 : !cir.complex<!cir.float> + } + + // CHECK: cir.func{{.*}} @cfloat(%arg0: !cir.vector<2 x !cir.float>) -> !cir.vector<2 x !cir.float> + + // _Complex double needs two SSE eightbytes, so it becomes a register pair + // and the arguments flatten. + cir.func @cdouble(%arg0: !cir.complex<!cir.double>) -> !cir.complex<!cir.double> { + cir.return %arg0 : !cir.complex<!cir.double> + } + + // CHECK: cir.func{{.*}} @cdouble(%arg0: !cir.double, %arg1: !cir.double) -> ![[F64PAIR]] + + // A _Complex of a sub-eightbyte integer packs both halves into one INTEGER + // eightbyte. + cir.func @cshort(%arg0: !cir.complex<!s16i>) -> !cir.complex<!s16i> { + cir.return %arg0 : !cir.complex<!s16i> + } + + // CHECK: cir.func{{.*}} @cshort(%arg0: !u32i) -> !u32i + + cir.func @cint(%arg0: !cir.complex<!s32i>) -> !cir.complex<!s32i> { + cir.return %arg0 : !cir.complex<!s32i> + } + + // CHECK: cir.func{{.*}} @cint(%arg0: !u64i) -> !u64i + + // A complex x87 long double passes its argument in memory while the return + // comes back as the st0/st1 pair. + cir.func @clongdouble(%arg0: !cir.complex<!cir.long_double<!cir.f80>>) + -> !cir.complex<!cir.long_double<!cir.f80>> { + cir.return %arg0 : !cir.complex<!cir.long_double<!cir.f80>> + } + + // CHECK: cir.func{{.*}} @clongdouble(%arg0: !cir.ptr<!cir.complex<!cir.long_double<!cir.f80>>> {{.*}}llvm.byval = !cir.complex<!cir.long_double<!cir.f80>>{{.*}}) -> ![[X87PAIR]] + + cir.func @call_cfloat(%arg0: !cir.complex<!cir.float>) -> !cir.complex<!cir.float> { + %0 = cir.call @cfloat(%arg0) : (!cir.complex<!cir.float>) -> !cir.complex<!cir.float> + cir.return %0 : !cir.complex<!cir.float> + } + + // CHECK: cir.call @cfloat(%{{.+}}) : (!cir.vector<2 x !cir.float>) -> !cir.vector<2 x !cir.float> +} + +// LLVM: define <2 x float> @cfloat(<2 x float> %{{.+}}) +// LLVM: define { double, double } @cdouble(double %{{.+}}, double %{{.+}}) +// LLVM: define i32 @cshort(i32 %{{.+}}) +// LLVM: define i64 @cint(i64 %{{.+}}) +// LLVM: define { x86_fp80, x86_fp80 } @clongdouble(ptr noalias noundef byval({ x86_fp80, x86_fp80 }) align 16 %{{.+}}) diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir index 7a531604073fb..23d0f792db178 100644 --- a/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-call.cir @@ -10,6 +10,7 @@ !rec_Two = !cir.struct<"Two" {!s64i, !s64i}> !rec_Big = !cir.struct<"Big" {!s64i, !s64i, !s64i}> !rec_E0 = !cir.struct<"E0" {}> +!rec_FF = !cir.struct<"FF" {!cir.float, !cir.float}> module attributes { cir.triple = "x86_64-unknown-linux-gnu", @@ -17,6 +18,7 @@ module attributes { #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, #dlti.dl_entry<i64, dense<64>: vector<2xi64>>, + #dlti.dl_entry<f32, dense<32>: vector<2xi64>>, #dlti.dl_entry<f64, dense<64>: vector<2xi64>>> } { @@ -108,6 +110,16 @@ module attributes { // CHECK: cir.func{{.*}} @pass_empty(%arg0: !cir.ptr<!s8i>, %arg1: !s32i) // CHECK: cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !s32i) -> !s32i + // An all-float record at the ellipsis coerces to the vector its eightbyte + // holds, the same as in a declared parameter position. + cir.func @pass_all_float(%arg0: !cir.ptr<!s8i>, %arg1: !rec_FF) { + %0 = cir.call @variadic(%arg0, %arg1) : (!cir.ptr<!s8i>, !rec_FF) -> !s32i + cir.return + } + + // CHECK: cir.func{{.*}} @pass_all_float(%arg0: !cir.ptr<!s8i>, %arg1: !cir.vector<2 x !cir.float>) + // CHECK: cir.call @variadic(%arg0, %{{.*}}) : (!cir.ptr<!s8i>, !cir.vector<2 x !cir.float>) -> !s32i + // A declared parameter is coerced the same way whether or not the call also // passes ellipsis arguments. cir.func @pass_declared_coerced(%arg0: !rec_Pair, %arg1: !s32i) { @@ -193,6 +205,9 @@ module attributes { // LLVM: define void @pass_empty(ptr %{{.+}}, i32 %{{.+}}) // LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, i32 %{{.+}}) +// LLVM: define void @pass_all_float(ptr %{{.+}}, <2 x float> %{{.+}}) +// LLVM: call i32 (ptr, ...) @variadic(ptr %{{.+}}, <2 x float> %{{.+}}) + // LLVM: define void @pass_declared_coerced(i64 %{{.+}}, i32 %{{.+}}) // LLVM: call i32 (i64, ...) @variadic_pair(i64 %{{.+}}, i32 %{{.+}}) diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir index aae2cf94c8125..491605971580c 100644 --- a/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-variadic-nyi.cir @@ -80,35 +80,6 @@ module attributes { // ----- -!s8i = !cir.int<s, 8> -!s32i = !cir.int<s, 32> -!rec_FF = !cir.struct<"FF" {!cir.float, !cir.float}> - -module attributes { - cir.triple = "x86_64-unknown-linux-gnu", - dlti.dl_spec = #dlti.dl_spec< - #dlti.dl_entry<i8, dense<8>: vector<2xi64>>, - #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, - #dlti.dl_entry<f32, dense<32>: vector<2xi64>>, - #dlti.dl_entry<i64, dense<64>: vector<2xi64>>> -} { - - cir.func private @variadic(!cir.ptr<!s8i>, ...) -> !s32i - - // Classifying the call's own operands can reach a coercion the bridge cannot - // represent, here the <2 x float> an all-float eightbyte pair coerces to. - cir.func @ellipsis_unrepresentable(%arg0: !cir.ptr<!s8i>) { - %slot = cir.alloca "p" align(4) : !cir.ptr<!rec_FF> - %v = cir.load %slot : !cir.ptr<!rec_FF>, !rec_FF - %0 = cir.call @variadic(%arg0, %v) : (!cir.ptr<!s8i>, !rec_FF) -> !s32i - cir.return - } - - // CHECK: error: 'cir.call' op x86_64 calling-convention lowering not yet implemented for the ABI coercion of type '!cir.struct<"FF" {!cir.float, !cir.float}>' -} - -// ----- - !s8i = !cir.int<s, 8> !s32i = !cir.int<s, 32> diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-vector.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-vector.cir new file mode 100644 index 0000000000000..4ec0254cdb152 --- /dev/null +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-vector.cir @@ -0,0 +1,62 @@ +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 | FileCheck %s +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 -cir-to-llvm -o - 2>/dev/null \ +// RUN: | mlir-translate -mlir-to-llvmir --allow-unregistered-dialect \ +// RUN: | FileCheck %s --check-prefix=LLVM + +!rec_FF = !cir.struct<"FF" {!cir.float, !cir.float}> +!rec_F3 = !cir.struct<"F3" {!cir.float, !cir.float, !cir.float}> +!rec_UFloats = !cir.union<"UFloats" {!cir.array<!cir.float x 2>, !cir.array<!cir.float x 2>}> + +module attributes { + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>, + #dlti.dl_entry<f32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<f64, dense<64>: vector<2xi64>>> +} { + + // Two floats in one eightbyte classify SSE, and the coercion type the + // classifier picks for that eightbyte is a vector. + cir.func @take_ff(%arg0: !rec_FF) { + cir.return + } + + // CHECK: cir.func{{.*}} @take_ff(%arg0: !cir.vector<2 x !cir.float>) + + cir.func @ret_ff() -> !rec_FF { + %0 = cir.alloca "r" align(4) : !cir.ptr<!rec_FF> + %1 = cir.load %0 : !cir.ptr<!rec_FF>, !rec_FF + cir.return %1 : !rec_FF + } + + // CHECK: cir.func{{.*}} @ret_ff() -> !cir.vector<2 x !cir.float> + + // The same holds for an all-float array. + cir.func @take_farr(%arg0: !cir.array<!cir.float x 2>) { + cir.return + } + + // CHECK: cir.func{{.*}} @take_farr(%arg0: !cir.vector<2 x !cir.float>) + + // Three floats span two eightbytes, so the coercion is a record whose first + // field is the vector covering the first two. + cir.func @take_f3(%arg0: !rec_F3) { + cir.return + } + + // CHECK: cir.func{{.*}} @take_f3(%arg0: !cir.vector<2 x !cir.float>, %arg1: !cir.float) + + // A union whose highest-aligned member is an all-float array reaches the same + // vector coercion through the union path. + cir.func @take_union_floats(%arg0: !rec_UFloats) { + cir.return + } + + // CHECK: cir.func{{.*}} @take_union_floats(%arg0: !cir.vector<2 x !cir.float>) +} + +// LLVM: define void @take_ff(<2 x float> %{{.+}}) +// LLVM: define <2 x float> @ret_ff() +// LLVM: define void @take_farr(<2 x float> %{{.+}}) +// LLVM: define void @take_f3(<2 x float> %{{.+}}, float %{{.+}}) +// LLVM: define void @take_union_floats(<2 x float> %{{.+}}) diff --git a/clang/test/CIR/Transforms/abi-lowering/x86_64-wide-floats.cir b/clang/test/CIR/Transforms/abi-lowering/x86_64-wide-floats.cir new file mode 100644 index 0000000000000..5db452acb917e --- /dev/null +++ b/clang/test/CIR/Transforms/abi-lowering/x86_64-wide-floats.cir @@ -0,0 +1,75 @@ +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 | FileCheck %s +// RUN: cir-opt %s -cir-call-conv-lowering=target=x86_64 -cir-to-llvm -o - 2>/dev/null \ +// RUN: | mlir-translate -mlir-to-llvmir --allow-unregistered-dialect \ +// RUN: | FileCheck %s --check-prefix=LLVM + +!s32i = !cir.int<s, 32> +!rec_SLD = !cir.struct<"SLD" {!cir.long_double<!cir.f80>}> + +module attributes { + dlti.dl_spec = #dlti.dl_spec< + #dlti.dl_entry<i32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<i64, dense<64>: vector<2xi64>>, + #dlti.dl_entry<f32, dense<32>: vector<2xi64>>, + #dlti.dl_entry<f64, dense<64>: vector<2xi64>>> +} { + + // A half occupies one SSE eightbyte and passes in its natural type. + cir.func @half(%arg0: !cir.f16) -> !cir.f16 { + cir.return %arg0 : !cir.f16 + } + + // CHECK: cir.func{{.*}} @half(%arg0: !cir.f16) -> !cir.f16 + + // bfloat16 classifies SSE the same way. + cir.func @bfloat(%arg0: !cir.bf16) -> !cir.bf16 { + cir.return %arg0 : !cir.bf16 + } + + // CHECK: cir.func{{.*}} @bfloat(%arg0: !cir.bf16) -> !cir.bf16 + + // f128 spans an SSE/SSEUP pair, which still passes in one xmm register pair + // rather than memory, so the signature is unchanged. + cir.func @quad(%arg0: !cir.f128) -> !cir.f128 { + cir.return %arg0 : !cir.f128 + } + + // CHECK: cir.func{{.*}} @quad(%arg0: !cir.f128) -> !cir.f128 + + // x87 long double is the X87/X87UP pair: returned in st0, and passed in + // memory only once it sits inside an aggregate. + cir.func @x87(%arg0: !cir.long_double<!cir.f80>) -> !cir.long_double<!cir.f80> { + cir.return %arg0 : !cir.long_double<!cir.f80> + } + + // CHECK: cir.func{{.*}} @x87(%arg0: !cir.long_double<!cir.f80>) -> !cir.long_double<!cir.f80> + + // A bare f80 without the long_double wrapper classifies identically. + cir.func @raw_f80(%arg0: !cir.f80) -> !cir.f80 { + cir.return %arg0 : !cir.f80 + } + + // CHECK: cir.func{{.*}} @raw_f80(%arg0: !cir.f80) -> !cir.f80 + + // A struct holding an x87 long double merges to MEMORY, so the argument is + // byval. + cir.func @take_sld(%arg0: !rec_SLD) { + cir.return + } + + // CHECK: cir.func{{.*}} @take_sld(%arg0: !cir.ptr<!rec_SLD> {{.*}}llvm.byval = !rec_SLD{{.*}}) + + cir.func @call_half(%arg0: !cir.f16) -> !cir.f16 { + %0 = cir.call @half(%arg0) : (!cir.f16) -> !cir.f16 + cir.return %0 : !cir.f16 + } + + // CHECK: cir.call @half(%arg0) : (!cir.f16) -> !cir.f16 +} + +// LLVM: define half @half(half %{{.+}}) +// LLVM: define bfloat @bfloat(bfloat %{{.+}}) +// LLVM: define fp128 @quad(fp128 %{{.+}}) +// LLVM: define x86_fp80 @x87(x86_fp80 %{{.+}}) +// LLVM: define x86_fp80 @raw_f80(x86_fp80 %{{.+}}) +// LLVM: define void @take_sld(ptr noalias noundef byval(%struct.SLD) align 16 %{{.+}}) _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
