[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
github-actions[bot] wrote: @MarwanTarik Congratulations on having your first Pull Request (PR) merged into the LLVM Project! Your changes will be combined with recent changes from other authors, then tested by our [build bots](https://lab.llvm.org/buildbot/). If there is a problem with a build, you may receive a report in an email or a comment on this PR. Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues. How to do this, and the rest of the post-merge process, is covered in detail [here](https://llvm.org/docs/MyFirstTypoFix.html#myfirsttypofix-issues-after-landing-your-pr). If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of [LLVM development](https://llvm.org/docs/DeveloperPolicy.html#patch-reversion-policy). You can fix your changes and open a new PR to merge them again. If you don't get any reports, no action is required from you. Your changes are working as expected, well done! https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/andykaylor closed https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +756,51 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i -> !cir.vector<32 x
!cir.int>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector<32 x !cir.int> -> !cir.vector<32 x !s16i>
MarwanTarik wrote:
I'm working on it now
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +756,51 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i -> !cir.vector<32 x
!cir.int>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector<32 x !cir.int> -> !cir.vector<32 x !s16i>
andykaylor wrote:
```suggestion
// CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i -> !cir.vector<32 x
!cir.int>
// CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector<32 x !cir.int> -> !cir.vector<32 x !s16i>
```
And similarly elsewhere in the tests.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +756,51 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i -> !cir.vector<32 x
!cir.int>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector<32 x !cir.int> -> !cir.vector<32 x !s16i>
+
+ // LLVM-LABEL: @test_mm512_movm_epi16
+ // LLVM: %{{.*}} = bitcast i32 %{{.*}} to <32 x i1>
+ // LLVM: %{{.*}} = sext <32 x i1> %{{.*}} to <32 x i16>
andykaylor wrote:
Oh, I was wrong. It looks like you'll need to rebase and update the checks in
this PR that are looking for `>`
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +756,51 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i -> !cir.vector<32 x
!cir.int>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector<32 x !cir.int> -> !cir.vector<32 x !s16i>
+
+ // LLVM-LABEL: @test_mm512_movm_epi16
+ // LLVM: %{{.*}} = bitcast i32 %{{.*}} to <32 x i1>
+ // LLVM: %{{.*}} = sext <32 x i1> %{{.*}} to <32 x i16>
andykaylor wrote:
I think this test, among others, will fail because of the sign-extend issue.
I'll wait until I've merged that before starting the test run for this PR.
Hopefully you won't have to make any changes.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -1,32 +1,21 @@ // RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s // RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s +// FIXME: CIR to LLVM lowering fails with "integer width of the output type is smaller or equal to the integer width of the input type" error +// RUN-DISABLED: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion MarwanTarik wrote: ok https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -1,32 +1,21 @@ // RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s // RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s +// FIXME: CIR to LLVM lowering fails with "integer width of the output type is smaller or equal to the integer width of the input type" error +// RUN-DISABLED: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion andykaylor wrote: Can you re-enable these runs now? https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -1,32 +1,21 @@ // RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s // RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s +// FIXME: CIR to LLVM lowering fails with "integer width of the output type is smaller or equal to the integer width of the input type" error MarwanTarik wrote: I’ve added it. https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 01/17] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::B
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -279,6 +285,102 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+return {};
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+return {};
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
+return {};
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+mlir::Value in, mlir::Location loc) {
+ cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
+}
+
+
+static std::optional emitX86SExtMask(CIRGenBuilderTy &builder,
+ mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(builder, loc, op, numberOfElements);
andykaylor wrote:
https://github.com/llvm/llvm-project/pull/172912
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -279,6 +285,102 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+return {};
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+return {};
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
+return {};
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+mlir::Value in, mlir::Location loc) {
+ cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
+}
+
+
+static std::optional emitX86SExtMask(CIRGenBuilderTy &builder,
+ mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(builder, loc, op, numberOfElements);
MarwanTarik wrote:
I will modify `getMaskVecValue` to return a signed vector and update the
related tests accordingly.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -279,6 +285,102 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+return {};
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+return {};
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
+return {};
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+mlir::Value in, mlir::Location loc) {
+ cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
+}
+
+
+static std::optional emitX86SExtMask(CIRGenBuilderTy &builder,
+ mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(builder, loc, op, numberOfElements);
andykaylor wrote:
We have a problem here. Currently, `getMaskVecValue` creates a vector of i1
like this: `!cir.vector<32 x !cir.int>`. Because we made the i1 type
unsigned there, it's going to get zero extended by the cast below. In general,
I find the idea of pretending that an i1 value has a sign-bit to be very
dubious, but in this particular case we explicitly need the sign-extending
behavior. So, we either need to change `getMaskVecValue` to return a signed i1
vector, or we need to cast it as such here. The former is probably best.
Ideally, it would be a vector of signless i1, but we don't do signless types in
CIR.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -1,32 +1,21 @@ // RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s // RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion // RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c++ -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s -// RUN: %clang_cc1 -x c -flax-vector-conversions=none -ffreestanding %s -triple=x86_64-unknown-linux -target-feature +avx512bw -fno-signed-char -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion -// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s +// FIXME: CIR to LLVM lowering fails with "integer width of the output type is smaller or equal to the integer width of the input type" error andykaylor wrote: I found the problem. It was a bad assumption in our lowering of VecCmpOp. This change should fix it. ``` diff --git a/clang/lib/CIR/Lowering/DirectToLLVM/LowerToLLVM.cpp b/clang/lib/CIR/Lowering/DirectToLLVM/LowerToLLVM.cpp index 7c9cf8e2c2e2..4dcde56f2917 100644 --- a/clang/lib/CIR/Lowering/DirectToLLVM/LowerToLLVM.cpp +++ b/clang/lib/CIR/Lowering/DirectToLLVM/LowerToLLVM.cpp @@ -3605,9 +3605,13 @@ mlir::LogicalResult CIRToLLVMVecCmpOpLowering::matchAndRewrite( } // LLVM IR vector comparison returns a vector of i1. This one-bit vector - // must be sign-extended to the correct result type. - rewriter.replaceOpWithNewOp( - op, typeConverter->convertType(op.getType()), bitResult); + // must be sign-extended to the correct result type, unless a vector of i1 is + // the type we need. + if (cast(cast(op.getType()).getElementType()) + .getWidth() > 1) +rewriter.replaceOpWithNewOp( +op, typeConverter->convertType(op.getType()), bitResult); + else +rewriter.replaceOp(op, bitResult); return mlir::success(); } ``` Can you add that to your PR? https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
MarwanTarik wrote:
I figured out that there were some bugs in the tests. I could summarize it in
the following:
- Removed @ prefix from CIR-LABEL patterns to match C++ mangled names
- Updated OGCG-LABEL patterns from @test_function to {{.*}}test_function{{.*}}(
for C/C++ compatibility
- Fixed vector syntax from old !cir.vector x 32> to new
!cir.vector<32 x !cir.int> format
Also, I figured out that it seems there's a backend bug: the CIR-to-LLVM
lowering incorrectly rejects valid 1-bit to N-bit integer extensions in
vectors. So I disabled CIR-to-LLVM RUN lines temporarily as a workaround. Pls
tell me your thoughts about that
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 01/16] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::B
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/andykaylor auto_merge_disabled https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 01/15] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::B
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/andykaylor auto_merge_enabled https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
github-actions[bot] wrote:
:warning: C/C++ code formatter, clang-format found issues in your code.
:warning:
You can test this locally with the following command:
``bash
git-clang-format --diff origin/main HEAD --extensions c,cpp --
clang/test/CIR/CodeGenBuiltins/X86/avx512vlbw-builtins.c
clang/test/CIR/CodeGenBuiltins/X86/avx512vldq-builtins.c
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
clang/test/CIR/CodeGenBuiltins/X86/avx512bw-builtins.c
clang/test/CIR/CodeGenBuiltins/X86/avx512dq-builtins.c --diff_from_common_commit
``
:warning:
The reproduction instructions above might return results for more than one PR
in a stack if you are using a stacked PR workflow. You can limit the results by
changing `origin/main` to the base branch/commit you want to compare against.
:warning:
View the diff from clang-format here.
``diff
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index 168944b52..422beefd6 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -25,8 +25,8 @@
#include "clang/CIR/Dialect/IR/CIRTypes.h"
#include "clang/CIR/MissingFeatures.h"
#include "llvm/ADT/Sequence.h"
-#include
#include "llvm/Support/ErrorHandling.h"
+#include
using namespace clang;
using namespace clang::CIRGen;
@@ -285,11 +285,12 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
mlir::ValueRange{lhsVec, rhsVec});
}
-// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
-static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
- mlir::Value cmp, unsigned
numElts,
- mlir::Value maskIn,
- mlir::Location loc) {
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional
+emitX86MaskedCompareResult(CIRGenFunction &cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned numElts,
+ mlir::Value maskIn, mlir::Location loc) {
if (maskIn) {
cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
return {};
@@ -307,15 +308,15 @@ static std::optional
emitX86MaskedCompareResult(CIRGenFunction &cgf
mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
}
- return builder.createBitcast(
- cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+ return builder.createBitcast(cmp, builder.getUIntNTy(std::max(numElts, 8U)));
}
-// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
-static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
-unsigned cc, bool isSigned,
-ArrayRef ops,
-mlir::Location loc) {
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional
+emitX86MaskedCompare(CIRGenFunction &cgf, CIRGenBuilderTy &builder, unsigned
cc,
+ bool isSigned, ArrayRef ops,
+ mlir::Location loc) {
assert((ops.size() == 2 || ops.size() == 4) &&
"Unexpected number of arguments");
unsigned numElts = cast(ops[0].getType()).getSize();
@@ -353,8 +354,7 @@ static std::optional
emitX86MaskedCompare(CIRGenFunction &cgf, CIRG
}
auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
-cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
-ops[1]);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0], ops[1]);
}
mlir::Value maskIn;
@@ -364,17 +364,20 @@ static std::optional
emitX86MaskedCompare(CIRGenFunction &cgf, CIRG
return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
}
-// TODO: The cgf parameter should be removed when all the NYI cases are
implemented.
-static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
-mlir::Value in, mlir::Location loc) {
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
+ CIRGenBuilderTy
&builder,
+ mlir::Value in,
+ mlir::Location loc) {
cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
}
-
static std::optional emitX86SExtMask(CIRGenBuilderTy &builder,
-
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/andykaylor approved this pull request. lgtm https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
MarwanTarik wrote: I added the missing `bitcast` checks and removed the `-fno-signed-char` checks from `avx512bw-builtins.c` and `avx512vlbw-builtins.c` https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 01/15] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::B
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +768,42 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i ->
!cir.vector x 32>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
32> -> !cir.vector
+ // LLVM-LABEL: @test_mm512_movm_epi16
+ // LLVM: %{{.*}} = bitcast i32 %{{.*}} to <32 x i1>
+ // LLVM: %{{.*}} = sext <32 x i1> %{{.*}} to <32 x i16>
+ return _mm512_movm_epi16(__A);
+}
+
+__mmask64 test_mm512_movepi8_mask(__m512i __A) {
+ // CIR-LABEL: @_mm512_movepi8_mask
+ // CIR: %{{.*}} = cir.vec.cmp(lt, %{{.*}}, %{{.*}}) :
!cir.vector<{{!s8i|!u8i}} x 64>, !cir.vector x 64>
+
+ // LLVM-LABEL: @test_mm512_movepi8_mask
+ // LLVM: [[CMP:%.*]] = icmp slt <64 x i8> %{{.*}}, zeroinitializer
+
+ // In the unsigned case below, the canonicalizer proves the comparison is
+ // always false (no i8 unsigned value can be < 0) and folds it away.
+ // LLVM-UNSIGNED-CHAR: store i64 0, ptr %{{.*}}, align 8
MarwanTarik wrote:
ok, I will remove it
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik edited https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 1/9] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::BI_
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +768,42 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i ->
!cir.vector x 32>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
32> -> !cir.vector
+ // LLVM-LABEL: @test_mm512_movm_epi16
+ // LLVM: %{{.*}} = bitcast i32 %{{.*}} to <32 x i1>
+ // LLVM: %{{.*}} = sext <32 x i1> %{{.*}} to <32 x i16>
+ return _mm512_movm_epi16(__A);
+}
+
+__mmask64 test_mm512_movepi8_mask(__m512i __A) {
+ // CIR-LABEL: @_mm512_movepi8_mask
+ // CIR: %{{.*}} = cir.vec.cmp(lt, %{{.*}}, %{{.*}}) :
!cir.vector<{{!s8i|!u8i}} x 64>, !cir.vector x 64>
+
+ // LLVM-LABEL: @test_mm512_movepi8_mask
+ // LLVM: [[CMP:%.*]] = icmp slt <64 x i8> %{{.*}}, zeroinitializer
+
+ // In the unsigned case below, the canonicalizer proves the comparison is
+ // always false (no i8 unsigned value can be < 0) and folds it away.
+ // LLVM-UNSIGNED-CHAR: store i64 0, ptr %{{.*}}, align 8
MarwanTarik wrote:
I found it in the original incubator tests, so I kept it
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 1/6] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::BI_
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -0,0 +1,110 @@
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx512dq -target-feature
+avx512vl -fclangir -emit-cir -o %t.cir -Wall -Werror
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx512dq -target-feature
+avx512vl -fclangir -emit-llvm -o %t.ll -Wall -Werror
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-apple-darwin -target-feature +avx512dq -target-feature +avx512vl
-emit-llvm -o - -Wall -Werror | FileCheck %s --check-prefixes=OGCG
+
+#include
+
+
+__m128i test_mm_movm_epi32(__mmask8 __A) {
+ // CIR-LABEL: _mm_movm_epi32
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u8i -> !cir.vector x 8>
+ // CIR: %{{.*}} = cir.vec.shuffle(%{{.*}}, %{{.*}} : !cir.vector x 8>) [#cir.int<0> : !s32i, #cir.int<1> : !s32i, #cir.int<2> : !s32i,
#cir.int<3> : !s32i] : !cir.vector x 4>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
4> -> !cir.vector
+
+ // LLVM-LABEL: @test_mm_movm_epi32
+ // LLVM: %{{.*}} = bitcast i8 %{{.*}} to <8 x i1>
+ // LLVM: %{{.*}} = shufflevector <8 x i1> %{{.*}}, <8 x i1> %{{.*}}, <4 x
i32>
+ // LLVM: %{{.*}} = sext <4 x i1> %{{.*}} to <4 x i32>
andykaylor wrote:
Add OGCG checks throughout this test
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +768,42 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i ->
!cir.vector x 32>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
32> -> !cir.vector
+ // LLVM-LABEL: @test_mm512_movm_epi16
+ // LLVM: %{{.*}} = bitcast i32 %{{.*}} to <32 x i1>
+ // LLVM: %{{.*}} = sext <32 x i1> %{{.*}} to <32 x i16>
andykaylor wrote:
Can you add OGCG checks here?
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -13,6 +13,17 @@
#include
+__m512i test_mm512_movm_epi64(__mmask8 __A) {
+ // CIR-LABEL: _mm512_movm_epi64
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u8i -> !cir.vector x 8>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
8> -> !cir.vector
+ // LLVM-LABEL: @test_mm512_movm_epi64
+ // LLVM: %{{.*}} = bitcast i8 %{{.*}} to <8 x i1>
+ // LLVM: %{{.*}} = sext <8 x i1> %{{.*}} to <8 x i64>
andykaylor wrote:
Add OGCG checks
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -674,6 +804,10 @@ CIRGenFunction::emitX86BuiltinExpr(unsigned builtinID,
const CallExpr *expr) {
case X86::BI__builtin_ia32_storesh128_mask:
case X86::BI__builtin_ia32_storess128_mask:
case X86::BI__builtin_ia32_storesd128_mask:
+ cgm.errorNYI(expr->getSourceRange(),
+ std::string("unimplemented x86 builtin call: ") +
+ getContext().BuiltinInfo.getName(builtinID));
+ return mlir::Value{};
andykaylor wrote:
These lines need to be indented.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -278,6 +284,130 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+return {};
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+return {};
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
+return {};
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+mlir::Value in, mlir::Location loc) {
+ cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenBuilderTy &builder, CIRGenFunction
&cgf,
andykaylor wrote:
This has already been upstreamed.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -278,6 +284,130 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+return {};
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+return {};
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
+return {};
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+mlir::Value in, mlir::Location loc) {
+ cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenBuilderTy &builder, CIRGenFunction
&cgf,
+ mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(builder.getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = builder.createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (auto i : llvm::seq(0, numElts))
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+
+maskVec = builder.createVecShuffle(loc, maskVec, maskVec, indices);
+ }
+
+ return maskVec;
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86SExtMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
andykaylor wrote:
```suggestion
static std::optional emitX86SExtMask(CIRGenBuilderTy &builder,
```
The extra parameter is not needed here.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -951,6 +951,17 @@ __m256i test_mm256_movm_epi32(__mmask8 __A) {
return _mm256_movm_epi32(__A);
}
+__m512i test_mm512_movm_epi32(__mmask16 __A) {
andykaylor wrote:
No problem, but please remove the changes from the `CodeGen/X86` versions of
these files.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -278,6 +284,130 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+return {};
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+return {};
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
+return {};
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cir::VectorType::get(builder.getSIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(builder, loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, builder, cmp, numElts, maskIn, loc);
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86ConvertToMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+mlir::Value in, mlir::Location loc) {
+ cir::ConstantOp zero = builder.getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, builder, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenBuilderTy &builder, CIRGenFunction
&cgf,
+ mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(builder.getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = builder.createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (auto i : llvm::seq(0, numElts))
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+
+maskVec = builder.createVecShuffle(loc, maskVec, maskVec, indices);
+ }
+
+ return maskVec;
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86SExtMask(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+ mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(builder, cgf, op, numberOfElements, loc);
andykaylor wrote:
```suggestion
mlir::Value mask = getMaskVecValue(builder, op, numberOfElements, loc);
```
This function has already been upstreamed.
https://github.com/llvm/llvm-project/pull/171694
___
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -768,3 +768,42 @@ unsigned char test_ktestz_mask64_u8(__mmask64 A, __mmask64
B) {
return _ktestz_mask64_u8(A, B);
}
+
+
+__m512i test_mm512_movm_epi16(__mmask32 __A) {
+ // CIR-LABEL: _mm512_movm_epi16
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u32i ->
!cir.vector x 32>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
32> -> !cir.vector
+ // LLVM-LABEL: @test_mm512_movm_epi16
+ // LLVM: %{{.*}} = bitcast i32 %{{.*}} to <32 x i1>
+ // LLVM: %{{.*}} = sext <32 x i1> %{{.*}} to <32 x i16>
+ return _mm512_movm_epi16(__A);
+}
+
+__mmask64 test_mm512_movepi8_mask(__m512i __A) {
+ // CIR-LABEL: @_mm512_movepi8_mask
+ // CIR: %{{.*}} = cir.vec.cmp(lt, %{{.*}}, %{{.*}}) :
!cir.vector<{{!s8i|!u8i}} x 64>, !cir.vector x 64>
+
+ // LLVM-LABEL: @test_mm512_movepi8_mask
+ // LLVM: [[CMP:%.*]] = icmp slt <64 x i8> %{{.*}}, zeroinitializer
+
+ // In the unsigned case below, the canonicalizer proves the comparison is
+ // always false (no i8 unsigned value can be < 0) and folds it away.
+ // LLVM-UNSIGNED-CHAR: store i64 0, ptr %{{.*}}, align 8
andykaylor wrote:
What is this?
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -0,0 +1,104 @@
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx512bw -target-feature
+avx512vl -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx512bw -target-feature
+avx512vl -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx512bw -target-feature
+avx512vl -fno-signed-char -fclangir -emit-cir -o %t.cir -Wall -Werror
-Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx512bw -target-feature
+avx512vl -fno-signed-char -fclangir -emit-llvm -o %t.ll -Wall -Werror
-Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM-UNSIGNED-CHAR --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx10.1-512 -target-feature
+avx512vl -fclangir -emit-cir -o %t.cir -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefix=CIR --input-file=%t.cir %s
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-unknown-linux -target-feature +avx10.1-512 -target-feature
+avx512vl -fclangir -emit-llvm -o %t.ll -Wall -Werror -Wsign-conversion
+// RUN: FileCheck --check-prefixes=LLVM --input-file=%t.ll %s
+
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-apple-darwin -target-feature +avx512bw -target-feature +avx512vl
-emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s
--check-prefixes=OGCG
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-apple-darwin -target-feature +avx512bw -target-feature +avx512vl
-fno-signed-char -emit-llvm -o - -Wall -Werror -Wsign-conversion | FileCheck %s
--check-prefixes=OGCG
+// RUN: %clang_cc1 -flax-vector-conversions=none -ffreestanding %s
-triple=x86_64-apple-darwin -target-feature +avx10.1-512 -emit-llvm -o - -Wall
-Werror -Wsign-conversion | FileCheck %s --check-prefixes=OGCG
+
+
+#include
+
+__m128i test_mm_movm_epi8(__mmask16 __A) {
+ // CIR-LABEL: _mm_movm_epi8
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u16i ->
!cir.vector x 16>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
16> -> !cir.vector<{{!s8i|!u8i}} x 16>
+
+ // LLVM-LABEL: @test_mm_movm_epi8
+ // LLVM: %{{.*}} = bitcast i16 %{{.*}} to <16 x i1>
+ // LLVM: %{{.*}} = sext <16 x i1> %{{.*}} to <16 x i8>
andykaylor wrote:
Add OGCG checks throughout this test
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -951,6 +951,17 @@ __m256i test_mm256_movm_epi32(__mmask8 __A) {
return _mm256_movm_epi32(__A);
}
+__m512i test_mm512_movm_epi32(__mmask16 __A) {
MarwanTarik wrote:
Ahh, I’m really sorry, it’s my fault. For some reason, I got the files confused
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 1/6] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::BI_
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -278,6 +284,127 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
+ } else if (cc == 7) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
andykaylor wrote:
```suggestion
cgf.cgm.errorNYI(loc, "emitX86MaskedCompare cc == 7");
return {};
```
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -3226,6 +3226,18 @@ __m256i test_mm256_movm_epi8(__mmask32 __A) {
return _mm256_movm_epi8(__A);
}
+__m512i test_mm512_movm_epi8(__mmask64 __A) {
+ // CIR-LABEL: _mm512_movm_epi8
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u64i ->
!cir.vector x 64>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
64> -> !cir.vector<{{!s8i|!u8i}} x 64>
+
+ // LLVM-LABEL: @test_mm512_movm_epi8
+ // LLVM: %{{.*}} = bitcast i64 %{{.*}} to <64 x i1>
+ // LLVM: %{{.*}} = sext <64 x i1> %{{.*}} to <64 x i8>
+ return _mm512_movm_epi8(__A);
andykaylor wrote:
You're modifying the wrong version of this test. You should be making changes
in `/clang/test/CIR/CodeGenBuiltins/avx512vlbw-builtins.c` (which doesn't exist
yet upstream). Note the `CIR` part of that path.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -278,6 +284,127 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
+ }
+ if (numElts < 8) {
+llvm::SmallVector indices;
+mlir::Type i64Ty = builder.getSInt64Ty();
+
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i));
+for (unsigned i = numElts; i != 8; ++i)
+ indices.push_back(cir::IntAttr::get(i64Ty, i % numElts + numElts));
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = builder.getNullValue(cmp.getType(), loc);
+cmp = builder.createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return builder.createBitcast(
+ cmp, builder.getUIntNTy(std::max(numElts, 8U)));
+}
+
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompare(CIRGenFunction &cgf,
CIRGenBuilderTy &builder,
+unsigned cc, bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
andykaylor wrote:
```suggestion
cgf.cgm.errorNYI(loc, "emitX86MaskedCompare: cc == 3");
return {};
```
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -278,6 +284,127 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+// TODO: The cgf parameter should be removed when all the NYI cases are
+// implemented.
+static std::optional emitX86MaskedCompareResult(CIRGenFunction
&cgf, CIRGenBuilderTy &builder,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
andykaylor wrote:
```suggestion
cgf.cgm.errorNYI(loc, "emitX86MaskedCompareResult");
return {};
```
Returning a null `mlir::Value` rather than `std::nullopt` indicates that we
already reported an error.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -951,6 +951,17 @@ __m256i test_mm256_movm_epi32(__mmask8 __A) {
return _mm256_movm_epi32(__A);
}
+__m512i test_mm512_movm_epi32(__mmask16 __A) {
andykaylor wrote:
Again, this is modifying the wrong version of the test.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
MarwanTarik wrote: I've resolved the issues. Please review the updated changes. https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH 1/4] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::BI_
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
@@ -3226,6 +3226,18 @@ __m256i test_mm256_movm_epi8(__mmask32 __A) {
return _mm256_movm_epi8(__A);
}
+__m512i test_mm512_movm_epi8(__mmask64 __A) {
+ // CIR-LABEL: _mm512_movm_epi8
+ // CIR: %{{.*}} = cir.cast bitcast %{{.*}} : !u64i ->
!cir.vector x 64>
+ // CIR: %{{.*}} = cir.cast integral %{{.*}} : !cir.vector x
64> -> !cir.vector<{{!s8i|!u8i}} x 64>
+
+ // LLVM-LABEL: @test_mm512_movm_epi8
+ // LLVM: %{{.*}} = bitcast i64 %{{.*}} to <64 x i1>
+ // LLVM: %{{.*}} = sext <64 x i1> %{{.*}} to <64 x i8>
+ return _mm512_movm_epi8(__A);
MarwanTarik wrote:
I had checked the incubator tests and the upstreamed ones, and I found that
most of the tests are already upstreamed.
https://github.com/llvm/llvm-project/pull/171694
___
cfe-commits mailing list
[email protected]
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik updated
https://github.com/llvm/llvm-project/pull/171694
>From 82529b8bfd35c9e8059b49e2f17b3c837232cf09 Mon Sep 17 00:00:00 2001
From: MarwanTarik
Date: Wed, 10 Dec 2025 22:21:55 +0200
Subject: [PATCH] Upstream CIR Codgen for convert to mask X86 builtins
---
clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp | 114 +++
clang/test/CodeGen/X86/avx512vlbw-builtins.c | 12 ++
2 files changed, 126 insertions(+)
diff --git a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
index fb17e31bf36d6..bba7249666aaf 100644
--- a/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
+++ b/clang/lib/CIR/CodeGen/CIRGenBuiltinX86.cpp
@@ -231,6 +231,113 @@ static mlir::Value emitX86MaskTest(CIRGenBuilderTy
&builder, mlir::Location loc,
return emitIntrinsicCallOp(builder, loc, intrinsicName, resTy,
mlir::ValueRange{lhsVec, rhsVec});
}
+static mlir::Value emitX86MaskedCompareResult(CIRGenFunction &cgf,
+ mlir::Value cmp, unsigned
numElts,
+ mlir::Value maskIn,
+ mlir::Location loc) {
+ if (maskIn) {
+llvm_unreachable("NYI");
+ }
+ if (numElts < 8) {
+int64_t indices[8];
+for (unsigned i = 0; i != numElts; ++i)
+ indices[i] = i;
+for (unsigned i = numElts; i != 8; ++i)
+ indices[i] = i % numElts + numElts;
+
+// This should shuffle between cmp (first vector) and null (second vector)
+mlir::Value nullVec = cgf.getBuilder().getNullValue(cmp.getType(), loc);
+cmp = cgf.getBuilder().createVecShuffle(loc, cmp, nullVec, indices);
+ }
+ return cgf.getBuilder().createBitcast(
+ cmp, cgf.getBuilder().getUIntNTy(std::max(numElts, 8U)));
+}
+
+static mlir::Value emitX86MaskedCompare(CIRGenFunction &cgf, unsigned cc,
+bool isSigned,
+ArrayRef ops,
+mlir::Location loc) {
+ assert((ops.size() == 2 || ops.size() == 4) &&
+ "Unexpected number of arguments");
+ unsigned numElts = cast(ops[0].getType()).getSize();
+ mlir::Value cmp;
+
+ if (cc == 3) {
+llvm_unreachable("NYI");
+ } else if (cc == 7) {
+llvm_unreachable("NYI");
+ } else {
+cir::CmpOpKind pred;
+switch (cc) {
+default:
+ llvm_unreachable("Unknown condition code");
+case 0:
+ pred = cir::CmpOpKind::eq;
+ break;
+case 1:
+ pred = cir::CmpOpKind::lt;
+ break;
+case 2:
+ pred = cir::CmpOpKind::le;
+ break;
+case 4:
+ pred = cir::CmpOpKind::ne;
+ break;
+case 5:
+ pred = cir::CmpOpKind::ge;
+ break;
+case 6:
+ pred = cir::CmpOpKind::gt;
+ break;
+}
+
+auto resultTy = cgf.getBuilder().getType(
+cgf.getBuilder().getUIntNTy(1), numElts);
+cmp = cir::VecCmpOp::create(cgf.getBuilder(), loc, resultTy, pred, ops[0],
+ops[1]);
+ }
+
+ mlir::Value maskIn;
+ if (ops.size() == 4)
+maskIn = ops[3];
+
+ return emitX86MaskedCompareResult(cgf, cmp, numElts, maskIn, loc);
+}
+
+static mlir::Value emitX86ConvertToMask(CIRGenFunction &cgf, mlir::Value in,
+mlir::Location loc) {
+ cir::ConstantOp zero = cgf.getBuilder().getNullValue(in.getType(), loc);
+ return emitX86MaskedCompare(cgf, 1, true, {in, zero}, loc);
+}
+
+// Convert the mask from an integer type to a vector of i1.
+static mlir::Value getMaskVecValue(CIRGenFunction &cgf, mlir::Value mask,
+ unsigned numElts, mlir::Location loc) {
+ cir::VectorType maskTy =
+ cir::VectorType::get(cgf.getBuilder().getSIntNTy(1),
+ cast(mask.getType()).getWidth());
+
+ mlir::Value maskVec = cgf.getBuilder().createBitcast(mask, maskTy);
+
+ // If we have less than 8 elements, then the starting mask was an i8 and
+ // we need to extract down to the right number of elements.
+ if (numElts < 8) {
+llvm::SmallVector indices;
+for (unsigned i = 0; i != numElts; ++i)
+ indices.push_back(i);
+maskVec = cgf.getBuilder().createVecShuffle(loc, maskVec, maskVec,
indices);
+ }
+
+ return maskVec;
+}
+
+static mlir::Value emitX86SExtMask(CIRGenFunction &cgf, mlir::Value op,
+ mlir::Type dstTy, mlir::Location loc) {
+ unsigned numberOfElements = cast(dstTy).getSize();
+ mlir::Value mask = getMaskVecValue(cgf, op, numberOfElements, loc);
+
+ return cgf.getBuilder().createCast(loc, cir::CastKind::integral, mask,
dstTy);
+}
static mlir::Value emitVecInsert(CIRGenBuilderTy &builder, mlir::Location loc,
mlir::Value vec, mlir::Value value,
@@ -558,6 +665,7 @@ mlir::Value CIRGenFunction::emitX86BuiltinExpr(unsigned
builtinID,
case X86::BI__builtin_ia32_storesh128_mask:
case X86::BI__bui
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
github-actions[bot] wrote: Thank you for submitting a Pull Request (PR) to the LLVM Project! This PR will be automatically labeled and the relevant teams will be notified. If you wish to, you can add reviewers by using the "Reviewers" section on this page. If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using `@` followed by their GitHub username. If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers. If you have further questions, they may be answered by the [LLVM GitHub User Guide](https://llvm.org/docs/GitHub.html). You can also ask questions in a comment on this PR, on the [LLVM Discord](https://discord.com/invite/xS7Z362) or on the [forums](https://discourse.llvm.org/). https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CIR] Upstream convert to mask builtins in CIR codegen (PR #171694)
https://github.com/MarwanTarik edited https://github.com/llvm/llvm-project/pull/171694 ___ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
