[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis converted_to_draft https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2477,6 +2493,15 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo
*LI,
// Save 'store' instructions. Abort if other instructions write to
memory.
if (I.mayWriteToMemory()) {
+// We can safety handle math functions that have vectorized
+// counterparts and have the memory write-only attribute set.
mgabka wrote:
Hi @fhahn,
you are right.
The whole logic was based on checking later in the
https://github.com/llvm/llvm-project/blob/332de4b2677ce7a95cc2df30d761fbb55376fe07/llvm/lib/Transforms/Vectorize/LoopVectorize.cpp#L6659
if the pointers are linear with the right step, but there is no checks for
aliasing.
@paschalis-mpeis that means that there is an existing bug with sincos, as
following code gets vectorised, without any checks:
```
double sin[N];
double cos[N];
sin[5] = M_PI;
for(int i=0; ihttps://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,116 @@ +; RUN: opt < %s -passes='print' -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s fhahn wrote: Please also check the `access-info` report. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,116 @@
+; RUN: opt < %s -passes='print' -debug-only=loop-accesses
-disable-output 2>&1 | FileCheck %s
+
+; REQUIRES: asserts
+
+
+define void @frexp_f64(ptr %in, ptr %out1, ptr %out2, i32 %N) {
+; CHECK: LAA: Allow to vectorize math function with write-only attribute:
%call = tail call double @frexp
+entry:
+ %cmp4 = icmp sgt i32 %N, 0
+ br i1 %cmp4, label %for.body.preheader, label %for.cond.cleanup
+
+for.body.preheader:
+ %wide.trip.count = zext nneg i32 %N to i64
+ br label %for.body
+
+for.cond.cleanup:
+ ret void
+
+for.body:
+ %indvars.iv = phi i64 [ 0, %for.body.preheader ], [ %indvars.iv.next,
%for.body ]
fhahn wrote:
please strip `%indvars.` prefix to keep value names more concise
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,116 @@
+; RUN: opt < %s -passes='print' -debug-only=loop-accesses
-disable-output 2>&1 | FileCheck %s
+
+; REQUIRES: asserts
+
+
+define void @frexp_f64(ptr %in, ptr %out1, ptr %out2, i32 %N) {
+; CHECK: LAA: Allow to vectorize math function with write-only attribute:
%call = tail call double @frexp
+entry:
+ %cmp4 = icmp sgt i32 %N, 0
fhahn wrote:
this shouldn't be needed, please try to simplify the test cases
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2477,6 +2493,15 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo
*LI,
// Save 'store' instructions. Abort if other instructions write to
memory.
if (I.mayWriteToMemory()) {
+// We can safety handle math functions that have vectorized
+// counterparts and have the memory write-only attribute set.
fhahn wrote:
I am not sure the reasoning here is correct; the writes could still alias with
other accesses,
e.g. something like below would be considered as safe, but `%out2` could
overlap with `%in` in a way that results in a backwards dependence at runtime.
```
define void @frexp_f64(ptr %in, ptr %out1, ptr %out2, i32 %N) {
entry:
%cmp4 = icmp sgt i32 %N, 0
br i1 %cmp4, label %for.body.preheader, label %for.cond.cleanup
for.body.preheader:
%wide.trip.count = zext nneg i32 %N to i64
br label %for.body
for.cond.cleanup:
ret void
for.body:
%indvars.iv = phi i64 [ 0, %for.body.preheader ], [ %indvars.iv.next,
%for.body ]
%arrayidx = getelementptr inbounds double, ptr %in, i64 %indvars.iv
%0 = load double, ptr %arrayidx, align 8
%add.ptr = getelementptr inbounds i32, ptr %out2, i64 %indvars.iv
%call = tail call double @frexp(double noundef %0, ptr noundef %add.ptr)
store double %call, ptr %arrayidx, align 8
%indvars.iv.next = add nuw nsw i64 %indvars.iv, 1
%exitcond.not = icmp eq i64 %indvars.iv.next, %wide.trip.count
br i1 %exitcond.not, label %for.cond.cleanup, label %for.body
}
declare double @frexp(double, ptr) #1
attributes #1 = { memory(argmem: write) }
```
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,116 @@
+; RUN: opt < %s -passes='print' -debug-only=loop-accesses
-disable-output 2>&1 | FileCheck %s
+
+; REQUIRES: asserts
+
+
+define void @frexp_f64(ptr %in, ptr %out1, ptr %out2, i32 %N) {
+; CHECK: LAA: Allow to vectorize math function with write-only attribute:
%call = tail call double @frexp
+entry:
+ %cmp4 = icmp sgt i32 %N, 0
+ br i1 %cmp4, label %for.body.preheader, label %for.cond.cleanup
+
+for.body.preheader:
+ %wide.trip.count = zext nneg i32 %N to i64
fhahn wrote:
this shouldn't be needed, please try to simplify the test cases
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2477,6 +2493,15 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo
*LI,
// Save 'store' instructions. Abort if other instructions write to
memory.
if (I.mayWriteToMemory()) {
+// We can safety handle math functions that have vectorized
fhahn wrote:
typo: safely
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2477,6 +2493,15 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo *LI, // Save 'store' instructions. Abort if other instructions write to memory. fhahn wrote: comment out of date https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/fhahn edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/fhahn requested changes to this pull request. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/mgabka approved this pull request. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
paschalis-mpeis wrote: ### Rebased to: 1. Update to latest main 2. Amend commits for doing tests in veclib-function-calls.ll. For (2), the commit history were modified as follows: - The initial commit (that showcases what was missing) was amended to add the tests in veclib-function-calls.ll. - Then, subsequent commits were similarly amended to include the updates that allow vectorization. - This currently latest commit simply dropped the no longer needed tests (veclib-function-calls-linear-ptrs.ll) https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,134 @@ +; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --filter "call.*(frexp|modf)" --version 4 mgabka wrote: sincos (which also takes linear pointers) has tests inside veclib-function-calls.ll, so I would suggest to move this tests to the same file. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,117 @@ +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s + +; REQUIRES: asserts + +target triple = "aarch64-unknown-linux-gnu" + +; TODO: add mappings for frexp/frexpf paschalis-mpeis wrote: Short answer: Not needed. --- **Longer answer:** Using `-passes='print'` (as per @mgabka suggestion), LAA pass can run regardless of vecLib existence, TLI mappings, and/or even SVE support. Previously, in the LAA test [attr-mem-write-only.ll](https://github.com/llvm/llvm-project/pull/78432/commits/c3aa63c98627c2edf16420c5a4aa53ba233d544c#diff-6e65eaeb2e328f78d3620c102adaa22dd2a4e6bd0e677208ecb32e772c21089c) I had: - `-mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize` and yet, I couldn't check for the fexpr calls, as `LoopVectorizationLegality` would not allow vectorization. With the latest patch, however, LAA runs regardless, so I'm able to do such checks already ([here](https://github.com/llvm/llvm-project/pull/78432/commits/c3aa63c98627c2edf16420c5a4aa53ba233d544c#diff-6e65eaeb2e328f78d3620c102adaa22dd2a4e6bd0e677208ecb32e772c21089cR7) and [here](https://github.com/llvm/llvm-project/pull/78432/commits/c3aa63c98627c2edf16420c5a4aa53ba233d544c#diff-6e65eaeb2e328f78d3620c102adaa22dd2a4e6bd0e677208ecb32e772c21089cR34)). In the future, when the mappings for frexp are added, only the LV tests would need to be updated ([AArch64/veclib-function-calls-linear-ptrs.ll](https://github.com/llvm/llvm-project/pull/78432/commits/c3aa63c98627c2edf16420c5a4aa53ba233d544c#diff-7dd871d3ed08b76c541f7abb3a1d170800dab9d90dd16c4795ad747c94a17e65)). https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,117 @@ +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s + +; REQUIRES: asserts + +target triple = "aarch64-unknown-linux-gnu" + +; TODO: add mappings for frexp/frexpf fhahn wrote: Are you planning to add support to LoopVectorizationLegality as well? Would probably good to do this first https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis updated
https://github.com/llvm/llvm-project/pull/78432
>From d3ca209a7690d9ecbe188d8a2145177f43ed1951 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 16 Jan 2024 10:53:09 +
Subject: [PATCH 1/7] LAA cannot vectorize lib calls like modf/modff
Functions like modf/modff are math lib calls that set memory write-only
attribute. Given that a target has vectorized mappings, LAA should allow
vectorization.
---
...arch64-veclib-function-calls-linear-ptrs.c | 57 +++
1 file changed, 57 insertions(+)
create mode 100644
clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 00..a449fac147058a
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,57 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
>From 1dc1b5cb8e52761027d85edcd057ef57710d2699 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Wed, 17 Jan 2024 09:44:45 +
Subject: [PATCH 2/7] [LV][LAA] Vectorize math lib calls with mem write-only
attribute
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by inferNonMandatoryLibFuncAttrs, in BuildLibCalls.cpp, and the
current ones are modf/modff and frexp/frexpf.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
...arch64-veclib-function-calls-linear-ptrs.c | 15 ++-
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 19 +++
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
index a449fac147058a..957b3f5cb235d3 100644
--- a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -17,7 +17,7 @@ vectorize.
// CHE
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,117 @@ +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s paschalis-mpeis wrote: Indeed, using `-passes='print'` is a better option here: - it allows running LAA on the code regardless of LV's Legality result - this would allow checking for `frexp` methods in a target agnostic manner. Great suggestion, thanks! https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis ready_for_review https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis updated
https://github.com/llvm/llvm-project/pull/78432
>From d3ca209a7690d9ecbe188d8a2145177f43ed1951 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 16 Jan 2024 10:53:09 +
Subject: [PATCH 1/7] LAA cannot vectorize lib calls like modf/modff
Functions like modf/modff are math lib calls that set memory write-only
attribute. Given that a target has vectorized mappings, LAA should allow
vectorization.
---
...arch64-veclib-function-calls-linear-ptrs.c | 57 +++
1 file changed, 57 insertions(+)
create mode 100644
clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 00..a449fac147058a
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,57 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
>From 1dc1b5cb8e52761027d85edcd057ef57710d2699 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Wed, 17 Jan 2024 09:44:45 +
Subject: [PATCH 2/7] [LV][LAA] Vectorize math lib calls with mem write-only
attribute
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by inferNonMandatoryLibFuncAttrs, in BuildLibCalls.cpp, and the
current ones are modf/modff and frexp/frexpf.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
...arch64-veclib-function-calls-linear-ptrs.c | 15 ++-
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 19 +++
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
index a449fac147058a..957b3f5cb235d3 100644
--- a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -17,7 +17,7 @@ vectorize.
// CHE
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis converted_to_draft https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,117 @@ +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s + +; REQUIRES: asserts + +target triple = "aarch64-unknown-linux-gnu" + +; TODO: add mappings for frexp/frexpf paschalis-mpeis wrote: It turns out without the mappings LV Legality does not let it vectorize earlier ([here](https://github.com/llvm/llvm-project/blob/eaf0d82529f30fd53b453886676821d67ccfe9af/llvm/lib/Transforms/Vectorize/LoopVectorizationLegality.cpp#L900)): > LV: Not vectorizing: Found a non-intrinsic callsite %call = tail call float > @frexpf(float noundef %0, ptr noundef %add.ptr) > .. >LV: Not vectorizing: Cannot prove legality I will keep the those LAA frexp/frexpf tests, but I'll convert TODO's in a similar fashion with the [next comment](https://github.com/llvm/llvm-project/pull/78432#discussion_r1581031468). https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2422,6 +2438,15 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo
*LI,
// Save 'store' instructions. Abort if other instructions write to
memory.
if (I.mayWriteToMemory()) {
+// We can safety handle math functions that have vectorized
+// counterparts and have the memory write-only attribute set.
+if (isMathLibCallMemWriteOnly(TLI, I)) {
mgabka wrote:
in the read case we also check for !VFDatabase::getMappings(*Call).empty(), but
i think it should be enough that LV checks for it, that allows also to test
your code in a target agnostic way, and without mappings for frexp
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,117 @@ +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s mgabka wrote: I think you should only be calling here : "-passes='print' -debug-only=loop-accesses -disable-output" https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,117 @@ +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -debug-only=loop-accesses -disable-output 2>&1 | FileCheck %s + +; REQUIRES: asserts + +target triple = "aarch64-unknown-linux-gnu" + +; TODO: add mappings for frexp/frexpf mgabka wrote: you can still test for the expected dbg output even if there is no mappings, mappings are tested later by LV I think. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2422,6 +2438,15 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo
*LI,
// Save 'store' instructions. Abort if other instructions write to
memory.
if (I.mayWriteToMemory()) {
+// We can safety handle math functions that have vectorized
+// counterparts and have the memory write-only attribute set.
+if (isMathLibCallMemWriteOnly(TLI, I)) {
+ LLVM_DEBUG(dbgs()
+ << "LAA: allow math function with write-only attribute:"
mgabka wrote:
I think it is worth to edit this message a bit so it gives the full context in
dbg output, what about:
"LAA: Allow to vectorize math function with write-only attribute:" ?
(also the dbg message start from capital letter)
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,132 @@ +; NOTE: Assertions have been autogenerated by utils/update_test_checks.py UTC_ARGS: --filter "call.*(frexp|modf)" --version 4 +; RUN: opt < %s -mattr=+sve -vector-library=ArmPL -passes=inject-tli-mappings,loop-vectorize -force-vector-interleave=1 -prefer-predicate-over-epilogue=predicate-dont-vectorize -S | FileCheck %s + +target triple = "aarch64-unknown-linux-gnu" + +; TODO: add mappings for frexp/frexpf mgabka wrote: is this actually a TODO? I would rather change it into a comment: ; Vectorization can not happen because there is no scalar to vector mapping in TLI for frexp/frexpf. Tests will need to be changed when such mapping are added. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
paschalis-mpeis wrote: Rebased to main after #80296 was merged: - #80296 https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis ready_for_review https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis edited https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis updated
https://github.com/llvm/llvm-project/pull/78432
>From a74ba110994e4535cd6c9206aa02d50503fb5577 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 27 Feb 2024 15:00:28 +
Subject: [PATCH 1/7] [AArch64][TLI] Add TLI mappings for ArmPL modf, sincos,
sincospi
---
llvm/include/llvm/Analysis/VecFuncs.def | 6 ++
.../AArch64/veclib-function-calls.ll | 66 ++-
llvm/test/Transforms/Util/add-TLI-mappings.ll | 32 +++--
3 files changed, 67 insertions(+), 37 deletions(-)
diff --git a/llvm/include/llvm/Analysis/VecFuncs.def
b/llvm/include/llvm/Analysis/VecFuncs.def
index 394e4a05fbc0cf..10f1333cf8885c 100644
--- a/llvm/include/llvm/Analysis/VecFuncs.def
+++ b/llvm/include/llvm/Analysis/VecFuncs.def
@@ -1005,6 +1005,8 @@ TLI_DEFINE_VECFUNC("llvm.log2.f32", "armpl_svlog2_f32_x",
SCALABLE(4), MASKED, "
TLI_DEFINE_VECFUNC("modf", "armpl_vmodfq_f64", FIXED(2), NOMASK,
"_ZGV_LLVM_N2vl8")
TLI_DEFINE_VECFUNC("modff", "armpl_vmodfq_f32", FIXED(4), NOMASK,
"_ZGV_LLVM_N4vl4")
+TLI_DEFINE_VECFUNC("modf", "armpl_svmodf_f64_x", SCALABLE(2), MASKED,
"_ZGVsMxvl8")
+TLI_DEFINE_VECFUNC("modff", "armpl_svmodf_f32_x", SCALABLE(4), MASKED,
"_ZGVsMxvl4")
TLI_DEFINE_VECFUNC("nextafter", "armpl_vnextafterq_f64", FIXED(2), NOMASK,
"_ZGV_LLVM_N2vv")
TLI_DEFINE_VECFUNC("nextafterf", "armpl_vnextafterq_f32", FIXED(4), NOMASK,
"_ZGV_LLVM_N4vv")
@@ -1033,9 +1035,13 @@ TLI_DEFINE_VECFUNC("llvm.sin.f32", "armpl_svsin_f32_x",
SCALABLE(4), MASKED, "_Z
TLI_DEFINE_VECFUNC("sincos", "armpl_vsincosq_f64", FIXED(2), NOMASK,
"_ZGV_LLVM_N2vl8l8")
TLI_DEFINE_VECFUNC("sincosf", "armpl_vsincosq_f32", FIXED(4), NOMASK,
"_ZGV_LLVM_N4vl4l4")
+TLI_DEFINE_VECFUNC("sincos", "armpl_svsincos_f64_x", SCALABLE(2), MASKED,
"_ZGVsMxvl8l8")
+TLI_DEFINE_VECFUNC("sincosf", "armpl_svsincos_f32_x", SCALABLE(4), MASKED,
"_ZGVsMxvl4l4")
TLI_DEFINE_VECFUNC("sincospi", "armpl_vsincospiq_f64", FIXED(2), NOMASK,
"_ZGV_LLVM_N2vl8l8")
TLI_DEFINE_VECFUNC("sincospif", "armpl_vsincospiq_f32", FIXED(4), NOMASK,
"_ZGV_LLVM_N4vl4l4")
+TLI_DEFINE_VECFUNC("sincospi", "armpl_svsincospi_f64_x", SCALABLE(2), MASKED,
"_ZGVsMxvl8l8")
+TLI_DEFINE_VECFUNC("sincospif", "armpl_svsincospi_f32_x", SCALABLE(4), MASKED,
"_ZGVsMxvl4l4")
TLI_DEFINE_VECFUNC("sinh", "armpl_vsinhq_f64", FIXED(2), NOMASK,
"_ZGV_LLVM_N2v")
TLI_DEFINE_VECFUNC("sinhf", "armpl_vsinhq_f32", FIXED(4), NOMASK,
"_ZGV_LLVM_N4v")
diff --git
a/llvm/test/Transforms/LoopVectorize/AArch64/veclib-function-calls.ll
b/llvm/test/Transforms/LoopVectorize/AArch64/veclib-function-calls.ll
index dd1495626eb984..d9cc630482fc80 100644
--- a/llvm/test/Transforms/LoopVectorize/AArch64/veclib-function-calls.ll
+++ b/llvm/test/Transforms/LoopVectorize/AArch64/veclib-function-calls.ll
@@ -2925,11 +2925,12 @@ define void @modf_f64(ptr noalias %a, ptr noalias %b,
ptr noalias %c) {
;
; ARMPL-SVE-LABEL: define void @modf_f64
; ARMPL-SVE-SAME: (ptr noalias [[A:%.*]], ptr noalias [[B:%.*]], ptr noalias
[[C:%.*]]) #[[ATTR0]] {
-; ARMPL-SVE:[[DATA:%.*]] = call double @modf(double [[NUM:%.*]], ptr
[[GEPB:%.*]]) #[[ATTR4:[0-9]+]]
+; ARMPL-SVE:[[TMP23:%.*]] = call
@armpl_svmodf_f64_x( [[WIDE_MASKED_LOAD:%.*]], ptr
[[TMP22:%.*]], [[ACTIVE_LANE_MASK:%.*]])
;
; ARMPL-SVE-NOPRED-LABEL: define void @modf_f64
; ARMPL-SVE-NOPRED-SAME: (ptr noalias [[A:%.*]], ptr noalias [[B:%.*]], ptr
noalias [[C:%.*]]) #[[ATTR0]] {
-; ARMPL-SVE-NOPRED:[[TMP5:%.*]] = call <2 x double> @armpl_vmodfq_f64(<2 x
double> [[WIDE_LOAD:%.*]], ptr [[TMP4:%.*]])
+; ARMPL-SVE-NOPRED:[[TMP17:%.*]] = call
@armpl_svmodf_f64_x( [[WIDE_LOAD:%.*]], ptr [[TMP16:%.*]],
shufflevector ( insertelement ( poison, i1 true, i64 0), poison,
zeroinitializer))
+; ARMPL-SVE-NOPRED:[[DATA:%.*]] = call double @modf(double [[NUM:%.*]],
ptr [[GEPB:%.*]]) #[[ATTR64:[0-9]+]]
;
entry:
br label %for.body
@@ -2970,11 +2971,12 @@ define void @modf_f32(ptr noalias %a, ptr noalias %b,
ptr noalias %c) {
;
; ARMPL-SVE-LABEL: define void @modf_f32
; ARMPL-SVE-SAME: (ptr noalias [[A:%.*]], ptr noalias [[B:%.*]], ptr noalias
[[C:%.*]]) #[[ATTR0]] {
-; ARMPL-SVE:[[DATA:%.*]] = call float @modff(float [[NUM:%.*]], ptr
[[GEPB:%.*]]) #[[ATTR5:[0-9]+]]
+; ARMPL-SVE:[[TMP23:%.*]] = call
@armpl_svmodf_f32_x( [[WIDE_MASKED_LOAD:%.*]], ptr
[[TMP22:%.*]], [[ACTIVE_LANE_MASK:%.*]])
;
; ARMPL-SVE-NOPRED-LABEL: define void @modf_f32
; ARMPL-SVE-NOPRED-SAME: (ptr noalias [[A:%.*]], ptr noalias [[B:%.*]], ptr
noalias [[C:%.*]]) #[[ATTR0]] {
-; ARMPL-SVE-NOPRED:[[TMP5:%.*]] = call <4 x float> @armpl_vmodfq_f32(<4 x
float> [[WIDE_LOAD:%.*]], ptr [[TMP4:%.*]])
+; ARMPL-SVE-NOPRED:[[TMP17:%.*]] = call
@armpl_svmodf_f32_x( [[WIDE_LOAD:%.*]], ptr [[TMP16:%.*]],
shufflevector ( insertelement ( poison, i1 true, i64 0), poison,
zeroinitializer))
+; ARMPL-SVE-NOPRED:[[DATA:%.*]] = call float @modff(float [[NUM:%.*]], ptr
[[GEPB:%.*]]) #[[A
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2405,6 +2421,11 @@ void LoopAccessInfo::analyzeLoop(AAResults *AA, LoopInfo
*LI,
// Save 'store' instructions. Abort if other instructions write to
memory.
if (I.mayWriteToMemory()) {
+// We can safety handle math functions that have vectorized
+// counterparts and have the memory write-only attribute set.
+if (isMathLibCallMemWriteOnly(TLI, I))
mgabka wrote:
can this change be tested with a loop-access analysis tests using a debug
output? if yes then it does not have to depend on the TLI mappings for
modf/modff
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis converted_to_draft https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,52 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -isystem
%S/../Headers/Inputs/include -mllvm -vector-library=ArmPL -mllvm
-force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
paschalis-mpeis wrote:
> This test needs to be in either llvm/test/Transforms/LoopVectorize or
> llvm/test/Analysis/LoopAccessAnalysis, depending on what exactly you want to
> test.
I've used a C test, hence the placement. You'd prefer an LLVM IR test?
> I don't really get what this test is checking for though, it doesn't look
> like anything was actually vectorized?
Actually, the below patch removed modf/modff TLI mappings since this one was
opened, due to some errors that were discovered. I'm converting this patch to a
draft until those mappings are back again.
- #80296
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -0,0 +1,52 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -isystem
%S/../Headers/Inputs/include -mllvm -vector-library=ArmPL -mllvm
-force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
nikic wrote:
This test needs to be in either `llvm/test/Transforms/LoopVectorize` or
`llvm/test/Analysis/LoopAccessAnalysis`, depending on what exactly you want to
test.
I don't really get what this test is checking for though, it doesn't look like
anything was actually vectorized?
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
paschalis-mpeis wrote: Rebased to main after a couple of weeks of inactivity. Note: windows x64 build failure seems unrelated; more likely a wide problem, failing at cmake configure. https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis updated
https://github.com/llvm/llvm-project/pull/78432
>From 869079fa57b3a8da9b9f9ea541f7d542be1bb8bc Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 16 Jan 2024 10:53:09 +
Subject: [PATCH 1/5] LAA cannot vectorize lib calls like modf/modff
Functions like modf/modff are math lib calls that set memory write-only
attribute. Given that a target has vectorized mappings, LAA should allow
vectorization.
---
...arch64-veclib-function-calls-linear-ptrs.c | 57 +++
1 file changed, 57 insertions(+)
create mode 100644
clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 00..a449fac147058a
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,57 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
>From 32d2b4581dd19e289f5b21de6de931b4f6268612 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Wed, 17 Jan 2024 09:44:45 +
Subject: [PATCH 2/5] [LV][LAA] Vectorize math lib calls with mem write-only
attribute
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by inferNonMandatoryLibFuncAttrs, in BuildLibCalls.cpp, and the
current ones are modf/modff and frexp/frexpf.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
...arch64-veclib-function-calls-linear-ptrs.c | 15 ++-
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 19 +++
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
index a449fac147058a..957b3f5cb235d3 100644
--- a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -17,7 +17,7 @@ vectorize.
// CHE
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis updated
https://github.com/llvm/llvm-project/pull/78432
>From fbe30cfa2e9474241be71f334228bb811678d0a2 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 16 Jan 2024 10:53:09 +
Subject: [PATCH 1/5] LAA cannot vectorize lib calls like modf/modff
Functions like modf/modff are math lib calls that set memory write-only
attribute. Given that a target has vectorized mappings, LAA should allow
vectorization.
---
...arch64-veclib-function-calls-linear-ptrs.c | 57 +++
1 file changed, 57 insertions(+)
create mode 100644
clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 0..a449fac147058
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,57 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
>From 80a86c8a75877cc1e00e2d72bc4ed424728ec6fc Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Wed, 17 Jan 2024 09:44:45 +
Subject: [PATCH 2/5] [LV][LAA] Vectorize math lib calls with mem write-only
attribute
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by inferNonMandatoryLibFuncAttrs, in BuildLibCalls.cpp, and the
current ones are modf/modff and frexp/frexpf.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
...arch64-veclib-function-calls-linear-ptrs.c | 15 ++-
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 19 +++
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
index a449fac147058..957b3f5cb235d 100644
--- a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -17,7 +17,7 @@ vectorize.
// CHECK-L
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis updated
https://github.com/llvm/llvm-project/pull/78432
>From 9e6020071d90decae5d61eced320a3f114b9d2a1 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 16 Jan 2024 10:53:09 +
Subject: [PATCH 1/4] LAA cannot vectorize lib calls like modf/modff
Functions like modf/modff are math lib calls that set memory write-only
attribute. Given that a target has vectorized mappings, LAA should allow
vectorization.
---
...arch64-veclib-function-calls-linear-ptrs.c | 57 +++
1 file changed, 57 insertions(+)
create mode 100644
clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 000..a449fac147058ae
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,57 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
>From 97ebe120a9971903ed2aacd03db3fccd35452a30 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Wed, 17 Jan 2024 09:44:45 +
Subject: [PATCH 2/4] [LV][LAA] Vectorize math lib calls with mem write-only
attribute
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by inferNonMandatoryLibFuncAttrs, in BuildLibCalls.cpp, and the
current ones are modf/modff and frexp/frexpf.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
...arch64-veclib-function-calls-linear-ptrs.c | 15 ++-
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 19 +++
2 files changed, 25 insertions(+), 9 deletions(-)
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
index a449fac147058ae..957b3f5cb235d31 100644
--- a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -17,7 +17,7 @@ vectorize.
//
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis converted_to_draft https://github.com/llvm/llvm-project/pull/78432 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
@@ -2274,6 +2274,26 @@ bool LoopAccessInfo::canAnalyzeLoop() {
return true;
}
+/// Returns whether \p I is a known math library call that has attribute
+/// 'memory(argmem: write)' set.
+static bool isMathLibCallMemWriteOnly(const TargetLibraryInfo *TLI,
+ const Instruction &I) {
+ auto *Call = dyn_cast(&I);
+ if (!Call)
+return false;
+
+ Function *F = Call->getCalledFunction();
+ if (!F->hasFnAttribute(Attribute::AttrKind::Memory))
+return false;
+
+ auto ME = F->getFnAttribute(Attribute::AttrKind::Memory).getMemoryEffects();
paschalis-mpeis wrote:
Good suggestion thanks, latest commit applies this.
https://github.com/llvm/llvm-project/pull/78432
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
llvmbot wrote:
@llvm/pr-subscribers-llvm-analysis
Author: Paschalis Mpeis (paschalis-mpeis)
Changes
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by `inferNonMandatoryLibFuncAttrs`, in BuildLibCalls.cpp, and the
current ones are `modf`/`modff` and `frexp`/`frexpf`.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
Full diff: https://github.com/llvm/llvm-project/pull/78432.diff
2 Files Affected:
- (added) clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c (+54)
- (modified) llvm/lib/Analysis/LoopAccessAnalysis.cpp (+19)
``diff
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 000..957b3f5cb235d31
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,54 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR5:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR5]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[TMP11:%.*]] = tail call
@armpl_svmodf_f64_x( [[WIDE_MASKED_LOAD:%.*]], ptr
[[TMP10:%.*]], [[ACTIVE_LANE_MASK:%.*]])
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP14:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR6:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[TMP11:%.*]] = tail call
@armpl_svmodf_f32_x( [[WIDE_MASKED_LOAD:%.*]], ptr
[[TMP10:%.*]], [[ACTIVE_LANE_MASK:%.*]])
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef
[[TMP14:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR7:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
diff --git a/llvm/lib/Analysis/LoopAccessAnalysis.cpp
b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
index aed60cc5a3f5ef0..0c8b4e51fcf5c16 100644
--- a/llvm/lib/Analysis/LoopAccessAnalysis.cpp
+++ b/llvm/lib/Analysis/LoopAccessAnalysis.cpp
@@ -2274,6 +2274,20 @@ bool LoopAccessInfo::canAnalyzeLoop() {
return true;
}
+/// Returns whether \p I is a known math library call that has memory
write-only
+/// attribute set.
+static bool isMathLibCallMemWriteOnly(const TargetLibraryInfo *TLI,
+ const Instruction &I) {
+ auto *Call = dyn_cast(&I);
+ if (!Call)
+return false;
+
+ LibFunc Func;
+ TLI->getLibFunc(*Call, Func);
+ return Func == LibFunc::LibFunc_modf || Func == LibFunc::LibFunc_modff ||
+ Func == LibFunc::LibFunc_frexp || Func == LibFunc::LibFunc_frexpf;
+}
+
void LoopAccessInfo::analyzeLoop(
[clang] [llvm] LV][LAA] Vectorize math lib calls with mem write-only attribute (PR #78432)
https://github.com/paschalis-mpeis created
https://github.com/llvm/llvm-project/pull/78432
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by `inferNonMandatoryLibFuncAttrs`, in BuildLibCalls.cpp, and the
current ones are `modf`/`modff` and `frexp`/`frexpf`.
This happens only when the calls are found through TLI to have
vectorized counterparts.
>From ff9557fe913fd7901cd5a18757a2a41bffbe5c0e Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Tue, 16 Jan 2024 10:53:09 +
Subject: [PATCH 1/2] LAA cannot vectorize lib calls like modf/modff
Functions like modf/modff are math lib calls that set memory write-only
attribute. Given that a target has vectorized mappings, LAA should allow
vectorization.
---
...arch64-veclib-function-calls-linear-ptrs.c | 57 +++
1 file changed, 57 insertions(+)
create mode 100644
clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
diff --git a/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
new file mode 100644
index 00..a449fac147058a
--- /dev/null
+++ b/clang/test/CodeGen/aarch64-veclib-function-calls-linear-ptrs.c
@@ -0,0 +1,57 @@
+// NOTE: Assertions have been autogenerated by utils/update_cc_test_checks.py
UTC_ARGS: --filter "call.*(frexp|modf)" --version 4
+// RUN: %clang --target=aarch64-linux-gnu -march=armv8-a+sve -O3 -mllvm
-vector-library=ArmPL -mllvm -force-vector-interleave=1 -mllvm
-prefer-predicate-over-epilogue=predicate-dont-vectorize -emit-llvm -S -o - %s
| FileCheck %s
+
+// REQUIRES: aarch64-registered-target
+
+/*
+Testing vectorization of math functions that have the attribute write-only to
+memory set. Given they have vectorized counterparts, they should be able to
+vectorize.
+*/
+
+// The following define is required to access some math functions.
+#define _GNU_SOURCE
+#include
+
+// frexp/frexpf have no TLI mappings yet.
+
+// CHECK-LABEL: define dso_local void @frexp_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0:[0-9]+]] {
+// CHECK:[[CALL:%.*]] = tail call double @frexp(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2:[0-9]+]]
+//
+void frexp_f64(double *in, double *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexp(in[i], out2+i);
+}
+
+// CHECK-LABEL: define dso_local void @frexp_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @frexpf(float noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR2]]
+//
+void frexp_f32(float *in, float *out1, int *out2, int N) {
+ for (int i = 0; i < N; ++i)
+*out1 = frexpf(in[i], out2+i);
+}
+
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f64(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call double @modf(double noundef
[[TMP0:%.*]], ptr noundef [[ADD_PTR:%.*]]) #[[ATTR3:[0-9]+]]
+//
+void modf_f64(double *in, double *out1, double *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modf(in[i], out2+i);
+}
+
+// TODO: LAA must allow vectorization.
+
+// CHECK-LABEL: define dso_local void @modf_f32(
+// CHECK-SAME: ptr nocapture noundef readonly [[IN:%.*]], ptr nocapture
noundef writeonly [[OUT1:%.*]], ptr nocapture noundef writeonly [[OUT2:%.*]],
i32 noundef [[N:%.*]]) local_unnamed_addr #[[ATTR0]] {
+// CHECK:[[CALL:%.*]] = tail call float @modff(float noundef [[TMP0:%.*]],
ptr noundef [[ADD_PTR:%.*]]) #[[ATTR4:[0-9]+]]
+//
+void modf_f32(float *in, float *out1, float *out2, int N) {
+ for (int i = 0; i < N; ++i)
+ out1[i] = modff(in[i], out2+i);
+}
>From 2a3b1e60c3f68c2f823a42bcd6e67111a0b8a529 Mon Sep 17 00:00:00 2001
From: Paschalis Mpeis
Date: Wed, 17 Jan 2024 09:44:45 +
Subject: [PATCH 2/2] [LV][LAA] Vectorize math lib calls with mem write-only
attribute
Teach LAA to consider safe specific math lib calls which are known to
have set the memory write-only attribute. Those attributes are set to
calls by inferNonMandatoryLibFuncAttrs, in BuildLibCalls.cpp, and the
current ones are modf/modff and frexp/frexpf.
This happens only when the calls are found through TLI to have
vectorized counterparts.
---
...arch64-veclib-function-calls-linear-ptrs.c | 15 ++-
llvm/lib/Analysis/LoopAccessAnalysis.cpp | 19 +++
2 files changed, 25 insertions(+), 9 deletions(-
