New tests verify that GCC can generate predicated vector-length
specific code for AArch64 if the vector length specified by
-msve-vector-bits is longer than the number of elements to be
processed (including if the specified length is sufficient but
the minimum scalable vector length would not be).
Other new tests verify that GCC can generate predicated
vector-length agnostic code for AArch64 if the minimum scalable
vector length (of 16 bytes) is longer than the number of elements
to be processed.
Other new tests verify that ASIMD instructions are still generated
if the number of elements to be processed is equal to the specified
or minimum scalable vector length.
Other new tests verify that a combination of ASIMD and SVE
instructions are generated if the configured or minimum scalable
vector length is shorter than the number of elements to be
processed (non-predicated head, predicated tail).
The eighth new test verifies a surprising invariant:
According to the default tuning model, the cost of a copy of two
contiguous bytes is the same whether the operation is vectorized
or not, therefore it is vectorised. That decision is reached
despite the fact that the model pessimistically allows for
predicate masks that are not ultimately produced as part of the
vectorized code. (2x scalar_store + 2x scalar_load = cost 4 vs.
1x vector_load + 1x vector_store + 2x vector_stmt = cost 4.)
The next five new tests verify that sharing of masks does not
go wrong: masks must only be shared between SLP groups of
different sizes and different truth-vector types, where a
conversion ratio is satisfied, or between SLP groups of the same
size and truth-vector type. However, this cannot be verified
solely using scan-assembler-times and scan_assembler-not because
the 'fre' (redundancy elimination) pass turns masks shared by
means of VIEW_CONVERT_EXPR back into separate constants. It does
so even if each mask requires two instructions to set up (mov
and whilelo).
The last three new tests verify that integer division and
floating point arithmetic operations are masked, but integer
addition is not.
gcc/testsuite/ChangeLog:
* gcc.target/aarch64/sve/slp_pred_1.c: New test.
* gcc.target/aarch64/sve/slp_pred_1_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_2.c: New test.
* gcc.target/aarch64/sve/slp_pred_2_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_3.c: New test.
* gcc.target/aarch64/sve/slp_pred_3_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_4.c: New test.
* gcc.target/aarch64/sve/slp_pred_4_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_5.c: New test.
* gcc.target/aarch64/sve/slp_pred_6.c: New test.
* gcc.target/aarch64/sve/slp_pred_6_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_7.c: New test.
* gcc.target/aarch64/sve/slp_pred_7_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_8.c: New test.
* gcc.target/aarch64/sve/slp_pred_8_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_9.c: New test.
* gcc.target/aarch64/sve/slp_pred_9_run.c: New test.
* gcc.target/aarch64/sve/slp_pred_10.c: New test.
* gcc.target/aarch64/sve/slp_pred_11.c: New test.
* gcc.target/aarch64/sve/slp_pred_12.c: New test.
* gcc.target/aarch64/sve/slp_pred_13.c: New test.
* gcc.target/aarch64/sve/slp_pred_14.c: New test.
* gcc.target/aarch64/sve/slp_pred_15.c: New test.
* gcc.target/aarch64/sve/slp_pred_16.c: New test.
* gcc.target/aarch64/sve/slp_pred_17.c: New test.
* gcc.target/aarch64/sve/slp_pred_18.c: New test.
* gcc.target/aarch64/sve/slp_pred_harness.h: Test harness
shared between tests for vectorization with SVE predication.
---
.../gcc.target/aarch64/sve/slp_pred_1.c | 35 ++++++++++++
.../gcc.target/aarch64/sve/slp_pred_10.c | 23 ++++++++
.../gcc.target/aarch64/sve/slp_pred_11.c | 36 ++++++++++++
.../gcc.target/aarch64/sve/slp_pred_12.c | 51 +++++++++++++++++
.../gcc.target/aarch64/sve/slp_pred_13.c | 45 +++++++++++++++
.../gcc.target/aarch64/sve/slp_pred_14.c | 35 ++++++++++++
.../gcc.target/aarch64/sve/slp_pred_15.c | 36 ++++++++++++
.../gcc.target/aarch64/sve/slp_pred_16.c | 26 +++++++++
.../gcc.target/aarch64/sve/slp_pred_17.c | 27 +++++++++
.../gcc.target/aarch64/sve/slp_pred_18.c | 24 ++++++++
.../gcc.target/aarch64/sve/slp_pred_1_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_2.c | 35 ++++++++++++
.../gcc.target/aarch64/sve/slp_pred_2_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_3.c | 40 +++++++++++++
.../gcc.target/aarch64/sve/slp_pred_3_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_4.c | 40 +++++++++++++
.../gcc.target/aarch64/sve/slp_pred_4_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_5.c | 38 +++++++++++++
.../gcc.target/aarch64/sve/slp_pred_6.c | 43 ++++++++++++++
.../gcc.target/aarch64/sve/slp_pred_6_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_7.c | 48 ++++++++++++++++
.../gcc.target/aarch64/sve/slp_pred_7_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_8.c | 42 ++++++++++++++
.../gcc.target/aarch64/sve/slp_pred_8_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_9.c | 57 +++++++++++++++++++
.../gcc.target/aarch64/sve/slp_pred_9_run.c | 6 ++
.../gcc.target/aarch64/sve/slp_pred_harness.h | 28 +++++++++
27 files changed, 757 insertions(+)
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_10.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_11.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_12.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_13.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_14.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_15.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_16.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_17.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_18.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c
new file mode 100644
index 00000000000..58c56f6d316
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c
@@ -0,0 +1,35 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+
+#include <stdint.h>
+
+/* Test that we can vectorize with SVE predication when generating
vector-length
+ agnostic code if the minimum possible vector length (of 16 bytes) is larger
+ than the number of elements to be processed. */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1; // one less than the minimum vector length
+}
+
+/* { dg-final { scan-assembler-times {\twhilelo\tp[0-7]\.b, xzr, x[0-9]+\n} 1
} } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-not {\tstr\tq[0-9]+} } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_10.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_10.c
new file mode 100644
index 00000000000..c21e1acdc81
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_10.c
@@ -0,0 +1,23 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only" } */
+/* { dg-final { check-function-bodies "**" "" } } */
+
+#include <stdint.h>
+
+/* According to the default tuning model, the cost of this function is the same
+ whether it is vectorized or not, therefore it is vectorised. That decision
+ is reached despite pessimistically allowing for predicate masks that are not
+ ultimately produced as part of the vectorized code. */
+
+/*
+** f:
+** ldr (h[0-9]+), \[x1\]
+** str \1, \[x0\]
+** ret
+*/
+void
+f (uint8_t (*dst)[2], uint8_t (*__restrict src)[2])
+{
+ (*dst)[0] = (*src)[0];
+ (*dst)[1] = (*src)[1];
+}
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_11.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_11.c
new file mode 100644
index 00000000000..d1ca8f4a5ba
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_11.c
@@ -0,0 +1,36 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that SVE predicate masks are not shared between SLP groups of the same
+ size if they use different vector types that have different element sizes.
*/
+
+void
+f (uint8_t *x, uint16_t *y)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+
+ y[0] += 1;
+ y[1] += 2;
+ y[2] += 1;
+ y[3] += 2;
+ y[4] += 1;
+ y[5] += 2;
+ y[6] += 1;
+}
+
+/* A single mask must not be shared between x and y in GIMPLE so expect two
+ masks to be set up. */
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 7, \{
0, \.\.\. \}\)} 2 "slp2" } } */
+/* { dg-final { scan-tree-dump-not {VIEW_CONVERT_EXPR} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.b, vl7\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl7\n} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_12.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_12.c
new file mode 100644
index 00000000000..3ec52099cbc
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_12.c
@@ -0,0 +1,51 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that SVE predicate masks are shared between SLP groups despite those
+ groups having different sizes and using vector types that have different
+ element sizes, if the ratio between element sizes is the reciprocal of the
+ ratio between group sizes. */
+
+void
+f (uint8_t *x, uint32_t *y)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+
+ y[0] += 1;
+ y[1] += 2;
+ y[2] += 1;
+}
+
+/* A twelve-element mask should be shared between x and y in GIMPLE, although y
+ has only three active elements. Expect two conversions: one for the load
+ operation and one for the store operation.
+ */
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 12,
\{ 0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump-times {VIEW_CONVERT_EXPR<vector\(\[4,4\]\)
<signed-boolean:4>>\(slp_mask_[0-9]+\)} 2 "slp2" } } */
+
+/* Although the mask is shared in GIMPLE, the 'fre' (redundancy elimination)
+ pass converts it to separate constants. It does so even if both masks
+ require a mov and whilelo (which is not the case here). */
+
+/* { dg-final { scan-assembler-times {\tmov\tw[0-7], 12\n} 1 } } */
+/* { dg-final { scan-assembler-times {\twhilelo\tp[0-7]\.b, xzr, x[0-9]+\n} 1
} } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+.b, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+.b, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.s, vl3\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1w\tz[0-9]+.s, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1w\tz[0-9]+.s, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_13.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_13.c
new file mode 100644
index 00000000000..f0245e8df52
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_13.c
@@ -0,0 +1,45 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that SVE predicate masks are not shared between SLP groups of different
+ sizes that use different vector types, if the ratio between element sizes is
+ not the reciprocal of the ratio between group sizes. */
+
+void
+f (uint8_t *x, uint16_t *y)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+
+ y[0] += 1;
+ y[1] += 2;
+ y[2] += 1;
+}
+
+/* A single mask must not be shared between x and y in GIMPLE so expect two
+ masks to be set up. */
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 12,
\{ 0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 3, \{
0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump-not {VIEW_CONVERT_EXPR} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tmov\tw[0-7], 12\n} 1 } } */
+/* { dg-final { scan-assembler-times {\twhilelo\tp[0-7]\.b, xzr, x[0-9]+\n} 1
} } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+.b, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+.b, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl3\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1h\tz[0-9]+.h, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1h\tz[0-9]+.h, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_14.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_14.c
new file mode 100644
index 00000000000..6eee3f5705d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_14.c
@@ -0,0 +1,35 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that SVE predicate masks are shared between SLP groups of the same
+ size that use the same vector type. */
+
+void
+f (uint16_t *x, uint16_t *y)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+
+ y[0] += 1;
+ y[1] += 2;
+ y[2] += 3;
+ y[3] += 4;
+ y[4] += 5;
+ y[5] += 6;
+}
+
+/* The six-element mask should be shared between x and y in GIMPLE. Expect no
+ conversions because they have the same truth type. */
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 6, \{
0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump-not {VIEW_CONVERT_EXPR} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl6\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1h\tz[0-9]+.h, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 2 } } */
+/* { dg-final { scan-assembler-times {\tst1h\tz[0-9]+.h, p[0-7],
\[x[0-9]+[^]]*\]} 2 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_15.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_15.c
new file mode 100644
index 00000000000..3978a1bd39b
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_15.c
@@ -0,0 +1,36 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that SVE predicate masks are shared between SLP groups of the same
+ size despite those groups using different vector types if those types have
+ the same element size. */
+
+void
+f (uint16_t *x, _Float16 *y)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+
+ y[0] += 1;
+ y[1] += 2;
+ y[2] += 3;
+ y[3] += 4;
+ y[4] += 5;
+ y[5] += 6;
+}
+
+/* The six-element mask should be shared between x and y in GIMPLE. Expect no
+ conversions because they have the same truth type. */
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 6, \{
0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump-not {VIEW_CONVERT_EXPR} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl6\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1h\tz[0-9]+.h, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 2 } } */
+/* { dg-final { scan-assembler-times {\tst1h\tz[0-9]+.h, p[0-7],
\[x[0-9]+[^]]*\]} 2 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_16.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_16.c
new file mode 100644
index 00000000000..751e43853c9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_16.c
@@ -0,0 +1,26 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that floating point arithmetic is masked. */
+
+void
+f (_Float16 *restrict x, _Float16 *restrict y)
+{
+ x[0] += y[0];
+ x[1] += y[1];
+ x[2] += y[2];
+ x[3] += y[3];
+ x[4] += y[4];
+ x[5] += y[5];
+ x[6] += y[6];
+}
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 7, \{
0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump {\.COND_ADD} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl7\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1h\tz[0-9]+.h, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 2 } } */
+/* { dg-final { scan-assembler-times {\tfadd\tz[0-9]+.h, p[0-7]/m, z[0-9]+.h,
z[0-9]+.h\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1h\tz[0-9]+.h, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_17.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_17.c
new file mode 100644
index 00000000000..2178caf88e4
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_17.c
@@ -0,0 +1,27 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that integer addition is not masked, but loads and stores are
+ still masked. */
+
+void
+f (int16_t *restrict x, int16_t *restrict y)
+{
+ x[0] += y[0];
+ x[1] += y[1];
+ x[2] += y[2];
+ x[3] += y[3];
+ x[4] += y[4];
+ x[5] += y[5];
+ x[6] += y[6];
+}
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 7, \{
0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump-not {\.COND_ADD} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl7\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1h\tz[0-9]+.h, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 2 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+.h, z[0-9]+.h,
z[0-9]+.h\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1h\tz[0-9]+.h, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_18.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_18.c
new file mode 100644
index 00000000000..ec2a2d7aac9
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_18.c
@@ -0,0 +1,24 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable
-fdump-tree-slp2" } */
+
+#include <stdint.h>
+
+/* Test that integer division is masked. We cannot use a narrower element type
+ such as int16_t with more than four lanes because integer division must be
+ performed as if on the promoted type, which is 32-bit. */
+
+void
+f (int32_t *restrict x, int32_t *restrict y)
+{
+ x[0] /= y[0];
+ x[1] /= y[1];
+ x[2] /= y[2];
+}
+
+/* { dg-final { scan-tree-dump-times {max_mask_[0-9]+ = \.WHILE_ULT \(0, 3, \{
0, \.\.\. \}\)} 1 "slp2" } } */
+/* { dg-final { scan-tree-dump {\.COND_DIV} "slp2" } } */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.s, vl3\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1w\tz[0-9]+.s, p[0-7]\/z,
\[x[0-9]+[^]]*\]} 2 } } */
+/* { dg-final { scan-assembler-times {\tsdiv\tz[0-9]+.s, p[0-7]/m, z[0-9]+.s,
z[0-9]+.s\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1w\tz[0-9]+.s, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c
new file mode 100644
index 00000000000..b76f8d52a1f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_1.c"
+
+HARNESS (15)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c
new file mode 100644
index 00000000000..603e7c72daf
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c
@@ -0,0 +1,35 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+
+#include <stdint.h>
+
+/* Test that we can vectorize with SVE predication when generating
vector-length
+ specific code if the configured vector length is larger than the number of
+ elements to be processed. */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1; // one less than the configured vector length
+}
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.b, mul3\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-not {\tstr\tq[0-9]+} } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2_run.c
new file mode 100644
index 00000000000..6e010b2ab4c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_2.c"
+
+HARNESS (15)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c
new file mode 100644
index 00000000000..d039b38a2c3
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c
@@ -0,0 +1,40 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+
+#include <stdint.h>
+
+/* Test that we produce ASIMD load and store instructions instead of an SVE
+ predicated load or store when generating vector-length agnostic code if the
+ minimum possible vector length (of 16 bytes) is equal to the number of
+ elements to be processed. */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2; // exactly fits the minimum vector length
+}
+
+/* Be specific about the addressing mode to disambiguate from a constant load.
*/
+/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]+\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+
+/* ptrue is used as part of making an immediate constant. */
+/* { dg-final { scan-assembler-not {\twhilelo\tp[0-7]} } } */
+/* { dg-final { scan-assembler-not {\tld1b\tz[0-9]+} } } */
+/* { dg-final { scan-assembler-not {\tst1b\tz[0-9]+} } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c
new file mode 100644
index 00000000000..eb3093059d1
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_3.c"
+
+HARNESS (16)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c
new file mode 100644
index 00000000000..150213e270f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c
@@ -0,0 +1,40 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+
+#include <stdint.h>
+
+/* Test that we produce ASIMD load and store instructions instead of an SVE
+ predicated load or store when generating vector-length specific code if the
+ configured vector length is equal to the number of elements to be processed.
+ */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2; // exactly fits the configured vector length
+}
+
+/* Be specific about the addressing mode to disambiguate from a constant load.
*/
+/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]+\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+
+/* { dg-final { scan-assembler-not {\tptrue\tp[0-7]} } } */
+/* { dg-final { scan-assembler-not {\twhilelo\tp[0-7]} } } */
+/* { dg-final { scan-assembler-not {\tld1b\tz[0-9]+} } } */
+/* { dg-final { scan-assembler-not {\tst1b\tz[0-9]+} } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4_run.c
new file mode 100644
index 00000000000..933d22ebe1d
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_4.c"
+
+HARNESS (16)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c
new file mode 100644
index 00000000000..1afb617ed35
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c
@@ -0,0 +1,38 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=256" } */
+
+#include <stdint.h>
+
+/* Test that we can vectorize with SVE predication when generating
+ vector-length specific code if the number of elements to be
+ processed is greater than the minimum possible vector length
+ (of 16 bytes) but less than the configured vector length. */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2;
+ x[16] += 1; // one more than the minimum vector length
+}
+
+/* { dg-final { scan-assembler-times {\twhilelo\tp[0-7]\.b, xzr, x[0-9]+\n} 1
} } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z,
\[x[0-9]+[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-not {\tstr\tq[0-9]+} } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c
new file mode 100644
index 00000000000..06b3b4f5d5f
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c
@@ -0,0 +1,43 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+
+#include <stdint.h>
+
+/* Test that we do not attempt to vectorize with SVE predication when
+ generating vector-length agnostic code if the minimum possible
+ vector length (of 16 bytes) is smaller than the number of elements
+ to be processed. */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2;
+ x[16] += 1; // one more than the minimum vector length
+}
+
+/* ptrue is used as part of making an immediate constant. */
+/* { dg-final { scan-assembler-not {\twhilelo\tp[0-7]} } } */
+
+/* Be specific about the addressing mode to disambiguate from a constant load.
*/
+/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]+\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+
+/* { dg-final { scan-assembler-times {\tldrb\tw[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+/* { dg-final { scan-assembler-times {\tadd\tw[0-9]+, w[0-9]+, 1\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tstrb\tw[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c
new file mode 100644
index 00000000000..d6b6537ac23
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_6.c"
+
+HARNESS (17)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c
new file mode 100644
index 00000000000..788480d737c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c
@@ -0,0 +1,48 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+
+#include <stdint.h>
+
+/* Test that we can vectorize with SVE predication when generating
vector-length
+ agnostic code if the minimum possible vector length (of 16 bytes) is so much
+ smaller than the number of elements to be processed that the vectorized
+ statements are split between two vectors (one predicated, the other not).
*/
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2;
+ x[16] += 1;
+ x[17] += 2;
+ x[18] += 1;
+ x[19] += 2;
+ x[20] += 1;
+ x[21] += 2;
+ x[22] += 1;
+}
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.b, vl7\n} 1 } } */
+
+/* Be specific about the addressing mode to disambiguate from a constant load.
*/
+/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]+\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.b, p[0-7]/z,
\[x[0-9]+[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 2 } } */
+
+/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.b, p[0-7],
\[x[0-9]+[^]]*\]} 1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7_run.c
new file mode 100644
index 00000000000..9e94ea0a711
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=scalable" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_7.c"
+
+HARNESS (23)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8.c
new file mode 100644
index 00000000000..119fe5cd577
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8.c
@@ -0,0 +1,42 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+
+#include <stdint.h>
+
+/* Test that we do not attempt to vectorize with SVE predication when
+ generating vector-length specific code if the configured vector
+ length is smaller than the number of elements to be processed. */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2;
+ x[16] += 1; // one more than the configured vector length
+}
+
+/* { dg-final { scan-assembler-not {\tptrue\tp[0-7]} } } */
+/* { dg-final { scan-assembler-not {\twhilelo\tp[0-7]} } } */
+
+/* Be specific about the addressing mode to disambiguate from a constant load.
*/
+/* { dg-final { scan-assembler-times {\tldr\tq[0-9]+, \[x[0-9]+\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+
+/* { dg-final { scan-assembler-times {\tldrb\tw[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
+/* { dg-final { scan-assembler-times {\tadd\tw[0-9]+, w[0-9]+, 1\n} 1 } } */
+/* { dg-final { scan-assembler-times {\tstrb\tw[0-9]+, \[x[0-9]+[^]]*\]} 1 } }
*/
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8_run.c
new file mode 100644
index 00000000000..9b7045a22e5
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_8.c"
+
+HARNESS (17)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9.c
new file mode 100644
index 00000000000..dcaeb8f6457
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9.c
@@ -0,0 +1,57 @@
+/* { dg-do compile } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+
+#include <stdint.h>
+
+/* Test that we can vectorize with SVE predication when generating
vector-length
+ specific code if the configured vector length is so much smaller than the
+ number of elements to be processed that the vectorized statements are split
+ between two vectors (one predicated, the other not). */
+
+void
+f (uint8_t *x)
+{
+ x[0] += 1;
+ x[1] += 2;
+ x[2] += 1;
+ x[3] += 2;
+ x[4] += 1;
+ x[5] += 2;
+ x[6] += 1;
+ x[7] += 2;
+ x[8] += 1;
+ x[9] += 2;
+ x[10] += 1;
+ x[11] += 2;
+ x[12] += 1;
+ x[13] += 2;
+ x[14] += 1;
+ x[15] += 2;
+ x[16] += 1;
+ x[17] += 2;
+ x[18] += 1;
+ x[19] += 2;
+ x[20] += 1;
+ x[21] += 2;
+ x[22] += 1;
+}
+
+/* A middle-end heuristic that picks the narrowest fixed-width vector type to
+ represent a partial vector means that the second SLP group is vectorized as
+ V8QI instead of as V16QI. The heuristic prevents regressions in x64 tests
+ but causes weird output for this test case:
+ - two extra instructions.
+ - one fewer load (because the first, V16QI, constant can be immediate).
+ - one extra predicate mask (because the second, V8QI, constant is expanded
+ from bytes to halfwords when loaded).
+ */
+
+/* { dg-final { scan-assembler-times {\tptrue\tp[0-7]\.h, vl7\n} 1 } } */
+
+/* { dg-final { scan-assembler {\tldr\tq[0-9]+, \[x0[^]]*\]} } } */
+/* { dg-final { scan-assembler-times {\tld1b\tz[0-9]+\.h, p[0-7]/z,
\[x0[^]]*\]} 1 } } */
+
+/* { dg-final { scan-assembler-times {\tadd\tz[0-9]+\.b, z[0-9]+\.b,
z[0-9]+\.b\n} 2 } } */
+
+/* { dg-final { scan-assembler-times {\tstr\tq[0-9]+, \[x0[^]]*\]} 1 } } */
+/* { dg-final { scan-assembler-times {\tst1b\tz[0-9]+\.h, p[0-7], \[x0[^]]*\]}
1 } } */
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9_run.c
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9_run.c
new file mode 100644
index 00000000000..11951536352
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9_run.c
@@ -0,0 +1,6 @@
+/* { dg-do run { target aarch64_sve_hw } } */
+/* { dg-options "-O2 -ftree-vectorize
--param=aarch64-autovec-preference=sve-only -msve-vector-bits=128" } */
+#include "slp_pred_harness.h"
+#include "slp_pred_9.c"
+
+HARNESS (23)
diff --git a/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h
b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h
new file mode 100644
index 00000000000..ac569fc670c
--- /dev/null
+++ b/gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h
@@ -0,0 +1,28 @@
+/* Test harness shared between tests for vectorization with SVE predication.
*/
+
+#define HARNESS(N)
\
+ int __attribute__ ((optimize (1))) main (void)
\
+ {
\
+ uint8_t a[N], b[N];
\
+ for (unsigned int i = 0; i < N; ++i)
\
+ {
\
+ a[i] = i * 2 + i % 5; \
+ b[i] = a[i]; \
+ asm volatile ("" ::: "memory"); \
+ }
\
+ f (a);
\
+ for (unsigned int i = 0; i < N; i += 2)
\
+ {
\
+ b[i]++; \
+ if (a[i] != b[i]) \
+ __builtin_abort (); \
+ if (i + 1 < N) \
+ { \
+ b[i + 1] += 2; \
+ if (a[i + 1] != b[i + 1]) \
+ __builtin_abort (); \
+ } \
+ asm volatile ("" ::: "memory"); \
+ }
\
+ return 0;
\
+ }
--
2.43.0
