GCC already supports fully-predicated vectorisation for loops, both
using "traditional" loop vectorisation and loop-aware SLP
(superword-level parallelism). For example, GCC can vectorise:
void
foo (char *x)
{
for (int i = 0; i < 6; i += 2)
{
x[i] += 1;
x[i + 1] += 2;
}
}
from which it generates the following assembly code (with -O2
-ftree-vectorize -march=armv9-a+sve -msve-vector-bits=scalable):
foo:
ptrue p7.b, vl6
mov w1, 513
ld1b z31.b, p7/z, [x0]
mov z30.h, w1
add z30.b, z31.b, z30.b
st1b z30.b, p7, [x0]
ret
However, GCC cannot yet vectorise the unrolled form of the same
function, even though it is semantically equivalent:
void
foo (char *x)
{
x[0] += 1;
x[1] += 2;
x[2] += 1;
x[3] += 2;
x[4] += 1;
x[5] += 2;
}
These patches implement support for vectorising the unrolled form of
the above function by enabling use of a predicate mask or length
limit for basic block SLP. For example, it can now be vectorised to
the following assembly code (using the same options as above):
foo:
adrp x1, .LANCHOR0
ptrue p7.b, vl6
ld1b z31.b, p7/z, [x0]
ldr d30, [x1, #:lo12:.LANCHOR0]
add z30.b, z31.b, z30.b
st1b z30.b, p7, [x0]
ret
The initial vector mode for an SLP region is "autodetected" by calling
aarch64_preferred_simd_mode, which prefers SVE modes if supported and
unless configured otherwise (e.g. VNx4SI for int). If at least one
profitable subgraph can be scheduled then GCC does not try to vectorise
the region using any other modes, even though their estimated costs
might otherwise have been lower.
That is mitigated by the fact that a sequence of GIMPLE stmts such as:
vectp.14_86 = x_50(D) + 16;
slp_mask_87 = .WHILE_ULT (0, 8, { 0, ... });
.MASK_STORE (vectp.14_86, 8B, slp_mask_87, vect__34.12_85);
are lowered to a fixed-length vector store (e.g., str d30, [x0, 16]) if
possible, instead of a more literal interpretation such as:
add x0, x0, 16
ptrue p7.b, vl8
st1b z30.b, p7, [x0]
Based on commit d6bac80f3dd.
Bootstrapped and regression-tested on aarch64-unknown-linux-gnu and
x86_64-pc-linux-gnu. No test result regressions are known but one
existing test has gained an xfail because of PR 127271.
Changes in v2:
- Updated regexes used by the vect-over-widen-*.c tests so that they
do not accidentally match text that is now part of the dump but
was not previously.
- Updated regexes used by the aarch64/popcnt-sve.c test so that it
expects 'all' as the operand of 'ptrue', instead of some specific
number of active elements.
- Removed a dump_printf from vect_get_num_copies because the
gcc.dg/vect/vect-shift-5.c test relies on a fixed dump layout
spanning multiple lines.
- Fixed a bug in vect_get_vector_types_for_stmt, which had
accidentally been modified to unconditionally set group_size to
zero (even for basic block vectorization).
- Relaxed an overzealous new check in
vect_maybe_update_slp_op_vectype, which now considers the
vectorization factor when checking the number of lanes of external
definitions during loop vectorization. e.g., lanes=11 is not
divisible by subparts=8 with vf=1 but vf*lanes=88 is divisible by
subparts=8 with vf=11.
- Removed the stmts vector ownership changes relating to mishandling
of failure of the vect_analyze_slp_instance function (to be fixed
separately).
- A check in get_vectype_for_scalar_type for whether the natural
choice of vector type satisfies the group size was too simplistic.
Instead of choosing a narrower vector type if the natural vector
type could be long enough but not definitely (variable length, by
proxy), get_len_load_store_mode is now used to explicitly query
whether the target supports mask- or length-limited loads and
stores. With the previous version, GCC preferred the natural vector
type if it was known to be long enough; sometimes that resulted in
better output than a narrower type, but it also caused some
bb-slp-* tests to fail.
- Shuffled dump_printfs around into separate patches.
- An assertion is now used to protect my change to use lower bound
of number of subparts in gimple_build_vector.
Changes in v3:
- Wrote changelog entries.
- Created the gimple_build_vector_with_zero_padding function and
used it place of the gimple_build_vector function.
- Reverted my change to use constant_lower_bound of subparts in the
gimple_build_vector function.
- Fixed a check for 'partial vectors required' in vect_analyze_stmt
to include cases in which the minimum bound of the length of a
variable-length vector type equals the number of active lanes in an
SLP tree node. (Maybe less than, instead of known to be less than.)
- Reverted my change to regexes used by the aarch64/popcnt-sve.c test
because it is not needed after the above fix.
- Replaced SLP_TREE_CAN_USE_MASK_P and SLP_TREE_CAN_USE_LEN_P with
SLP_TREE_PARTIAL_VECTORS_STYLE.
- Titivated the documentation of vect_record_nunits.
- Renamed local variable max_nunits to nunits in the
vect_analyze_slp_reduc_chain function.
- Added an assertion in vect_create_constant_vectors to verify a
claimed relationship between group size and number of subparts.
- Fixed a mistake in the description of vect_slp_get_bb_len.
- Clarified the descriptions of vect_record_mask and vect_record_len
(from 'would be required' to 'could be used ... if required').
- Renamed the vect_can_use_mask_p function as vect_fully_masked_p
and vect_can_use_len_p as vect_fully_with_length_p.
- Tightened assertions in vect_record_mask and vect_record_len:
instead of just 'the other function must not have been called', we
now assert that no partial vectors style has already been set.
- Updated the documentation of check_load_store_for_partial_vectors
to describe its role in BB SLP vectorization.
- Renamed masked_loop_p and len_loop_p as masked_p and len_p in
contexts where they may now be used for BB SLP vectorization.
- Guard against a potential null pointer dereference in
LOOP_VINFO_PARTIAL_LOAD_STORE_BIAS when used by the
vectorizable_operation and vectorizable_call functions for BB SLP
vectorization (instead of assuming !vect_fully_with_length_p).
- Guard against calling dump_printf_loc with null instead of
a vector type in vectorizable_comparison_1.
- Clear any partial vectors style that might have been set by callees
of vect_analyze_stmt if it finds that partial vectors aren't needed.
- Revert a change to make the vect_is_simple_use function more robust
when the requested SLP tree child node is not an internal def and
has no scalar operand to return.
- Revert a change to reserve slots for the definitions to be pushed
for narrow FLOAT_EXPR vectorization before using quick_push in the
vectorizable_conversion function.
Changes in v4:
- Resolved code generation regressions in
gcc.target/aarch64/sve/slp_6.c and
gcc.target/aarch64/sve/slp_7_costly.c by adding code to
handle variable-length vector types in store_constructor.
(Still had to update expectations for slp_6.c though.)
- Removed a default constructor from the definition of
struct slp_tree_nunits in order to fix a compilation error in
the vect_update_nunits function.
- Renamed the gimple_build_vector_with_zero_padding function as
gimple_build_vector_from_elems.
- Changed gimple_build_vector_from_elems to take a vector type
and list of element values instead of a tree_vector_builder.
- Rewrote the description of gimple_build_vector_from_elems.
- Removed redundant masking from gimple_build_vector_from_elems. It's
now almost equivalent to gimple_build_vector in use cases where
some element value is not constant.
- Assert that the constant lower bound of the number of subparts
in a vector type is a power of 2 instead of merely a multiple of 2
in vect_build_slp_tree_1.
Changes in v5:
- Removed the patch to track the minimum and maximum number of lanes
for BB SLP.
- Added a new function, vect_slp_tree_min_nunits, which is invoked by
vect_analyze_slp_instance to compute the minimum number of subparts
for all of the vector types used in an SLP tree for which an SLP
instance is about to be created.
Changes in v6:
- Moved the BB SLP implementations of vect_record_mask and
vect_record_len into tree-vect-slp.cc.
- Relaxed (failing) assertions: the style of an SLP node is not always
vect_partial_vectors_none on entry to vect_record_mask and
vect_record_len. Allow the same style to be set multiple times.
- Updated an existing pattern named 'Simplify vector extracts' to
guard against invalid invocations of tree_to_uhwi when a
BIT_FIELD_REF has an unsuitable polynomial offset or size.
- Rebased on 43afcb3a83c3648141285d80cd3d8a562047fb43.
Changes in v7:
- Fixed the vectorizable_live_operation function so that it no
longer generates invalid offsets such as BIT_FIELD_REF
<_251, 32, POLY_INT_CST [96, 128]> for BB SLP with a
variable-length vector type.
- Revert changes to the existing pattern named 'Simplify vector
extracts' because those changes to make the pattern more robust
are now redundant.
- Rebased on 630c1bfbb5bc3ba9fafca5e97096263ab8f0a04b.
Changes in v8:
- Fixed a regression in vectorizable_simd_clone_call that was
caused by an error when rebasing. (Silent conflict with
commit cea34ac07e3bd7.)
Changes in v9:
- Optimised vec_init for partial SVE vector modes and added a
test to guard against any return to the pathological stack usage
previously observed.
- Removed -march=armv9-a+sve from the dg-options of new tests
because it was redundant.
- More arguments are now passed through to vect_slp_record_bb_mask
and vect_slp_record_bb_len (from vect_slp_record_mask or
vect_slp_record_len) in anticipation of a future implementation
that shares masks, at least within an SLP sub-tree.
- A new per-SLP-node variable, SLP_TREE_NUM_PARTIAL_VECTORS, is
now used to calculate a worst-case estimate of the cost of
creating masks or lengths for partial vector tails. This aligns
BB SLP with existing costing of vectorised loops.
- Replaced early returns from vect_slp_get_bb_mask and
vect_slp_get_bb_len (when more than one vector, the requested vector
is not the last, or not partial) with code to construct an
appropriate mask or length. The value of NULL_TREE previously
returned in such edge is not a valid mask, so this change makes the
code more robust. vect_slp_record_bb_mask has been seen to be called
with nvectors=2 when running the asrdiv_2 test (msve-vector-bits=128,
group_size=8, nunits=4). The vector type was 128 bit: vector(4) int.
- Added a defensive check for BB SLP as part of the refactoring of
check_load_store_for_partial_vectors that is done by the first patch,
although we don't expect that function to be called for BB SLP until
vect_can_use_partial_vectors_p has been updated to sometimes return
true for BB SLP (which is done by the 'Extend BB SLP vectorization
to use predicated tails' patch).
- Rebased on commit a28bb06b3e20a26579e06dc1b5bd6344ce4f88f0.
Changes in v10:
- store_constructor now zeros RTL elements up to the lower bound of the
number of subparts in a variable-length vector type instead of
only zeroing the number of elements specified by the constructor.
- Documented the treatment of partial constructors of vector type in
the description of store_constructor.
- Reverted some of my previous changes to vect_create_constant_vectors
because it is no longer deemed necessary to pad the length of
variable-length vector constants with zeros in GIMPLE.
- Rebased on commit ee7d0c7b4102b8ef0bcaa725c75e141248b774cb.
Consequently, vect_get_len and vect_slp_get_bb_len gained an extra
parameter.
- Update gcc.target/aarch64/sve/vec_construct_*.c to expect SVE
instead of ASIMD instructions and add
--param=aarch64-autovec-preference=sve-only to avoid flip-flopping.
- Increase the number of elements stored in vec_construct_3.c to
ensure vectorization using SVE is deemed profitable despite predicate
mask costs.
- Added a new test for vectorization of a copy of two contiguous bytes.
According to the default tuning, the cost is the same whether the
operation is vectorized or not, therefore it is vectorised.
Changes in v11:
- Update the description of vec_init to specify its behavior for
scalable vector modes.
Changes in v12:
- Removed 'Add extra conditional dump output to the vectorizer' (by request).
- Removed 'AArch64/SVE: Optimize vec_init for partial SVE vector modes'
and 'AArch64/SVE: Relax the expectations of the popcnt-sve test'
(both were pushed separately).
- Removed 'Handle variable-length vector types in store_constructor'
and 'Specify behavior of vec_init for scalable vector modes'
(squashed together and submitted for review separately by request).
- Added 'aarch64: improve code generation in aarch64_expand_sve_const_vector'.
(new patch, not previously submitted for review).
- Rework of 'Update constant creation for BB SLP with predicated tails'
to clarify when repetition of vector elements is actually required
(as opposed to an unwanted side-effect with obscure failure modes), and
to simplify code that previously called gimple_build_vector.
Changes in v13:
- Split the 'Prepare promotion/demotion for BB SLP with predicated tails'
patch from this series by request, although it is still a key dependency.
- Reworked constant creation for BB SLP with predicated tails to avoid
introducing new GIMPLE helper functions.
- Moved state for partial-vector masks and lengths from the
loop vectorizer class to its superclass vec_info, to allow them to be
shared with BB SLP vectorization.
- Partial-vector controls (masks and lengths) can now be shared between
different vectorized statements in an SLP region.
- Implemented deferred verification that partial vector requirements
recorded during the analysis phase are consistent and can be satisfied.
- Partial vector setup for BB SLP is now deferred instead of occurring
in the mask and length SSA name accessor functions.
- Several more existing tests are also now run-tests.
- Added tests for mask sharing, predicated arithmetic and splitting of SLP
groups.
- Existing tests are more exacting for important properties and more
permissive for incidental properties such as choice of addressing mode.
- The patch series has been reorganized into a larger number of more
focused commits. Several supporting changes and refactorings are now
presented separately.
Changes in v14:
- Substituted aarch64_sve128_hw for aarch64_sve_hw as the target for three
of the new SLP tests that run code compiled for a vector length of 128
bits (one of which previously failed on a machine with 256-bit vectors).
- Added three new tests to ensure that 'rev' instructions are still
generated instead of inappropriately vectorising byte order reversals
using BB SLP. 10c43486134 must therefore be an ancestor so that truncated
scalar stores are costed correctly by the vectorizer.
- Split the 'Tests for use of predicated vector tails for BB SLP' patch
into three patches because it had grown to incorporate unrelated kinds
of test that were already described separately.
- Rebased on ece564fdaa1.
Changes in v15:
- Rebased on d6bac80f3dd and fixed many conflicts with upstream.
- Updated vect_get_num_copies for BB SLP to calculate the number of vectors
using can_div_away_from_zero_p instead of returning 1 whenever the vector
type is long enough for the SLP node.
- Reworked recording of partial-vector requirements for BB SLP to support
sequences of more than one fixed-length vector. Full vectors require
no recorded control. Mask/length rgroups are now indexed by the number of
active lanes in the tail vector rather than by the whole SLP group size.
- Fixed vect_transform_slp_perm_load_1 to complete permutation-mask encodings
for partial SLP vectors by padding inactive lanes.
- Updated the new vect_get_num_copies_for_invariant helper introduced
upstream to support BB SLP with variable-length vector types when the SLP
group fits within the minimum vector length.
- Reworked vect_create_constant_vectors to use
vect_get_num_copies_for_invariant and its excess-element count, and to
distinguish the number of scalar operands in the final vector from the
minimum or fixed vector length.
- Brought the 'Prepare promotion/demotion for BB SLP with predicated tails'
patch back into this series but removed its promotion-side changes to
vect_create_vectorized_promotion_stmts, because upstream now limits the
number of results it produces.
- Cleared partial_vector_style when loop analysis ultimately decides not
to use partial vectors, to avoid creating SSA names for masks or lengths
that will not subsequently be defined.
- Changed vect_analyze_slp_instance to use the constant lower bound of a
VLA vector type when choosing an SLP-group split point, provided that the
matching prefix is divisible by that value. Deleted
vect_slp_tree_min_nunits and vect_update_slp_min_nunits_for_node.
- Adapted vect_build_slp_tree_1 to upstream changes while preserving the
v14 behaviour of using the minimum number of subparts of a VLA type to
choose an SLP splitting point.
- Renamed vect_min_prec_for_max_group_size as vect_min_prec_for_slp.
- Renamed max_group_size as max_normalized_group_size and group_size as
normalized_group_size in vect_min_prec_for_slp.
- Removed the vec_info parameter of vect_maybe_update_slp_op_vectype.
- The patch to guard against an infinite loop in vect_create_constant_vectors
was merged separately.
- Added a comment in vectorizable_conversion to explain why an extra zero
is pushed.
- Removed the code previously added to vect_maybe_update_slp_op_vectype
for the purpose of rejecting attempts to set a vector type for external
defs whose number of lanes is not divisible by the number of subparts.
Conversions for which it is impossible to determine the number of input
vectors are now rejected in vectorizable_conversion instead.
- Corrected the description of max_nscalars_per_iter for BB SLP
vectorization with vect_partial_vectors_len.
- Updated outdated descriptions of the vec_construct_1.c,
vec_construct_3.c and vec_construct_5.c tests.
- Changed the type of group_size to unsigned int in
check_load_store_for_partial_vectors.
- Fixed the spacing of some uses of template functions.
Link to v1:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v2:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v3:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v4:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v5:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v6:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v7:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v8:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v9:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v10:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v11:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v12:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v13:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Link to v14:
https://inbox.sourceware.org/gcc-patches/[email protected]/
Christopher Bazley (19):
vect: Prepare demotion for BB SLP with predicated tails
vect: Handle partial SLP vectors in vect_transform_slp_perm_load_1
vect: Preparation to support predicated tails for BB SLP
vect: Update constant creation for BB SLP with predicated tails
vect: Refactor check_load_store_for_partial_vectors
vect: Return a failure indication from
check_load_store_for_partial_vectors
vect: Handle BB SLP in check_load_store_for_partial_vectors
vect: Fix vectorizable_live_operation for BB SLP with a VLA type
vect: Prepare vectorizable_* functions for BB SLP with predicated
tails
vect: Record/cost/set BB SLP partial vector controls for WHILE_ULT
vect: Record/cost/set BB SLP partial vector controls for AVX512
vect: Record/cost/set BB SLP partial vector controls for LEN
vect: Extend BB SLP vectorization to use predicated tails
AArch64/SVE: Tests for use of predicated vector tails for BB SLP
AArch64/SVE: Tests for sharing of masks by BB SLP
AArch64/SVE: Tests for masked arithmetic with BB SLP
AArch64/SVE: Regression tests for use of 'rev' instead of BB SLP
AArch64/SVE: Optimize aarch64_expand_sve_const_vector
vect: Rename masked_loop_p and len_loop_p variables
gcc/config/aarch64/aarch64.cc | 135 ++-
gcc/gimple-fold.cc | 33 +-
.../gcc.dg/vect/vect-over-widen-10.c | 2 +-
.../gcc.dg/vect/vect-over-widen-13.c | 2 +-
.../gcc.dg/vect/vect-over-widen-14.c | 2 +-
.../gcc.dg/vect/vect-over-widen-17.c | 2 +-
.../gcc.dg/vect/vect-over-widen-18.c | 2 +-
gcc/testsuite/gcc.dg/vect/vect-over-widen-5.c | 2 +-
gcc/testsuite/gcc.dg/vect/vect-over-widen-6.c | 2 +-
gcc/testsuite/gcc.dg/vect/vect-over-widen-7.c | 2 +-
gcc/testsuite/gcc.dg/vect/vect-over-widen-8.c | 2 +-
gcc/testsuite/gcc.dg/vect/vect-over-widen-9.c | 2 +-
gcc/testsuite/gcc.target/aarch64/sve/slp_6.c | 18 +-
.../gcc.target/aarch64/sve/slp_pred_1.c | 43 +
.../gcc.target/aarch64/sve/slp_pred_10.c | 23 +
.../gcc.target/aarch64/sve/slp_pred_11.c | 36 +
.../gcc.target/aarch64/sve/slp_pred_12.c | 51 +
.../gcc.target/aarch64/sve/slp_pred_13.c | 45 +
.../gcc.target/aarch64/sve/slp_pred_14.c | 35 +
.../gcc.target/aarch64/sve/slp_pred_15.c | 36 +
.../gcc.target/aarch64/sve/slp_pred_16.c | 26 +
.../gcc.target/aarch64/sve/slp_pred_17.c | 27 +
.../gcc.target/aarch64/sve/slp_pred_18.c | 24 +
.../gcc.target/aarch64/sve/slp_pred_19.c | 35 +
.../gcc.target/aarch64/sve/slp_pred_1_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_2.c | 35 +
.../gcc.target/aarch64/sve/slp_pred_20.c | 31 +
.../gcc.target/aarch64/sve/slp_pred_21.c | 31 +
.../gcc.target/aarch64/sve/slp_pred_22.c | 31 +
.../gcc.target/aarch64/sve/slp_pred_23.c | 35 +
.../gcc.target/aarch64/sve/slp_pred_24.c | 29 +
.../gcc.target/aarch64/sve/slp_pred_25.c | 29 +
.../gcc.target/aarch64/sve/slp_pred_2_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_3.c | 40 +
.../gcc.target/aarch64/sve/slp_pred_3_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_4.c | 40 +
.../gcc.target/aarch64/sve/slp_pred_4_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_5.c | 38 +
.../gcc.target/aarch64/sve/slp_pred_6.c | 43 +
.../gcc.target/aarch64/sve/slp_pred_6_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_7.c | 48 +
.../gcc.target/aarch64/sve/slp_pred_7_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_8.c | 42 +
.../gcc.target/aarch64/sve/slp_pred_8_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_9.c | 57 ++
.../gcc.target/aarch64/sve/slp_pred_9_run.c | 6 +
.../gcc.target/aarch64/sve/slp_pred_harness.h | 28 +
.../gcc.target/aarch64/sve/slp_rev_32_1.c | 25 +
.../gcc.target/aarch64/sve/slp_rev_32_2.c | 29 +
.../gcc.target/aarch64/sve/slp_rev_32_3.c | 26 +
.../aarch64/sve/truncated_concatenation_1.c | 5 +-
.../gcc.target/aarch64/sve/vec_construct_1.c | 11 +-
.../gcc.target/aarch64/sve/vec_construct_2.c | 4 +-
.../gcc.target/aarch64/sve/vec_construct_3.c | 25 +-
.../gcc.target/aarch64/sve/vec_construct_4.c | 5 +-
.../gcc.target/aarch64/sve/vec_construct_5.c | 13 +-
gcc/tree-vect-loop.cc | 768 ++++++++++-----
gcc/tree-vect-slp.cc | 460 +++++++--
gcc/tree-vect-stmts.cc | 876 +++++++++++-------
gcc/tree-vectorizer.cc | 7 +-
gcc/tree-vectorizer.h | 346 +++----
61 files changed, 2956 insertions(+), 836 deletions(-)
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_10.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_11.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_12.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_13.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_14.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_15.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_16.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_17.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_18.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_19.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_1_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_20.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_21.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_22.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_23.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_24.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_25.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_2_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_3_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_4_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_5.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_6_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_7_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_8_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_9_run.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_pred_harness.h
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_rev_32_1.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_rev_32_2.c
create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/slp_rev_32_3.c
