When basic block vectorization is extended to utilise length-agnostic
vector types such as VNx8HF (which can store at least 8 scalars), SLP
groups for which the number of required vectors cannot be determined
will be split during discovery. For conversions, we will not be able
to predict the input vector type until it is too late to split a group.
Consequently, we must instead prepare to reject conversions for which
the number of input vectors cannot be determined in
vectorizable_conversion.
Promotion no longer produces redundant definitions, therefore demotion
can no longer rely on an even number of defs being produced. If
necessary, vectorizable_conversion now pushes a single constant zero
def.
gcc/ChangeLog:
* tree-vect-stmts.cc (vectorizable_conversion): Reject
conversions for which the number of input vectors cannot be
determined. If necessary, push a single constant zero
definition as the second operand of the final demotion
operation done by the vect_create_vectorized_demotion_stmts
function.
---
gcc/tree-vect-stmts.cc | 25 +++++++++++++++++++++++++
1 file changed, 25 insertions(+)
diff --git a/gcc/tree-vect-stmts.cc b/gcc/tree-vect-stmts.cc
index 09e45d0629a..98c16a43228 100644
--- a/gcc/tree-vect-stmts.cc
+++ b/gcc/tree-vect-stmts.cc
@@ -5453,6 +5453,19 @@ vectorizable_conversion (vec_info *vinfo,
modifier = WIDEN;
}
+ /* SLP groups whose number of vectors cannot be determined are split during
+ discovery. For conversions, we cannot predict the input vector type until
+ later, when it is too late to split a group. For example, a conversion of
+ 8 scalars from VNx4SF to VNx8HF is rejected here because the input can be
+ either 1 or 2 vectors (whereas the output is always 1 vector). */
+ if (slp_node && is_a<bb_vec_info> (vinfo))
+ {
+ unsigned int nvectors;
+ if (!can_div_away_from_zero_p (poly_uint64 (SLP_TREE_LANES (slp_node)),
+ nunits_in, &nvectors))
+ return false;
+ }
+
bool found_mode = false;
scalar_mode lhs_mode = SCALAR_TYPE_MODE (lhs_type);
scalar_mode rhs_mode = SCALAR_TYPE_MODE (rhs_type);
@@ -5879,6 +5892,18 @@ vectorizable_conversion (vec_info *vinfo,
generate more than one vector stmt - i.e - we need to "unroll"
the vector stmt by a factor VF/nunits. */
vect_get_vec_defs (vinfo, slp_node, true, &vec_oprnds0);
+
+ /* Promotion no longer produces redundant defs (since support was
+ added for length/mask-predicated BB SLP of awkward-sized groups),
+ therefore demotion now has to handle that case too. */
+ if (vec_oprnds0.length () % 2 != 0)
+ {
+ /* The zero supplies the second operand for the final demotion
+ operation. */
+ tree vectype = TREE_TYPE (vec_oprnds0[0]);
+ vec_oprnds0.safe_push (build_zero_cst (vectype));
+ }
+
/* Arguments are ready. Create the new vector stmts. */
if (cvt_type && modifier == NARROW_DST)
FOR_EACH_VEC_ELT (vec_oprnds0, i, vop0)
--
2.43.0
