Hi,
(Obviously for GCC 17)
This patch adds versioning to ensure that the loop control IV does
not overflow when vectorizing. It does so by adding a no-overflow
assumption in the vectorizer as well as a new loop constraint
LOOP_C_NOWRAP. In niter, we can make use of this constraint to
prove that a specific candidate IV does not overflow.
A drawback of this approach is that we might version loops too eagerly.
Before dataref analysis has been done we don't know whether the control
IV's no-overflow property is actually needed. In situations where the
dataref just depends on a pointer IV that does not wrap, we only realize
that after having decided to perform versioning. This leads to a
pessimization and additional loop prolog overhead.
I experimented with keeping a second "assumptions_without_nowrap"
around. Just before the actual versioning, we can then do another
dataref check, excluding all the nowrap control IVs, clearing
SCEV caches etc. If that check still result in "no overflow", we know
that the assumption was redundant and fall back to
"assumptions_without_nowrap". What makes this difficult is having a
second niter assumption throughout the analysis flow and needing to
duplicate or enhance several checks against the original assumptions to
operate on assumptions_without_nowrap as well.
My gut feeling is that this isn't worth it, so I left that part out of
the patch.
Another difference to before is that the default assumption
now, obviously, is that no overflow happens at runtime. If it does
happen, we fall back to scalar code. Before, we would vectorize using
gather/scatter that can handle overflow. Given that gather/scatter is
usually slow and a wrapping of their indices would likely make it even
slower, I'd say the new assumption is reasonable.
There's still tension regarding interpretation of a user's intent of
course. If somebody deliberately wrote a wrapping IV, expecting
gather/scatter vectorization, we pessimize now. A better solution could
be to fall back to gather/scatter instead of scalar code right away
and, depending on costs either vectorize or not. Vectorizing twice
is another can of worms that I didn't want to open.
For SPECint2017 on x86 we add assumptions to 107 loops in total, the bulk
of which are in xalan, 10 in 502.gcc, and 14 in 557.xz.
Performance, unfortunately, is a bit disappointing. On my x86 test
machine (Sandy Bridge) xz regresses a bit (2-3%). Same on aarch64
without SVE, improvements are, small at best:
Input 1 degrades by 4%, input 2 improves by 12%, input 3 regresses by 7%.
Input 2 has a longer window size, while input 1 and 3 have 64 bytes and
typically "break early" for <= 10 iterations.
My suspicion is that the additional prolog assumption checking eats away
profitability. We even statically estimate 11 iters for profitability.
On top, we need an epilogue, and with early break we currently re-do the
full vector iteration in scalar. I expect fully-masking architectures to
fare better but IMHO these experiments confirm the difficulty of costing
small early-break loops.
Bootstrapped and regtested on x86, power10, and aarch64.
Regtested on riscv64.
Regards
Robin
PR tree-optimization/121908
gcc/ChangeLog:
* cfgloop.h (struct GTY): Add condition.
(LOOP_C_NOWRAP): New constraint.
* tree-ssa-loop-niter.cc (number_of_iterations_ne):
Use LOOP_C_NOWRAP.
(record_control_iv): Record condition.
(loop_exits_before_overflow): Use LOOP_C_NOWRAP.
* tree-vect-loop.cc (vect_get_loop_niters): Add no-overflow
assumption and set LOOP_C_NOWRAP.
* tree-vectorizer.cc (vect_free_loop_info_assumptions): Clear
LOOP_C_NOWRAP.
(try_vectorize_loop_1): Ditto.
(pass_vectorize::execute): Ditto.
gcc/testsuite/ChangeLog:
* gcc.dg/tree-ssa/scev-8.c: Don't vectorize.
* gcc.dg/vect/vect-unsigned-assump-1.c: New test.
* gcc.dg/vect/vect-unsigned-assump-2.c: New test.
* gcc.dg/vect/vect-unsigned-assump-3.c: New test.
* gcc.dg/vect/vect-unsigned-assump-4.c: New test.
* gcc.dg/vect/vect-unsigned-assump-5.c: New test.
* gcc.dg/vect/vect-unsigned-assump-6.c: New test.
* gcc.dg/vect/vect-unsigned-assump-7.c: New test.
* gcc.dg/vect/vect-unsigned-assump-8.c: New test.
---
gcc/cfgloop.h | 4 ++
gcc/testsuite/gcc.dg/tree-ssa/scev-8.c | 2 +-
.../gcc.dg/vect/vect-unsigned-assump-1.c | 22 ++++++
.../gcc.dg/vect/vect-unsigned-assump-2.c | 24 +++++++
.../gcc.dg/vect/vect-unsigned-assump-3.c | 20 ++++++
.../gcc.dg/vect/vect-unsigned-assump-4.c | 16 +++++
.../gcc.dg/vect/vect-unsigned-assump-5.c | 14 ++++
.../gcc.dg/vect/vect-unsigned-assump-6.c | 13 ++++
.../gcc.dg/vect/vect-unsigned-assump-7.c | 17 +++++
.../gcc.dg/vect/vect-unsigned-assump-8.c | 16 +++++
gcc/tree-ssa-loop-niter.cc | 60 +++++++++++++++-
gcc/tree-vect-loop.cc | 69 +++++++++++++++++++
gcc/tree-vectorizer.cc | 11 ++-
13 files changed, 284 insertions(+), 4 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-1.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-2.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-3.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-4.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-5.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-6.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-7.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-8.c
diff --git a/gcc/cfgloop.h b/gcc/cfgloop.h
index de8d97c6141..9678b1363a2 100644
--- a/gcc/cfgloop.h
+++ b/gcc/cfgloop.h
@@ -113,6 +113,7 @@ enum loop_estimation
struct GTY ((chain_next ("%h.next"))) control_iv {
tree base;
tree step;
+ HOST_WIDE_INT condition;
struct control_iv *next;
};
@@ -281,6 +282,9 @@ public:
#define LOOP_C_INFINITE (1 << 0)
/* Set if the loop is known to be finite without any assumptions. */
#define LOOP_C_FINITE (1 << 1)
+/* Set if the loop is known to be non-wrapping under assumptions,
+ thus only valid in vectorizer context where the loop is versioned. */
+#define LOOP_C_NOWRAP (1 << 2)
/* Set C to the LOOP constraint. */
inline void
diff --git a/gcc/testsuite/gcc.dg/tree-ssa/scev-8.c
b/gcc/testsuite/gcc.dg/tree-ssa/scev-8.c
index a5b2ff71958..abdbf0250dd 100644
--- a/gcc/testsuite/gcc.dg/tree-ssa/scev-8.c
+++ b/gcc/testsuite/gcc.dg/tree-ssa/scev-8.c
@@ -1,5 +1,5 @@
/* { dg-do compile } */
-/* { dg-options "-O2 -fdump-tree-ivopts-details" } */
+/* { dg-options "-O2 -fno-tree-vectorize -fdump-tree-ivopts-details" } */
int *a;
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-1.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-1.c
new file mode 100644
index 00000000000..5d9c297c921
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-1.c
@@ -0,0 +1,22 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_early_break } */
+
+#define uint8_t unsigned char
+#define uint32_t unsigned int
+
+int foo (const uint8_t *const cur, uint32_t len, uint32_t len_limit,
+ uint32_t pos, uint32_t cur_match)
+{
+ const uint32_t delta = pos - cur_match;
+ const uint8_t *pb = cur - delta;
+
+ while (++len != len_limit)
+ if (pb[len] != cur[len])
+ break;
+
+ return len;
+}
+
+/* { dg-final { scan-tree-dump "vectorized 1 loops in function" "vect" } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-2.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-2.c
new file mode 100644
index 00000000000..249afe0127a
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-2.c
@@ -0,0 +1,24 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_early_break } */
+/* { dg-require-effective-target vect_load_lanes } */
+
+#define uint8_t unsigned char
+#define uint32_t unsigned int
+
+int foo (const uint8_t *const cur, uint32_t len, uint32_t len_limit,
+ uint32_t pos, uint32_t cur_match)
+{
+ const uint32_t delta = pos - cur_match;
+ const uint8_t *pb = cur - delta;
+
+ /* We vectorize this with struct loads right now. */
+ while ((len += 2) != len_limit)
+ if (pb[len] != cur[len])
+ break;
+
+ return len;
+}
+
+/* { dg-final { scan-tree-dump "vectorized 1 loops in function" "vect" } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-3.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-3.c
new file mode 100644
index 00000000000..5547c478ffb
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-3.c
@@ -0,0 +1,20 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_early_break } */
+
+#define uint8_t unsigned char
+#define uint32_t unsigned int
+
+int foo (const uint8_t *const cur, uint32_t n)
+{
+ uint32_t i = 15;
+
+ while (i++ != n)
+ if (cur[i - 15] != cur[i])
+ break;
+
+ return i;
+}
+
+/* { dg-final { scan-tree-dump "vectorized 1 loops in function" "vect" } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-4.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-4.c
new file mode 100644
index 00000000000..e5d4022a5bd
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-4.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-fno-builtin" } */
+/* { dg-require-effective-target vect_int } */
+
+/* We should vectorize this without additional assumptions.
+ The data-ref uses a pointer-based access which never wraps.
+ However, this is not implemented right now and we always version.
+ Thus, the test is xfailed unconditionally. */
+
+void f4 (int *p, unsigned start, unsigned end) {
+ for (unsigned i = start; i != end; i++)
+ *p++ = 0;
+}
+
+/* { dg-final { scan-tree-dump "Discarding tentative nowrap" "vect" { xfail
*-*-* } } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-5.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-5.c
new file mode 100644
index 00000000000..05b0cb40cd3
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-5.c
@@ -0,0 +1,14 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_gather_load_ifn } */
+
+/* This should be vectorized without gather/scatter. */
+
+void f5 (int *__restrict a, int *__restrict b, unsigned start, unsigned end) {
+ for (unsigned i = start; i != end; i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-tree-dump-not "gather" "vect" } } */
+/* { dg-final { scan-tree-dump "vectorized 1 loops in function" "vect" } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-6.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-6.c
new file mode 100644
index 00000000000..e869e8095ea
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-6.c
@@ -0,0 +1,13 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target vect_int } */
+/* { dg-require-effective-target vect_gather_load_ifn } */
+
+/* This should be vectorized without gather/scatter. */
+
+void f6 (int *__restrict a, int *__restrict b, unsigned start, unsigned end) {
+ for (unsigned i = start; i <= end; i++)
+ a[i] = b[i];
+}
+
+/* { dg-final { scan-tree-dump-not "gather" "vect" } } */
+/* { dg-final { scan-tree-dump "vectorized 1 loops in function" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-7.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-7.c
new file mode 100644
index 00000000000..0d588b6e19e
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-7.c
@@ -0,0 +1,17 @@
+/* { dg-do compile } */
+/* { dg-require-effective-target vect_int } */
+
+/* This should be vectorized without gather/scatter. */
+
+void f7 (int *__restrict dst, unsigned n) {
+ unsigned i = 1;
+ while (i != n)
+ {
+ dst[i] = i;
+ i++;
+ }
+}
+
+/* { dg-final { scan-tree-dump-not "gather" "vect" } } */
+/* { dg-final { scan-tree-dump "vectorized 1 loops in function" "vect" } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-8.c
b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-8.c
new file mode 100644
index 00000000000..cfa66a3807b
--- /dev/null
+++ b/gcc/testsuite/gcc.dg/vect/vect-unsigned-assump-8.c
@@ -0,0 +1,16 @@
+/* { dg-do compile } */
+/* { dg-additional-options "-fno-builtin" } */
+/* { dg-require-effective-target vect_int } */
+
+/* We should vectorize this without additional assumptions.
+ The data-ref uses a pointer-based access which never wraps.
+ However, this is not implemented right now and we always version.
+ Thus, the test is xfailed unconditionally. */
+
+void f8 (int *__restrict a, unsigned start, unsigned n) {
+ for (unsigned i = start; i != n; i++)
+ a[i - start] = 0;
+}
+
+/* { dg-final { scan-tree-dump "Discarding tentative nowrap" "vect" { xfail
*-*-* } } } */
+/* { dg-final { scan-tree-dump "adding no-overflow assumption" "vect" } } */
diff --git a/gcc/tree-ssa-loop-niter.cc b/gcc/tree-ssa-loop-niter.cc
index 207bf8ccf60..bca1909d698 100644
--- a/gcc/tree-ssa-loop-niter.cc
+++ b/gcc/tree-ssa-loop-niter.cc
@@ -1098,6 +1098,14 @@ number_of_iterations_ne (class loop *loop, tree type,
affine_iv *iv,
}
}
+ /* If we know the loop does not wrap, we're done here. */
+ if (loop_constraint_set_p (loop, LOOP_C_NOWRAP))
+ {
+ niter->control.no_overflow = true;
+ niter->niter = fold_build2 (EXACT_DIV_EXPR, niter_type, c, s);
+ return true;
+ }
+
/* Let nsd (step, size of mode) = d. If d does not divide c, the loop
is infinite. Otherwise, the number of iterations is
(inverse(s/d) * (c/d)) mod (size of mode/d). */
@@ -4145,6 +4153,7 @@ record_control_iv (class loop *loop, class
tree_niter_desc *niter)
iv = ggc_alloc<control_iv> ();
iv->base = niter->control.base;
iv->step = niter->control.step;
+ iv->condition = niter->cmp;
iv->next = loop->control_ivs;
loop->control_ivs = iv;
@@ -5490,6 +5499,23 @@ loop_exits_before_overflow (tree base, tree step,
by proving the reverse conditions are false using loop's initial
condition. */
+
+ /* Before proving the above we can try to prove
+ base < civ_base ;; step > 0
+ base > civ_base ;; step < 0
+ (or rather its respective reverse condition).
+ That's a stronger guarantee than
+ civ_base = base + step. */
+ if (TREE_TYPE (expanded_base) == TREE_TYPE (civ->base))
+ {
+ e = fold_build2
+ (tree_int_cst_sign_bit (step) ? LT_EXPR : GT_EXPR,
+ boolean_type_node, expanded_base, civ->base);
+ e = simplify_using_initial_conditions (loop, e);
+ if (integer_zerop (e))
+ return true;
+ }
+
if (POINTER_TYPE_P (TREE_TYPE (base)))
code = POINTER_PLUS_EXPR;
else
@@ -5503,6 +5529,9 @@ loop_exits_before_overflow (tree base, tree step,
{
tree extreme;
+ if (loop_constraint_set_p (loop, LOOP_C_NOWRAP))
+ return true;
+
if (tree_int_cst_sign_bit (step))
{
code = LT_EXPR;
@@ -5519,7 +5548,36 @@ loop_exits_before_overflow (tree base, tree step,
if (integer_zerop (e))
return true;
}
- }
+
+ /* There are also "after stepping" controls like
+
+ {civ_base, step} = {base - step, step}
+
+ where BASE is one step ahead of CIV_BASE.
+ We would need to prove that BASE does not overflow in the last
+ step. That's difficult without knowing the exact number of
+ iterations.
+ What we can do, though, is make use of the loop condition:
+ As the candidate IV is one step ahead of the non-overflowing
+ CIV, the critical point is when CIV reaches BOUND and the next
+ step could overflow.
+ If the loop condition guarantees that we only enter the loop
+ body when CIV != BOUND, we are good. */
+ if (loop_constraint_set_p (loop, LOOP_C_NOWRAP)
+ && (civ->condition == LT_EXPR
+ || civ->condition == GT_EXPR
+ || civ->condition == NE_EXPR)
+ && !POINTER_TYPE_P (TREE_TYPE (base)))
+ {
+ code = MINUS_EXPR;
+ stepped = fold_build2 (code, TREE_TYPE (base), base, step);
+ expanded_stepped = fold_build2 (code, TREE_TYPE (base),
+ expanded_base, step);
+ if (operand_equal_p (stepped, civ->base, 0)
+ || operand_equal_p (expanded_stepped, civ->base, 0))
+ return true;
+ }
+ }
}
return false;
diff --git a/gcc/tree-vect-loop.cc b/gcc/tree-vect-loop.cc
index 7121edb8d81..b86743d2a7d 100644
--- a/gcc/tree-vect-loop.cc
+++ b/gcc/tree-vect-loop.cc
@@ -616,6 +616,75 @@ vect_get_loop_niters (class loop *loop, const_edge
main_exit, tree *assumptions,
continue;
}
+ /* For NE_EXPR number_of_iterations_ne will only add assumptions to
+ ensure the loop is finite. In order to vectorize, we want stricter
+ boundary conditions, i.e. no overflow so we can use regular vector
+ loads/stores rather than gather/scatter.
+ Adding assumptions in number_of_iterations_ne to enforce no overflow
+ might penalize other passes that don't perform versioning and can't
+ use niters with assumptions.
+ Thus, we add the no-overflow condition here after the analysis is
+ done and NITER_DESC's properties are conservatively correct. */
+
+ if (niter_desc.cmp == NE_EXPR
+ && !niter_desc.control.no_overflow
+ && niter_desc.control.base
+ && niter_desc.control.step
+ && niter_desc.bound
+ && TYPE_OVERFLOW_WRAPS (TREE_TYPE (niter_desc.control.base))
+ && TREE_CODE (niter_desc.control.step) == INTEGER_CST
+ && !integer_zerop (niter_desc.control.step)
+ && TREE_CODE (niter_desc.bound) != INTEGER_CST)
+ {
+ tree nowrap;
+ /* For "before stepping" controls, like
+ { base + step, step }
+ that loop-ch creates we don't just need to ensure that
+ base + step <= bound, but also that
+ base + step does not overflow. */
+ tree nowrap_first_step;
+ if (tree_int_cst_sign_bit (niter_desc.control.step))
+ {
+ nowrap = fold_build2 (GE_EXPR, boolean_type_node,
+ niter_desc.control.base, niter_desc.bound);
+ nowrap_first_step
+ = fold_build2 (GE_EXPR, boolean_type_node,
+ niter_desc.control.base,
+ fold_build1 (NEGATE_EXPR, TREE_TYPE (niter),
+ niter_desc.control.step));
+ }
+ else
+ {
+ nowrap = fold_build2 (LE_EXPR, boolean_type_node,
+ niter_desc.control.base, niter_desc.bound);
+ nowrap_first_step
+ = fold_build2 (GE_EXPR, boolean_type_node,
+ fold_build2 (MINUS_EXPR, TREE_TYPE (niter),
+ TYPE_MAX_VALUE (TREE_TYPE (niter)),
+ niter_desc.control.step),
+ niter_desc.control.base);
+ }
+
+ /* Add both assumptions for versioning. */
+ niter_assumptions = fold_build2 (TRUTH_AND_EXPR, boolean_type_node,
+ niter_assumptions,
+ nowrap);
+ niter_assumptions = fold_build2 (TRUTH_AND_EXPR, boolean_type_node,
+ niter_assumptions,
+ nowrap_first_step);
+ if (dump_enabled_p ())
+ dump_printf_loc (MSG_NOTE, vect_location, "adding no-overflow "
+ "assumption.\n");
+
+ /* Mark control IV accordingly. */
+ niter_desc.control.no_overflow = true;
+ }
+
+ /* Set LOOP_C_NOWRAP so we can use it in
+ loop_exits_before_overflow. */
+ if (niter_desc.control.no_overflow)
+ loop_constraint_set (loop, LOOP_C_NOWRAP);
+
/* Loop assumptions are based off the normal exit. */
*assumptions = niter_assumptions;
*number_of_iterationsm1 = niter;
diff --git a/gcc/tree-vectorizer.cc b/gcc/tree-vectorizer.cc
index b73243252c0..f64e3987699 100644
--- a/gcc/tree-vectorizer.cc
+++ b/gcc/tree-vectorizer.cc
@@ -862,6 +862,7 @@ vect_free_loop_info_assumptions (class loop *loop)
loop->any_likely_upper_bound = false;
free_numbers_of_iterations_estimates (loop);
loop_constraint_clear (loop, LOOP_C_FINITE);
+ loop_constraint_clear (loop, LOOP_C_NOWRAP);
}
/* If LOOP has been versioned during ifcvt, return the internal call
@@ -1111,7 +1112,8 @@ try_vectorize_loop_1 (hash_table<simduid_to_vf>
*&simduid_to_vf_htab,
{
/* Free existing information if loop is analyzed with some
assumptions. */
- if (loop_constraint_set_p (loop, LOOP_C_FINITE))
+ if (loop_constraint_set_p (loop, LOOP_C_FINITE)
+ || loop_constraint_set_p (loop, LOOP_C_NOWRAP))
vect_free_loop_info_assumptions (loop);
/* If we applied if-conversion then try to vectorize the
@@ -1176,7 +1178,8 @@ try_vectorize_loop_1 (hash_table<simduid_to_vf>
*&simduid_to_vf_htab,
{
/* Free existing information if loop is analyzed with some
assumptions. */
- if (loop_constraint_set_p (loop, LOOP_C_FINITE))
+ if (loop_constraint_set_p (loop, LOOP_C_FINITE)
+ || loop_constraint_set_p (loop, LOOP_C_NOWRAP))
vect_free_loop_info_assumptions (loop);
return ret;
}
@@ -1431,6 +1434,10 @@ pass_vectorize::execute (function *fun)
vect_slp_fini ();
+ /* LOOP_C_NOWRAP is only valid during vectorizer time, so clear it now. */
+ for (auto loop : loops_list (fun, 0))
+ loop_constraint_clear (loop, LOOP_C_NOWRAP);
+
return ret;
}
--
2.53.0
