[Bug tree-optimization/85934] [8/9 Regression] ICE: verify_gimple failed (error: type mismatch in vector pack expression)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85934

--- Comment #3 from Richard Biener  ---
Author: rguenth
Date: Mon May 28 10:28:38 2018
New Revision: 260847

URL: https://gcc.gnu.org/viewcvs?rev=260847=gcc=rev
Log:
2018-05-28  Richard Biener  

PR tree-optimization/85934
* tree-vect-generic.c (expand_vector_operations_1): Hoist
vector boolean check before scalar optimization.

* gcc.target/i386/pr85934.c: New testcase.

Added:
trunk/gcc/testsuite/gcc.target/i386/pr85934.c
Modified:
trunk/gcc/testsuite/ChangeLog
trunk/gcc/tree-vect-generic.c

[Bug tree-optimization/85934] [8/9 Regression] ICE: verify_gimple failed (error: type mismatch in vector pack expression)

[rguenth at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85934

Richard Biener  changed:

   What|Removed |Added

   Priority|P3  |P2
 Status|NEW |ASSIGNED
   Assignee|unassigned at gcc dot gnu.org  |rguenth at gcc dot 
gnu.org

--- Comment #2 from Richard Biener  ---
The issue is that vector lowering does anything to the vectorized code, that
is, a disconnect between what the vectorizer thinks the target can do and what
vector lowering thinks.  And then some bug in vector lowering generating sth
"stupid"
for mask vector lowerings.  Ah, it runs into

  /* If the vector operation is operating on all same vector elements
 implement it with a scalar operation and a splat if the target
 supports the scalar operation.  */
  tree srhs1, srhs2 = NULL_TREE;
  if ((srhs1 = ssa_uniform_vector_p (rhs1)) != NULL_TREE
  && (rhs2 == NULL_TREE
  || (! VECTOR_TYPE_P (TREE_TYPE (rhs2))
  && (srhs2 = rhs2))
  || (srhs2 = ssa_uniform_vector_p (rhs2)) != NULL_TREE)
  /* As we query direct optabs restrict to non-convert operations.  */
  && TYPE_MODE (TREE_TYPE (type)) == TYPE_MODE (TREE_TYPE (srhs1)))
{
  op = optab_for_tree_code (code, TREE_TYPE (type), optab_scalar);
  if (op >= FIRST_NORM_OPTAB && op <= LAST_NORM_OPTAB
  && optab_handler (op, TYPE_MODE (TREE_TYPE (type))) !=
CODE_FOR_nothing)
{

for vector boolean operations to optimize ~{_88,_88...} to {~_88,~_88...}.  And
then it ends up doing this to

mask_patt_63.10_95 = VEC_PACK_TRUNC_EXPR ;

but of course this trick doesn't work for that and the optab query is
"confused"
here because the vector mode is QImode and the component mode is QImode as
well.

The easiest fix is to hoist the VECTOR_BOOLEAN_TYPE_P check above this
transform.

[Bug tree-optimization/85934] [8/9 Regression] ICE: verify_gimple failed (error: type mismatch in vector pack expression)

[marxin at gcc dot gnu.org]

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=85934

Martin Liška  changed:

   What|Removed |Added

 Status|UNCONFIRMED |NEW
   Last reconfirmed||2018-05-28
 CC||marxin at gcc dot gnu.org,
   ||rguenth at gcc dot gnu.org
   Target Milestone|--- |8.2
 Ever confirmed|0   |1
  Known to fail||8.1.0, 9.0

--- Comment #1 from Martin Liška  ---
Confirmed, started with r247048.