From: Kyrylo Tkachov <[email protected]> aarch64_detect_vector_stmt_subtype returns zero for every cost recorded against a PHI. e4180ab2fea added this for loop inductions, whose scalar code "generally needs the same induction but doesn't cost it".
Basic block SLP is different. vect_bb_slp_scalar_cost costs the scalar PHIs, while the zero removes the cost of the vector PHI and of the lane extracts recorded against it. Those extracts are real FMOV and UMOV instructions, and the asymmetry makes a vector PHI look cheaper than the scalar PHIs it replaces. Keep the exemption for loop vectorization and cost PHIs normally for basic block SLP. Bootstrapped and regression tested on aarch64-unknown-linux-gnu and x86_64-pc-linux-gnu. Ok for trunk? Thanks, Kyrill Cc: Tamar Christina <[email protected]> gcc/ChangeLog: * config/aarch64/aarch64.cc (aarch64_detect_vector_stmt_subtype): Only skip costing PHIs for loop vectorization. gcc/testsuite/ChangeLog: * gcc.target/aarch64/slp-phi-cost-1.c: New test. Signed-off-by: Kyrylo Tkachov <[email protected]> --- gcc/config/aarch64/aarch64.cc | 9 +++--- .../gcc.target/aarch64/slp-phi-cost-1.c | 31 +++++++++++++++++++ 2 files changed, 36 insertions(+), 4 deletions(-) create mode 100644 gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c diff --git a/gcc/config/aarch64/aarch64.cc b/gcc/config/aarch64/aarch64.cc index 107271b0404..8e0c45d7bd9 100644 --- a/gcc/config/aarch64/aarch64.cc +++ b/gcc/config/aarch64/aarch64.cc @@ -18288,10 +18288,11 @@ aarch64_detect_vector_stmt_subtype (vec_info *vinfo, vect_cost_for_stmt kind, if (aarch64_sve_mode_p (TYPE_MODE (vectype))) sve_costs = aarch64_tune_params.vec_costs->sve; - /* It's generally better to avoid costing inductions, since the induction - will usually be hidden by other operations. This is particularly true - for things like COND_REDUCTIONS. */ - if (is_a<gphi *> (stmt_info->stmt)) + /* It's generally better to avoid costing loop inductions, since the + induction will usually be hidden by other operations. This is + particularly true for things like COND_REDUCTIONS. For BB SLP, retain + the costs recorded for PHIs and their associated lane extracts. */ + if (is_a<gphi *> (stmt_info->stmt) && is_a<loop_vec_info> (vinfo)) return 0; /* Detect cases in which vec_to_scalar is describing the extraction of a diff --git a/gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c b/gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c new file mode 100644 index 00000000000..18e3f3e89d9 --- /dev/null +++ b/gcc/testsuite/gcc.target/aarch64/slp-phi-cost-1.c @@ -0,0 +1,31 @@ +/* { dg-do compile } */ +/* { dg-require-effective-target lp64 } */ +/* { dg-options "-O2 -mtune=generic -fdump-tree-slp-details" } */ + +/* Basic block SLP costs the scalar PHIs, so it must also cost the vector PHI + and the extracts needed for the scalar call arguments. */ + +extern unsigned long consume (unsigned long, unsigned long); + +unsigned long +foo (unsigned long *out, const unsigned long *x, const unsigned long *y, + int cond) +{ + unsigned long a, b; + if (cond) + { + a = x[0] + 1; + b = x[1] + 1; + } + else + { + a = y[0] ^ 2; + b = y[1] ^ 2; + } + out[0] = a; + out[1] = b; + return consume (a, b); +} + +/* { dg-final { scan-tree-dump {PHI <[^\n]*> 1 times vector_stmt costs 1 in body} "slp2" } } */ +/* { dg-final { scan-tree-dump-times {PHI <[^\n]*> 1 times vec_to_scalar costs 2 in body} 2 "slp2" } } */ -- 2.50.1 (Apple Git-155)
