Hello All, Thank you all for the reviews and feedback for the V2 series.
I have attempted to addressed this feedback in the new series. Major changes: -------------- 1. HSSR regions are now stored in a vec in vect_loop_masks and their lifetimes are tied to the vect_loop_masks struct. (as suggested by Robin). This makes the API and accounting for these signifficantly cleaner. Additionally, it makes the lifetimes much easier to reason about. It also makes the API less messy if the SLP trees were to be reused for epilogue vectorization (not currently supported), as only the HSSR regions would need to be recreated, but the analysis of HSSR regions could be reused. 2. aarch64 patches separated from target independent changes. 3. general code improvements and refactors. 4. The patch series now depends both on Tamars patch series adding count_active and my patch series for aarch64 to allow --param=aaarch64-autovec-preference to force vectorization with a specific mode. Bootstrapped, O3+LTO bootstrapped, and regression tested on aarch64, x86_64, and armhf. Spec v6 and v8 both built and run successfully with this series for aarch64 with and without --param=vect-hssr-usage=3. Thoughts and feedback? Okay for trunk? :) Many thanks, Alfie Alfie Richards (17): Add count active fix: vect: Consider IV increment being invariant for loop bounds vect: Change IV increment position to standard_iv_increment_position ifn: optab: Add HSSR ifns and optabs vect: Make vect_maybe_permute_loop_masks optional and retargetable vect: Add vect-hssr-usage param and testsuite target checks Add hssr_read entry to vect_cost_for_stmt vect: Add HSSR analysis vect: Add HSSR transformation vect: Add HSSR tests vect: Add HSSR versioning aarch64: Add read_ffr_cost item to cost model aarch64: Make reads of FFR expensive aarch64: Add fold_contiguous_load helper function. aarch64: Implement HSSR for aarch64 aarch64: Add HSSR tests vect: Enable HSSR by default. gcc/config/aarch64/aarch64-acle-builtins.cc | 24 + gcc/config/aarch64/aarch64-acle-builtins.h | 1 + gcc/config/aarch64/aarch64-protos.h | 10 +- .../aarch64/aarch64-sve-builtins-base.cc | 99 ++-- gcc/config/aarch64/aarch64-sve.md | 260 ++++++++- gcc/config/aarch64/aarch64.cc | 28 +- gcc/config/aarch64/aarch64.md | 2 +- gcc/config/aarch64/neoversev2.md | 4 +- gcc/config/aarch64/olympus.md | 4 +- gcc/config/aarch64/tuning_models/a64fx.h | 3 +- gcc/config/aarch64/tuning_models/cortexa320.h | 3 +- gcc/config/aarch64/tuning_models/cortexx925.h | 3 +- gcc/config/aarch64/tuning_models/generic.h | 3 +- .../aarch64/tuning_models/generic_armv8_a.h | 3 +- .../aarch64/tuning_models/generic_armv9_a.h | 3 +- gcc/config/aarch64/tuning_models/hip12.h | 3 +- .../aarch64/tuning_models/neoverse512tvb.h | 3 +- gcc/config/aarch64/tuning_models/neoversen2.h | 3 +- gcc/config/aarch64/tuning_models/neoversen3.h | 3 +- gcc/config/aarch64/tuning_models/neoversev1.h | 3 +- gcc/config/aarch64/tuning_models/neoversev2.h | 3 +- gcc/config/aarch64/tuning_models/neoversev3.h | 3 +- .../aarch64/tuning_models/neoversev3ae.h | 3 +- gcc/config/aarch64/tuning_models/olympus.h | 3 +- gcc/doc/ifn.texi | 29 + gcc/doc/md.texi | 42 ++ gcc/gimple-fold.cc | 9 + gcc/internal-fn.cc | 39 ++ gcc/internal-fn.def | 20 + gcc/internal-fn.h | 1 + gcc/optabs.def | 11 + gcc/params.opt | 4 + gcc/target.h | 7 +- gcc/targhooks.cc | 3 + gcc/testsuite/gcc.dg/vect/hssr-1.c | 124 ++++ gcc/testsuite/gcc.dg/vect/hssr-2.c | 34 ++ gcc/testsuite/gcc.dg/vect/hssr-3.c | 25 + gcc/testsuite/gcc.dg/vect/hssr-4.c | 25 + gcc/testsuite/gcc.dg/vect/hssr-5.c | 19 + gcc/testsuite/gcc.dg/vect/hssr-6.c | 17 + gcc/testsuite/gcc.dg/vect/hssr-ice-1.c | 12 + gcc/testsuite/gcc.dg/vect/hssr-ice-2.c | 16 + gcc/testsuite/gcc.dg/vect/hssr-ice-3.c | 14 + gcc/testsuite/gcc.dg/vect/hssr-ice-4.c | 17 + gcc/testsuite/gcc.target/aarch64/sve/ffr_1.c | 16 + gcc/testsuite/gcc.target/aarch64/sve/ffr_2.c | 44 ++ gcc/testsuite/gcc.target/aarch64/sve/ffr_3.c | 21 + gcc/testsuite/gcc.target/aarch64/sve/ffr_4.c | 21 + .../gcc.target/aarch64/sve/noeffect11.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_1.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_11.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_12.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_13.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_14.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_15.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_16.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_17.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_2.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_3.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_4.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_5.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_6.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_7.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_8.c | 2 +- .../gcc.target/aarch64/sve/peel_ind_9.c | 2 +- gcc/testsuite/gcc.target/aarch64/sve/peeled.c | 2 +- .../gcc.target/aarch64/sve/peeled1.c | 2 +- .../gcc.target/aarch64/sve/peeled2.c | 2 +- .../gcc.target/aarch64/sve/peeled3.c | 2 +- .../gcc.target/aarch64/sve/pr119351.c | 2 +- gcc/testsuite/lib/target-supports.exp | 41 ++ gcc/tree-data-ref.cc | 2 + gcc/tree-ssa-alias.cc | 1 + gcc/tree-ssa-loop-ivopts.cc | 2 + gcc/tree-vect-data-refs.cc | 6 +- gcc/tree-vect-loop-manip.cc | 530 +++++++++++++++++- gcc/tree-vect-loop.cc | 475 +++++++++++++++- gcc/tree-vect-slp.cc | 31 + gcc/tree-vect-stmts.cc | 303 ++++++++-- gcc/tree-vectorizer.cc | 51 +- gcc/tree-vectorizer.h | 118 +++- 81 files changed, 2399 insertions(+), 252 deletions(-) create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-1.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-2.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-3.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-4.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-5.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-6.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-ice-1.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-ice-2.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-ice-3.c create mode 100644 gcc/testsuite/gcc.dg/vect/hssr-ice-4.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_1.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_2.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_3.c create mode 100644 gcc/testsuite/gcc.target/aarch64/sve/ffr_4.c -- 2.43.0
