From: Ezra Sitorus <[email protected]>
This patch series attempts to recognise this pattern:
struct cint16_t
{
int8_t real;
int8_t imag;
};
int32_t
cdot (cint16_t *a, cint16_t *b, uint32_t N)
{
int32_t res = 0;
for (uint32_t i = 0; i < N; i++)
res += (a[i].real * b[i].real) - (a[i].imag * b[i].imag);
return res;
}
and emit something like this (in AArch64 with SVE2):
cdot acc, a, b, #0
The process for doing this is like so:
1) Split up using match.pd
2) Recognise it as a dot_prod and neg_dot_prod vect patterns. We end up
with something like this:
acc1 = DOT_PROD_EXPR (a_re, b_re, acc); // dot_prod pattern
acc2 = 0 - acc1; // neg_dot_prod pattern
acc3 = DOT_PROD_EXPR (a_im, b_im, acc2);
acc4 = 0 - acc3;
3) Recognise the pattern above and replace with IFN to emit cdot
instruction.
AArch64's cdot instruction has a rotation operand which rotates the
second source vector by 0, 90, 180 or 270 degrees. A rotation of means
the destination vector accumulates the real parts of a complex dot
product. For now, only the 0 rotation is supported in this patch
series.
This is my first time working with vectorizer/SLP so I'd appreciate any
pointers on what can be done better!
Bootstrapped and regression tested on AArch64 and x86_64.
Ezra Sitorus (5):
match.pd: (a*b - c*d) + acc -> (acc + a*b) - c*d
vect: Add vect_recog_neg_dot_prod_pattern
gcc: Add new COMPLEX_DOT_PROD_ROT0 IFN/optabs
gcc: make COMPLEX_DOT_PROD_ROT0 lane_reducing
gcc: add complex dot prod SLP pattern
gcc/config/aarch64/aarch64-sve2.md | 10 +
gcc/internal-fn.cc | 5 +
gcc/internal-fn.def | 1 +
gcc/match.pd | 8 +
gcc/optabs.def | 1 +
.../gcc.dg/tree-ssa/neg-dot-prod-1.c | 19 +
.../vect/complex/bb-slp-complex-dot-int8.c | 25 ++
.../gcc.dg/vect/vect-reduc-neg-dot-1.c | 46 +++
.../gcc.dg/vect/vect-reduc-neg-dot-2.c | 45 +++
gcc/tree-vect-loop.cc | 10 +-
gcc/tree-vect-patterns.cc | 124 ++++++
gcc/tree-vect-slp-patterns.cc | 358 ++++++++++++++++++
gcc/tree-vectorizer.h | 18 +-
13 files changed, 665 insertions(+), 5 deletions(-)
create mode 100644 gcc/testsuite/gcc.dg/tree-ssa/neg-dot-prod-1.c
create mode 100644 gcc/testsuite/gcc.dg/vect/complex/bb-slp-complex-dot-int8.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-neg-dot-1.c
create mode 100644 gcc/testsuite/gcc.dg/vect/vect-reduc-neg-dot-2.c
--
2.53.0
