> -----Original Message-----
> From: Richard Biener <[email protected]>
> Sent: 15 September 2026 10:16
> To: Tamar Christina <[email protected]>
> Cc: Ezra Sitorus <[email protected]>; [email protected];
> [email protected]
> Subject: RE: [RFC PATCH 3/5] gcc: Add new COMPLEX_DOT_PROD_ROT0
> IFN/optabs
>
> On Tue, 15 Sep 2026, Tamar Christina wrote:
>
> > > -----Original Message-----
> > > From: Richard Biener <[email protected]>
> > > Sent: 15 September 2026 09:23
> > > To: Tamar Christina <[email protected]>
> > > Cc: Ezra Sitorus <[email protected]>; [email protected];
> > > [email protected]
> > > Subject: RE: [RFC PATCH 3/5] gcc: Add new COMPLEX_DOT_PROD_ROT0
> > > IFN/optabs
> > >
> > > On Tue, 15 Sep 2026, Tamar Christina wrote:
> > >
> > > >
> > > >
> > > > > -----Original Message-----
> > > > > From: [email protected] <[email protected]>
> > > > > Sent: 09 September 2026 19:12
> > > > > To: [email protected]
> > > > > Cc: Tamar Christina <[email protected]>; [email protected];
> > > > > [email protected]; Ezra Sitorus
> <[email protected]>
> > > > > Subject: [RFC PATCH 3/5] gcc: Add new COMPLEX_DOT_PROD_ROT0
> > > > > IFN/optabs
> > > > >
> > > > > From: Ezra Sitorus <[email protected]>
> > > > >
> > > > > Add new internal function COMPLEX_DOT_PROD_ROT0 and optab
> > > > > cdot_prod_rot0_optab
> > > > > to represent complex integer dot product.
> > > > >
> > > > > COMPLEX_DOT_PROD_ROT0 (A, B, ACC) is a lane-reducing signed
> integer
> > > > > complex
> > > > > dot product. The 2 input vectors contain interleaved W-bit real and
> > > imaginary
> > > > > components in even/odd numbered lanes respectively. Each 4W-bit
> result
> > > > > element
> > > > > accumulates the 0-rotation products of the 2 complex input pairs in
> > > > > the
> > > > > corresponding group of 4 input lanes:
> > > > >
> > > > > result[j] = acc[j]
> > > > > + a[k].re * b[k].re
> > > > > - a[k].im * b[k].im
> > > > > + a[k + 1].re * b[k + 1].re
> > > > > - a[k + 1].im * b[k + 1].im
> > > > >
> > > > > where 2*j = k.
> > > > >
> > > > > 0-rotation means that the imaginary components in the second source
> > > vector
> > > > > are
> > > > > negated, and so the overall operation is accumulating the real part of
> the
> > > > > complex dot product. Other rotations can be added in future patches.
> > >
> > > I other places we use _conj, not rot0:
> > >
> > > OPTAB_D (cmul_conj_optab, "cmul_conj$a3")
> > >
> > > so why not use that for consistency?
> >
> > That's a different operation. complex dotprod is numerically
>
> Sure, I meant why cdot_prod_rot0 rather than cdot_prod_conj,
> complex conjugate is exactly negating the imaginary part?
Because the existing cmul/cadd/cdot *_conj optabs are intended to
represent the full mathematical complex operation which create both
the real and imaginary results.
Yes, cdot #0 conjugates the second operand in the sense that it uses
ar * br - ai * bi. But it only computes the real component of the
complex dot product. The imaginary component is not computed.
For example, cmul_conj today lowers to both:
fcmla #180
fcmla #270
because both operations are needed to compute the full complex multiply
by a conjugated operand.
Similarly, computing both components of a complex dot product would need
Two instructions and rotations:
cdot #0
cdot #90
So naming the single cdot #0 operation cdot_prod_conj would be
misleading: it suggests the optab computes the full conjugated complex
dot product, whereas it only computes one rotated component.
This is the same distinction as for complex multiply rotations, discussed
in PR121925, where we settled on the "rot" suffix for the lane-rotation forms.
This isn't dissimilar to complex multiply and rotate which we discussed here
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=121925 and settled on the "rot"
suffix for them.
Thanks,
Tamar
>
> > #include <stdint.h>
> >
> > typedef struct {
> > int8_t real;
> > int8_t imag;
> > } cint16_t;
> >
> > typedef struct {
> > int32_t real;
> > int32_t imag;
> > } cint32_t;
> >
> > cint32_t cdot(cint16_t *a, cint16_t *b, uint32_t N)
> > {
> > cint32_t res = {0, 0};
> >
> > for (uint32_t i = 0; i < N; i++) {
> > res.real += a[i].real * b[i].real
> > - a[i].imag * b[i].imag;
> >
> > res.imag += a[i].real * b[i].imag
> > + a[i].imag * b[i].real;
> > }
> >
> > return res;
> > }
> >
> > Which would be what we'd call cdot.
> >
> > cdot_conj would be
> >
> > #include <stdint.h>
> >
> > typedef struct {
> > int8_t real;
> > int8_t imag;
> > } cint16_t;
> >
> > typedef struct {
> > int32_t real;
> > int32_t imag;
> > } cint32_t;
> >
> > cint32_t cdot_conj(const cint16_t *a,
> > const cint16_t *b,
> > uint32_t N)
> > {
> > cint32_t res = {0, 0};
> >
> > for (uint32_t i = 0; i < N; i++) {
> > res.real += a[i].real * b[i].real
> > + a[i].imag * b[i].imag;
> >
> > res.imag += a[i].real * b[i].imag
> > - a[i].imag * b[i].real;
> > }
> >
> > return res;
> > }
> >
> > i.e. normal dual lane operations.
> >
> > however this patch recognizes the single lane reduction,
> >
> > #include <stdint.h>
> >
> > typedef struct {
> > int8_t real;
> > int8_t imag;
> > } cint16_t;
> >
> > int32_t cdot0(const cint16_t *a, const cint16_t *b, uint32_t N)
> > {
> > int32_t res = 0;
> >
> > for (uint32_t i = 0; i < N; i++)
> > res += a[i].real * b[i].real
> > - a[i].imag * b[i].imag;
> >
> > return res;
> > }
> >
> > Which is not numerically a complex dot product. Note that the end goal
> > is to recognize all of these.
> >
> > Thanks,
> > Tamar
> > >
> > > > > In internal-fn.cc: complex_dot_prod_direct is defined to { -1, 1,
> > > > > false }
> > > > > because it uses the return type as the accumulator/result type and the
> > > input
> > > > > type is either of the narrower source type.
> > > > >
> > > > > In AArch64, this corresponds to the CDOT instruction. This patch also
> > > teaches
> > > > > GCC to convert the optab into the correct define_insn pattern.
> > > > >
> > > > > gcc/ChangeLog:
> > > > > * config/aarch64/aarch64-sve2.md
> > > > > (cdot_prod_rot0<mode><vsi2qi>): New
> > > > > define_expand.
> > > > > * internal-fn.cc (complex_dot_prod_direct): New static
> > > > > initializers.
> > > > > * internal-fn.def (COMPLEX_DOT_PROD_ROT0) New internal
> > > > > function.
> > > > > * optabs.def (cdot_prod_rot0_optab): New optab.
> > > > > ---
> > > > > gcc/config/aarch64/aarch64-sve2.md | 10 ++++++++++
> > > > > gcc/internal-fn.cc | 5 +++++
> > > > > gcc/internal-fn.def | 1 +
> > > > > gcc/optabs.def | 1 +
> > > >
> > > > The new optabs and IFN need to be documented in ifn.texi and md.texi
> > > > and split the aarch64 implementation to it's own patch.
> > > >
> > > > But why not add the other rotations at the same time?
> > > >
> > > > We'd likely also later (in a different patch) need cdot_prod and
> > > cdot_prod_conj,
> > > > because then both lanes are visible the SLP tree will look much
> > > > different.
> i.e.
> > > >
> > > > #include <stdint.h>
> > > >
> > > > typedef _Complex int8_t cint16_t;
> > > > typedef _Complex int32_t cint32_t;
> > > >
> > > > cint32_t cdot(cint16_t *a, cint16_t *b, uint32_t N)
> > > > {
> > > > cint32_t res = 0;
> > > >
> > > > for (uint32_t i = 0; i < N; i++)
> > > > res += a[i] * b[i];
> > > >
> > > > return res;
> > > > }
> > > >
> > > > > 4 files changed, 17 insertions(+)
> > > > >
> > > > > diff --git a/gcc/config/aarch64/aarch64-sve2.md
> > > > > b/gcc/config/aarch64/aarch64-sve2.md
> > > > > index c33df283cc2a..ad4febb60bf8 100644
> > > > > --- a/gcc/config/aarch64/aarch64-sve2.md
> > > > > +++ b/gcc/config/aarch64/aarch64-sve2.md
> > > > > @@ -3578,6 +3578,16 @@
> > > > > ;; - CDOT
> > > > > ;;
> > > > > -------------------------------------------------------------------------
> > > > >
> > > > > +(define_expand "cdot_prod_rot0<mode><vsi2qi>"
> > > > > + [(set (match_operand:SVE_FULL_SDI 0 "register_operand")
> > > > > + (unspec:SVE_FULL_SDI
> > > > > + [(match_operand:SVE_FULL_SDI 3 "register_operand")
> > > > > + (match_operand:<VSI2QI> 1 "register_operand")
> > > > > + (match_operand:<VSI2QI> 2 "register_operand")]
> > > > > + UNSPEC_CDOT))]
> > > > > + "TARGET_SVE2"
> > > > > +)
> > > > > +
> > > > > (define_insn "@aarch64_sve_<optab><mode>"
> > > > > [(set (match_operand:SVE_FULL_SDI 0 "register_operand")
> > > > > (unspec:SVE_FULL_SDI
> > > > > diff --git a/gcc/internal-fn.cc b/gcc/internal-fn.cc
> > > > > index c57b594ac591..cc968109261b 100644
> > > > > --- a/gcc/internal-fn.cc
> > > > > +++ b/gcc/internal-fn.cc
> > > > > @@ -180,6 +180,7 @@ init_internal_fns ()
> > > > > #define unary_convert_direct { -1, 0, true }
> > > > > #define binary_direct { 0, 0, true }
> > > > > #define ternary_direct { 0, 0, true }
> > > > > +#define complex_dot_prod_direct { -1, 1, false }
> > > > > #define cond_unary_direct { 1, 1, true }
> > > > > #define cond_binary_direct { 1, 1, true }
> > > > > #define cond_ternary_direct { 1, 1, true }
> > > > > @@ -4315,6 +4316,9 @@ expand_reduc_sbool_optab_fn (internal_fn
> fn,
> > > > > gcall *stmt, direct_optab optab)
> > > > > #define expand_select_vl_optab_fn(FN, STMT, OPTAB) \
> > > > > expand_convert_optab_fn (FN, STMT, OPTAB, 3)
> > > > >
> > > > > +#define expand_complex_dot_prod_optab_fn(FN, STMT, OPTAB) \
> > > > > + expand_convert_optab_fn (FN, STMT, OPTAB, 3)
> > > > > +
> > > > > /* Expanders for optabs that can use expand_convert_optab_fn. */
> > > > >
> > > > > #define expand_unary_convert_optab_fn(FN, STMT, OPTAB) \
> > > > > @@ -4398,6 +4402,7 @@ multi_vector_optab_supported_p
> > > (convert_optab
> > > > > optab, tree_pair types,
> > > > > #define direct_unary_convert_optab_supported_p
> > > > > convert_optab_supported_p
> > > > > #define direct_binary_optab_supported_p direct_optab_supported_p
> > > > > #define direct_ternary_optab_supported_p direct_optab_supported_p
> > > > > +#define direct_complex_dot_prod_optab_supported_p
> > > > > convert_optab_supported_p
> > > > > #define direct_cond_unary_optab_supported_p
> > > direct_optab_supported_p
> > > > > #define direct_cond_binary_optab_supported_p
> > > direct_optab_supported_p
> > > > > #define direct_cond_ternary_optab_supported_p
> > > direct_optab_supported_p
> > > > > diff --git a/gcc/internal-fn.def b/gcc/internal-fn.def
> > > > > index af9f92950c73..08bdedc30045 100644
> > > > > --- a/gcc/internal-fn.def
> > > > > +++ b/gcc/internal-fn.def
> > > > > @@ -453,6 +453,7 @@ DEF_INTERNAL_OPTAB_FN
> > > (COMPLEX_ADD_ROT90,
> > > > > ECF_CONST, cadd90, binary)
> > > > > DEF_INTERNAL_OPTAB_FN (COMPLEX_ADD_ROT270, ECF_CONST,
> > > cadd270,
> > > > > binary)
> > > > > DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL, ECF_CONST, cmul,
> binary)
> > > > > DEF_INTERNAL_OPTAB_FN (COMPLEX_MUL_CONJ, ECF_CONST,
> > > cmul_conj,
> > > > > binary)
> > > > > +DEF_INTERNAL_OPTAB_FN (COMPLEX_DOT_PROD_ROT0,
> ECF_CONST,
> > > > > cdot_prod_rot0, complex_dot_prod)
> > > > > DEF_INTERNAL_OPTAB_FN (VEC_ADDSUB, ECF_CONST, vec_addsub,
> > > binary)
> > > > > DEF_INTERNAL_OPTAB_FN (VEC_TRUNC_ADD_HIGH, ECF_CONST |
> > > > > ECF_NOTHROW,
> > > > > vec_trunc_add_high, binary)
> > > > > diff --git a/gcc/optabs.def b/gcc/optabs.def
> > > > > index 9d95f047cb50..b30e2bec6b2b 100644
> > > > > --- a/gcc/optabs.def
> > > > > +++ b/gcc/optabs.def
> > > > > @@ -114,6 +114,7 @@ OPTAB_CD(vec_init_optab, "vec_init$a$b")
> > > > > OPTAB_CD (sdot_prod_optab, "sdot_prod$I$a$b")
> > > > > OPTAB_CD (udot_prod_optab, "udot_prod$I$a$b")
> > > > > OPTAB_CD (usdot_prod_optab, "usdot_prod$I$a$b")
> > > > > +OPTAB_CD (cdot_prod_rot0_optab, "cdot_prod_rot0$I$a$b")
> > > > >
> > > > > OPTAB_CD (while_ult_optab, "while_ult$a$b")
> > > > > OPTAB_CD (select_vl_optab, "select_vl$a$b")
> > > > > --
> > > > > 2.53.0
> > > >
> > > >
> > >
> > > --
> > > Richard Biener <[email protected]>
> > > SUSE Software Solutions Germany GmbH,
> > > Frankenstrasse 146, 90461 Nuernberg, Germany;
> > > GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri; (HRB 36809, AG Nuernberg)
> >
>
> --
> Richard Biener <[email protected]>
> SUSE Software Solutions Germany GmbH,
> Frankenstrasse 146, 90461 Nuernberg, Germany;
> GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri; (HRB 36809, AG Nuernberg)
