Ping. https://gcc.gnu.org/pipermail/gcc-patches/2026-September/732877.html Thanks, Kyrill
> On 29 Sep 2026, at 14:54, Kyrylo Tkachov <[email protected]> wrote: > > Ping. > Thanks, > Kyrill > >> On 18 Sep 2026, at 13:11, Kyrylo Tkachov <[email protected]> wrote: >> >> Ping. >> >> https://gcc.gnu.org/pipermail/gcc-patches/2026-June/719927.html >> >> Thanks, >> Kyrill >> >>> On 24 Aug 2026, at 19:01, Kyrylo Tkachov <[email protected]> wrote: >>> >>> Ping. >>> Thanks, >>> Kyrill >>> >>>> On 10 Jun 2026, at 13:37, Kyrylo Tkachov <[email protected]> wrote: >>>> >>>> From: Kyrylo Tkachov <[email protected]> >>>> >>>> eliminated_by_inlining_prob only credited a write to the return value as >>>> free-after-inlining when the RHS was a register or invariant (a scalar >>>> "return x;"). A function that returns an aggregate by value >>>> ("V t = a; ...; return t;") therefore had the copy of the result into the >>>> return slot charged in full to the inliner's growth estimate, even though >>>> return-slot optimization / SRA eliminates that copy once the call is >>>> inlined >>>> (the inlined callee writes the caller's destination in place). >>>> >>>> For a small value-type wrapper -- a SIMD/array lane type, a std::array-like >>>> POD, std::complex, a short Tensor/Matrix -- whose overloaded operators are >>>> written "V op(const V &a, const V &b) { V t = a; t op= b; return t; }", the >>>> mispriced return-slot copy pushes the wrapper over --param >>>> early-inlining-insns, so the early inliner leaves it out of line. Such >>>> operators are called pervasively, and keeping them out of line also forces >>>> their operands and result through memory in the caller. >>>> >>>> Credit aggregate copies whose destination is the return value (a >>>> RESULT_DECL, >>>> or a store through the invisible-reference return pointer) as eliminated by >>>> inlining, mirroring the existing treatment of reads of by-reference >>>> parameters. Statements with volatile operands are excluded: a volatile >>>> access is never elided, so such a copy is not free after inlining. >>>> >>>> This is a size-model refinement only; no IR is changed and no semantics are >>>> affected. >>>> >>>> This gives about 8.5% on an aarch64 machine on the 766.femflow_r >>>> benchmark from SPEC2026 where these routines are critical to performance >>>> and their vectorised form needs to be inlined for good performance. >>>> >>>> Bootstrapped and tested on aarch64-none-linux-gnu. >>>> >>>> Signed-off-by: Kyrylo Tkachov <[email protected]> >>>> >>>> gcc/ChangeLog: >>>> >>>> * ipa-fnsummary.cc (eliminated_by_inlining_prob): Credit aggregate >>>> copies into the return slot (a RESULT_DECL or invisible-reference >>>> return) as eliminated by inlining. >>>> >>>> gcc/testsuite/ChangeLog: >>>> >>>> * g++.dg/ipa/inline-aggregate-retval.C: New test. >>>> * gcc.dg/ipa/inline-aggregate-retval.c: New test. >>>> --- >>>> gcc/ipa-fnsummary.cc | 27 ++++++++++++++++ >>>> .../g++.dg/ipa/inline-aggregate-retval.C | 21 +++++++++++++ >>>> .../gcc.dg/ipa/inline-aggregate-retval.c | 31 +++++++++++++++++++ >>>> 3 files changed, 79 insertions(+) >>>> create mode 100644 gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C >>>> create mode 100644 gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c >>>> >>>> diff --git a/gcc/ipa-fnsummary.cc b/gcc/ipa-fnsummary.cc >>>> index a6ccdb1852a..c05ffc313ed 100644 >>>> --- a/gcc/ipa-fnsummary.cc >>>> +++ b/gcc/ipa-fnsummary.cc >>>> @@ -1476,6 +1476,33 @@ eliminated_by_inlining_prob (ipa_func_body_info >>>> *fbi, gimple *stmt) >>>> if (lhs_free >>>> && (is_gimple_reg (rhs) || is_gimple_min_invariant (rhs))) >>>> rhs_free = true; >>>> + /* A by-value aggregate return ("T t = ...; return t;") stores the >>>> + result into the return slot. Under the return-slot optimization the >>>> + inlined callee writes the caller's destination in place and SRA then >>>> + scalarises the small aggregate, so the store is eliminated by >>>> inlining >>>> + -- the return-value analogue of the by-reference parameter reads >>>> + credited above. The is_gimple_reg/is_gimple_min_invariant test only >>>> + credits scalar returns; also credit an aggregate return whose source >>>> is >>>> + an ordinary memory value. This is the value-type wrapper idiom >>>> + (a SIMD/array lane type, std::array, std::complex, a short Tensor): >>>> + "T operator OP (const T &a, const T &b) { T t = a; t OP= b; return >>>> t; }". >>>> + This is the whole-aggregate-copy case. */ >>>> + if (!rhs_free >>>> + && !gimple_has_volatile_ops (stmt) >>>> + && AGGREGATE_TYPE_P (TREE_TYPE (lhs)) >>>> + && gimple_assign_rhs_class (stmt) == GIMPLE_SINGLE_RHS >>>> + && (TREE_CODE (inner_lhs) == RESULT_DECL >>>> + || (TREE_CODE (inner_lhs) == MEM_REF >>>> + && TREE_CODE (TREE_OPERAND (inner_lhs, 0)) == SSA_NAME >>>> + && SSA_NAME_VAR (TREE_OPERAND (inner_lhs, 0)) >>>> + && TREE_CODE (SSA_NAME_VAR (TREE_OPERAND (inner_lhs, 0))) >>>> + == RESULT_DECL)) >>>> + && (TREE_CODE (inner_rhs) == MEM_REF >>>> + || TREE_CODE (inner_rhs) == COMPONENT_REF >>>> + || VAR_P (inner_rhs) >>>> + || TREE_CODE (inner_rhs) == PARM_DECL >>>> + || TREE_CODE (inner_rhs) == RESULT_DECL)) >>>> + return 2; >>>> if (lhs_free && rhs_free) >>>> return 1; >>>> } >>>> diff --git a/gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C >>>> b/gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C >>>> new file mode 100644 >>>> index 00000000000..a60bad00b66 >>>> --- /dev/null >>>> +++ b/gcc/testsuite/g++.dg/ipa/inline-aggregate-retval.C >>>> @@ -0,0 +1,21 @@ >>>> +// The value-type operator-overload idiom: a by-value 'operator*' returns >>>> its >>>> +// result aggregate, and the copy of that result into the return slot is >>>> +// eliminated by inlining (return-slot optimization / SRA). It used to be >>>> +// charged to the early-inliner growth, pushing the operator over >>>> +// --param early-inlining-insns. -fno-inline-functions isolates the early >>>> +// inliner so the operator is inlined only when the copy is credited. >>>> +// { dg-do compile } >>>> +// { dg-options "-O3 --param early-inlining-insns=14 >>>> -fno-inline-functions -fno-partial-inlining -fdump-tree-einline-optimized" >>>> } >>>> + >>>> +struct V { >>>> + double d[8]; >>>> + V &operator*= (const V &o) { for (int i = 0; i < 8; i++) d[i] *= >>>> o.d[i]; return *this; } >>>> +}; >>>> + >>>> +static V operator* (const V &a, const V &b) { V t = a; return t *= b; } >>>> + >>>> +V ga, gb, gc, r; >>>> + >>>> +void use () { r = ga * gb * gc; } >>>> + >>>> +// { dg-final { scan-tree-dump "Inlining.*operator.*into.*use" "einline" >>>> } } >>>> diff --git a/gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c >>>> b/gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c >>>> new file mode 100644 >>>> index 00000000000..e21cff3438f >>>> --- /dev/null >>>> +++ b/gcc/testsuite/gcc.dg/ipa/inline-aggregate-retval.c >>>> @@ -0,0 +1,31 @@ >>>> +/* A small by-value wrapper that returns its aggregate result by value. >>>> The >>>> + copy of the result into the return slot is eliminated by inlining >>>> + (return-slot optimization / SRA combines it with the caller's >>>> destination), >>>> + but it used to be charged in full to the early-inliner growth estimate, >>>> + pushing the wrapper over --param early-inlining-insns and leaving it >>>> out of >>>> + line. -fno-inline-functions isolates the early inliner, so 'mul' can >>>> only >>>> + be inlined when the return-slot copy is correctly credited as >>>> eliminable. */ >>>> +/* { dg-do compile } */ >>>> +/* { dg-options "-O3 --param early-inlining-insns=14 >>>> -fno-inline-functions -fno-partial-inlining -fdump-tree-einline-optimized" >>>> } */ >>>> + >>>> +struct V { double d[8]; }; >>>> + >>>> +static struct V >>>> +mul (const struct V *a, const struct V *b) >>>> +{ >>>> + struct V t = *a; >>>> + for (int i = 0; i < 8; i++) >>>> + t.d[i] *= b->d[i]; >>>> + return t; >>>> +} >>>> + >>>> +struct V ga, gb, gc, r; >>>> + >>>> +void >>>> +use (void) >>>> +{ >>>> + struct V x = mul (&ga, &gb); >>>> + r = mul (&x, &gc); >>>> +} >>>> + >>>> +/* { dg-final { scan-tree-dump "Inlining mul.* into use" "einline" } } */ >>>> -- >>>> 2.50.1 (Apple Git-155) >>>> >>> >> >
