> On 1 Oct 2026, at 16:04, Richard Biener <[email protected]> wrote:
>
> On Thu, 17 Sep 2026, [email protected] wrote:
>
>> From: Kyrylo Tkachov <[email protected]>
>>
>> This a respin after Richard's feedback, with an attempt to improve the
>> locality of the splitted code.
>>
>> genmatch writes one function for each outer operation. Output partitioning
>> places those functions whole, so one operation matcher can dominate an output
>> part: on trunk gimple_simplify_COND_EXPR is 58429 lines, and the largest
>> GIMPLE part is 2.417 times the smallest. Compiling it with profile
>> generation for Armv8-A Thumb also used to fail to assemble due to branch out
>> of
>> range assembly errors. That is a separate arm bug that can be fixed
>> elsewhere.
>>
>> Split a large per-operation matcher between its top-level alternatives. The
>> entry function keeps the dispatch and calls the function that holds the group
>> it selected, so a match enters one group rather than walking all of them.
>>
>> Before, one function holds the dispatch and every alternative it reaches:
>>
>> gimple_simplify_COND_EXPR (gimple_match_op *res_op, ...)
>> {
>> switch (TREE_CODE (_p0))
>> {
>> case SSA_NAME:
>> if (gimple *_d1 = get_def (valueize, _p0))
>> if (gassign *_a1 = dyn_cast <gassign *> (_d1))
>> switch (gimple_assign_rhs_code (_a1))
>> {
>> case LT_EXPR:
>> {
>> tree _q20 = gimple_assign_rhs1 (_a1);
>> ... <- 58429 lines in all
>> }
>> case NE_EXPR:
>> { ... }
>>
>> After, the entry holds the dispatch and nothing else:
>>
>> gimple_simplify_COND_EXPR (gimple_match_op *res_op, ...)
>> {
>> switch (TREE_CODE (_p0))
>> {
>> case SSA_NAME:
>> if (gimple *_d1 = get_def (valueize, _p0))
>> if (gassign *_a1 = dyn_cast <gassign *> (_d1))
>> switch (gimple_assign_rhs_code (_a1))
>> {
>> case LT_EXPR:
>> case GE_EXPR:
>> case GT_EXPR:
>> case LE_EXPR:
>> case EQ_EXPR:
>> {
>> if (genmatch_gimple_simplify_COND_EXPR_chunk_1 (res_op, seq,
>> valueize, code, type, _p0, _p1, _p2))
>> return true;
>> break;
>> }
>> case NE_EXPR:
>> {
>> if (genmatch_gimple_simplify_COND_EXPR_chunk_2 (...))
>> return true;
>> break;
>> }
>>
>> and each group becomes a function of its own, which output partitioning is
>> then free to place in any file:
>>
>> genmatch_gimple_simplify_COND_EXPR_chunk_1 (gimple_match_op *res_op, ...)
>> {
>> switch (TREE_CODE (_p0))
>> {
>> case SSA_NAME:
>> if (gimple *_d1 = get_def (valueize, _p0))
>> if (gassign *_a1 = dyn_cast <gassign *> (_d1))
>> switch (gimple_assign_rhs_code (_a1))
>> {
>> case LT_EXPR:
>> {
>> tree _q20 = gimple_assign_rhs1 (_a1);
>> ...
>>
>> The case labels of consecutive groups that land in the same function fall
>> through to one call rather than each getting its own, which is what the run
>> of
>> labels above is.
>>
>> The groups are the ones collect_operand_groups already forms, which merges
>> any two alternatives whose cases overlap. Distinct groups therefore have
>> disjoint cases, and selecting one is what trying them in turn does, so the
>> order in which patterns are tried does not change. Alternatives that carry
>> no case, the predicates and the remaining decision-tree nodes, stay in the
>> entry function. Use one tenth of the average number of decision-tree nodes
>> per output part as the target group size. Keep single-file output unchanged
>> because it has no shared declaration header.
>>
>> On aarch64 with the default ten parts the largest-to-smallest GIMPLE part
>> ratio goes from 2.417 to 1.116, compiling the twenty generated sources at a
>> parallelism of ten takes about a fifth less wall time, and a bootstrap at
>> -j110 comes out 10.7% shorter. The generated text
>> grows by 2.4%, which is the dispatch in the entry plus the scaffolding each
>> group repeats to reach its own alternatives.
>>
>> The compile time performance is not worse. Over seven large preprocessed
>> translation units, against trunk: instructions +0.035%, cycles
>> -0.078%, instruction-TLB misses -7.62%, L1I refills -1.18%.
>> A match enters the dispatcher and one group, where trunk walks one function
>> holding all of them, so the working set of a match is smaller.
>>
>> Bootstrapped and tested on aarch64-none-linux-gnu.
>> Ok for trunk?
>
> Sorry for not jumping to this immediately. Given the numbers I'm
> not sure it's worth the increased complexity in the already
> complex code generator in genmatch.
Improving bootstrap time is the main motivation for this. I have some patches
in progress to improve the configure serialization bottleneck and once that’s
done the large genmatch simplifiers would be the next bottleneck.
I’ve tried to implement some improvements to reduce the size of the emitted
code to pay for the bit of extra complexity in this patch.
Resent as a series.
Thanks for looking at this.
Kyrill
> Instead I see
>
> bool
> gimple_simplify_VEC_COND_EXPR (gimple_match_op *res_op, gimple_seq *seq,
> ...
> switch (TREE_CODE (_p0))
> {
> case SSA_NAME:
> ...
> switch (TREE_CODE (_p1))
> {
> case SSA_NAME:
> ...
> switch (TREE_CODE (_p2))
> {
> ...
> switch (TREE_CODE (_p1))
> {
> ...
> switch (TREE_CODE (_p2))
> {
> case SSA_NAME:
> ...
>
> which is I think caused by the order-preserving processing of
> matches and in the above case toplevel catch-all. In almost all
> cases preserving the order of pattern matches is _not_ important.
> I do not remember exactly which case prompted me to implement
> that in the first place, but I remember there was one.
>
> With -v genmatch prints cases like
>
> /space/rguenther/src/gcc-clean/gcc/match.pd:1599:9 warning: failed to
> merge decision tree node
> (mult (rdiv@3 REAL_CST@0 @1) REAL_CST@2)
> ^
> /space/rguenther/src/gcc-clean/gcc/match.pd:854:10 warning: with the
> following
> (mult (rdiv@4 @0 @1) (rdiv@5 @2 @3))
> ^
> /space/rguenther/src/gcc-clean/gcc/match.pd:1450:10 warning: because of
> the following which serves as ordering barrier
> (mult:c @0 (vec_cond (ne @0 integer_zerop) integer_zerop@2 @1))
> ^
>
> for true (aka @0) there's 2% reduction in line numbers, for
> the 'overlap_node' case another 1% (don't remember exactly what
> that barrier case was about). Similarly improving (for ...)
> factoring to for example also cover toplevel operations
> (cond and vec_cond) would reduce duplication.
>
> -v also prints duplicate patterns that are hard to avoid, like
>
> /space/rguenther/src/gcc-clean/gcc/match.pd:3930:5 warning: duplicate
> pattern
> (bit_ior:c (code1:c@3 @0 @1) (code2:c@4 (convert?@c0 @0) @2))
> ^
> /space/rguenther/src/gcc-clean/gcc/match.pd:3930:5 warning: previous
> pattern defined here
> (bit_ior:c (code1:c@3 @0 @1) (code2:c@4 (convert?@c0 @0) @2))
> ^
>
> diagnosed by
>
> dt_node *
> dt_node::append_simplify (simplify *s, unsigned pattern_no,
> dt_operand **indexes)
> {
> dt_simplify *s2;
> dt_simplify *n = new dt_simplify (s, pattern_no, indexes);
> for (unsigned i = 0; i < kids.length (); ++i)
> if ((s2 = dyn_cast <dt_simplify *> (kids[i]))
> && (verbose >= 1
> || s->match->location != s2->s->match->location))
> {
> /* With a nested patters, it's hard to avoid these in order
> to keep match.pd rules relatively small. */
> warning_at (s->match->location, "duplicate pattern");
> warning_at (s2->s->match->location, "previous pattern defined
> here");
> print_operand (s->match, stderr);
> fprintf (stderr, "\n");
> }
> return append_node (n);
>
> where we generate code for those duplicates (I think, have to
> double-check), rather than optimistically eliding them. The above
> example is from
>
> (for code1 (eq ne)
> (for code2 (eq ne lt gt le ge)
> (simplify
> (bit_ior:c (code1:c@3 @0 @1) (code2:c@4 (convert?@c0 @0) @2))
>
> where the bit_ior:c produces duplicates when the conditional convert
> is stripped.
>
> Richard.
>
>> Thanks,
>> Kyrill
>>
>> gcc/ChangeLog:
>>
>> * genmatch.cc (MATCHER_FN_FILE_FRACTION): New constant.
>> (class fn_splitter): New class.
>> (fn_splitter::full, fn_splitter::same_chunk_p)
>> (fn_splitter::print_params, fn_splitter::print_args)
>> (fn_splitter::choose_chunk_file, fn_splitter::open_chunk)
>> (fn_splitter::close_chunk, fn_splitter::emit_dispatch)
>> (collect_alt_units, fn_splitter::emit_alternatives): New functions.
>> (dt_node::gen_kids): Add an optional fn_splitter argument and use it
>> at valid function-level boundaries.
>> (dt_node::gen_kids_1): Add an optional fn_splitter argument and call
>> the function holding a group in place of its body.
>> (decision_tree::gen): Set the matcher function budget and use
>> fn_splitter for large per-operation matchers.
>>
>> Signed-off-by: Kyrylo Tkachov <[email protected]>
>> ---
>> gcc/genmatch.cc | 435 +++++++++++++++++++++++++++++++++++++++++++++---
>> 1 file changed, 411 insertions(+), 24 deletions(-)
>>
>> diff --git a/gcc/genmatch.cc b/gcc/genmatch.cc
>> index 42a04436c57..7b786057dab 100644
>> --- a/gcc/genmatch.cc
>> +++ b/gcc/genmatch.cc
>> @@ -2516,6 +2516,7 @@ lower (vec<simplify *>& simplifiers, bool gimple)
>> simplifies and has those as its leafs. */
>>
>> class dt_simplify;
>> +class fn_splitter;
>>
>> /* A hash-map collecting semantically equivalent leafs in the decision
>> tree for splitting out to separate functions. */
>> @@ -2571,11 +2572,12 @@ public:
>>
>> virtual void gen (FILE *, int, bool, int) {}
>>
>> - void gen_kids (FILE *, int, bool, int, bool);
>> + void gen_kids (FILE *, int, bool, int, bool, fn_splitter * = NULL);
>> void gen_kids_1 (FILE *, int, bool, int,
>> const vec<dt_operand *> &, const vec<dt_operand *> &,
>> const vec<dt_operand *> &, const vec<dt_operand *> &,
>> - const vec<dt_operand *> &, const vec<dt_node *> &, bool);
>> + const vec<dt_operand *> &, const vec<dt_node *> &, bool,
>> + fn_splitter * = NULL);
>>
>> void analyze (sinfo_map_t &);
>> };
>> @@ -4259,11 +4261,326 @@ emit_fn_group (FILE *f, int indent, bool gimple,
>> int depth,
>> fprintf_indent (f, indent, " break;\n");
>> }
>>
>> +/* Target matcher functions at this fraction of the decision-tree nodes that
>> + one output file receives on average. */
>> +
>> +static const unsigned int MATCHER_FN_FILE_FRACTION = 10;
>> +
>> +/* Split a generated matcher across functions. The entry function keeps the
>> + dispatch that selects between the top-level alternatives and hands each
>> + group of them to the function that holds it, so that matching walks one
>> + group rather than all of them. */
>> +
>> +class fn_splitter
>> +{
>> +public:
>> + /* These are the alternative vectors accepted by gen_kids_1, in their
>> + emission order. ALT_OTHERS identifies the separate dt_node vector. */
>> + enum alt_cat
>> + {
>> + ALT_GIMPLE_EXPRS,
>> + ALT_FNS,
>> + ALT_GENERIC_EXPRS,
>> + ALT_GENERIC_FNS,
>> + ALT_PREDS,
>> + ALT_OTHERS
>> + };
>> +
>> + /* One complete group from category CAT at INDEX. NODES estimates the
>> + amount of generated code for the group. */
>> + struct alt_unit
>> + {
>> + alt_cat cat;
>> + unsigned index;
>> + unsigned nodes;
>> + };
>> +
>> + fn_splitter (vec<FILE *> &files_, FILE *f_, const char *fname_,
>> + bool gimple_, unsigned nops_, unsigned budget_)
>> + : f (f_), files (files_), fname (fname_), gimple (gimple_),
>> + nops (nops_), budget (budget_), nchunks (0), dispatching (false)
>> + {}
>> +
>> + void emit_alternatives (dt_node *, int, int,
>> + const vec<dt_operand *> &,
>> + const vec<dt_operand *> &,
>> + const vec<dt_operand *> &,
>> + const vec<dt_operand *> &,
>> + const vec<dt_operand *> &,
>> + const vec<dt_node *> &);
>> + bool emit_dispatch (FILE *, int, alt_cat, unsigned);
>> +
>> + /* Return true if groups A and B of category CAT go to the same function,
>> + so that the labels of A can fall through to the call emitted for B. */
>> + bool same_chunk_p (alt_cat cat, unsigned a, unsigned b) const
>> + {
>> + return (dispatching
>> + && b < owner[cat].length ()
>> + && owner[cat][a]
>> + && owner[cat][a] == owner[cat][b]);
>> + }
>> +
>> + /* The output file for the entry function. */
>> + FILE *f;
>> +
>> +private:
>> + /* Return true if adding MORE nodes to HAVE nodes requires a new function.
>> + Always let an empty function accept one complete group. A zero budget
>> + disables splitting. */
>> + bool full (unsigned have, unsigned more) const
>> + {
>> + return budget && have && have + more > budget;
>> + }
>> +
>> + void print_params (FILE *);
>> + void print_args (FILE *);
>> + FILE *open_chunk (unsigned);
>> + void close_chunk (FILE *);
>> + FILE *choose_chunk_file ();
>> +
>> + /* All output files used for the generated matcher. */
>> + vec<FILE *> &files;
>> + /* The base name for new functions. */
>> + const char *fname;
>> + /* True for GIMPLE matching and false for GENERIC matching. */
>> + bool gimple;
>> + /* The number of operands passed to each function. */
>> + unsigned nops;
>> + /* The target maximum number of nodes accumulated in one function. */
>> + unsigned budget;
>> + /* The number of new functions. */
>> + unsigned nchunks;
>> + /* True while the entry function is being emitted, when the body of a
>> + group is replaced by a call to the function that holds it. */
>> + bool dispatching;
>> + /* For each category, the function holding each group. */
>> + auto_vec<unsigned> owner[ALT_PREDS];
>> +};
>> +
>> +/* Print the parameter list of a new function to TO. Also print its
>> + declaration to the generated header. */
>> +
>> +void
>> +fn_splitter::print_params (FILE *to)
>> +{
>> + if (gimple)
>> + fp_decl (to, " (gimple_match_op *ARG_UNUSED (res_op),"
>> + " gimple_seq *ARG_UNUSED (seq),\n"
>> + "\t\t tree (*valueize)(tree) ATTRIBUTE_UNUSED,\n"
>> + "\t\t code_helper ARG_UNUSED (code),"
>> + " tree ARG_UNUSED (type)");
>> + else
>> + fp_decl (to, " (location_t ARG_UNUSED (loc),"
>> + " enum tree_code ARG_UNUSED (code),\n"
>> + "\t\t const tree ARG_UNUSED (type)");
>> + for (unsigned i = 0; i < nops; ++i)
>> + fp_decl (to, ", tree ARG_UNUSED (_p%d)", i);
>> + fp_decl (to, ")");
>> +}
>> +
>> +/* Print to TO the arguments that forward the current matcher state to a new
>> + function. */
>> +
>> +void
>> +fn_splitter::print_args (FILE *to)
>> +{
>> + fputs (gimple
>> + ? " (res_op, seq, valueize, code, type"
>> + : " (loc, code, type", to);
>> + for (unsigned i = 0; i < nops; ++i)
>> + fprintf (to, ", _p%d", i);
>> + fputs (")", to);
>> +}
>> +
>> +/* Return the shortest output file that is not the one the entry function is
>> + being written to, which must not receive another function while it is
>> + open. */
>> +
>> +FILE *
>> +fn_splitter::choose_chunk_file ()
>> +{
>> + FILE *best = NULL;
>> + long best_len = 0;
>> + for (FILE *cand : files)
>> + {
>> + if (cand == f)
>> + continue;
>> + long len = ftell (cand);
>> + if (!best || len < best_len)
>> + {
>> + best = cand;
>> + best_len = len;
>> + }
>> + }
>> + /* Splitting is only enabled with more than one output file. */
>> + gcc_assert (best);
>> + return best;
>> +}
>> +
>> +/* Open the function holding chunk N and return its output file. */
>> +
>> +FILE *
>> +fn_splitter::open_chunk (unsigned n)
>> +{
>> + /* Multiple output files require a header, which also carries the
>> + declarations for functions emitted in a different file. */
>> + gcc_assert (header_file);
>> +
>> + FILE *cf = choose_chunk_file ();
>> + fp_decl (cf, "\n%s\n%s_chunk_%u", gimple ? "bool" : "tree", fname, n);
>> + print_params (cf);
>> + fp_decl_done (cf, "");
>> + fputs ("{\n", cf);
>> + fprintf_indent (cf, 2, "const bool debug_dump = "
>> + "dump_file && (dump_flags & TDF_FOLDING);\n");
>> + return cf;
>> +}
>> +
>> +/* Close the function being written to CF. */
>> +
>> +void
>> +fn_splitter::close_chunk (FILE *cf)
>> +{
>> + fprintf (cf, gimple ? " return false;\n}\n" : " return
>> NULL_TREE;\n}\n");
>> +}
>> +
>> +/* Emitted from gen_kids_1 in place of the body of group INDEX of category
>> + CAT. Hand control to the function holding that group and return true, or
>> + return false to have the body emitted here after all. */
>> +
>> +bool
>> +fn_splitter::emit_dispatch (FILE *to, int indent, alt_cat cat, unsigned
>> index)
>> +{
>> + if (!dispatching || index >= owner[cat].length () || !owner[cat][index])
>> + return false;
>> +
>> + fprintf_indent (to, indent, " {\n");
>> + if (gimple)
>> + {
>> + fprintf_indent (to, indent, " if (%s_chunk_%u", fname,
>> + owner[cat][index]);
>> + print_args (to);
>> + fprintf (to, ")\n");
>> + fprintf_indent (to, indent, " return true;\n");
>> + }
>> + else
>> + {
>> + fprintf_indent (to, indent, " if (tree _chunk_res = %s_chunk_%u",
>> + fname, owner[cat][index]);
>> + print_args (to);
>> + fprintf (to, ")\n");
>> + fprintf_indent (to, indent, " return _chunk_res;\n");
>> + }
>> + fprintf_indent (to, indent, " break;\n");
>> + fprintf_indent (to, indent, " }\n");
>> + return true;
>> +}
>> +
>> +/* Append the complete operand GROUPS to UNITS as category CAT. */
>> +
>> +static void
>> +collect_alt_units (vec<fn_splitter::alt_unit> &units,
>> + fn_splitter::alt_cat cat,
>> + const vec<dt_operand_group *> &groups)
>> +{
>> + for (unsigned i = 0; i < groups.length (); ++i)
>> + {
>> + /* Only alternatives with a level-one parent inspect a parameter of
>> the
>> + matcher entry function. */
>> + unsigned nodes = 0;
>> + for (auto op : groups[i]->operands)
>> + {
>> + gcc_assert (op->parent && op->parent->level == 1);
>> + nodes += op->total_size;
>> + }
>> + units.safe_push (fn_splitter::alt_unit { cat, i, nodes });
>> + }
>> +}
>> +
>> +/* Emit the alternatives for NODE. The entry function keeps the dispatch
>> and
>> + each group of alternatives that does not fit the budget alongside the
>> ones
>> + before it goes to a function of its own. Groups reached from one
>> dispatch
>> + have disjoint cases, so selecting one is what trying them in turn does.
>> */
>> +
>> +void
>> +fn_splitter::emit_alternatives (dt_node *node, int indent, int depth,
>> + const vec<dt_operand *> &gimple_exprs,
>> + const vec<dt_operand *> &generic_exprs,
>> + const vec<dt_operand *> &fns,
>> + const vec<dt_operand *> &generic_fns,
>> + const vec<dt_operand *> &preds,
>> + const vec<dt_node *> &others)
>> +{
>> + auto_delete_vec<dt_operand_group> group_storage[ALT_PREDS];
>> + collect_operand_groups (gimple_exprs, group_storage[ALT_GIMPLE_EXPRS]);
>> + collect_operand_groups (fns, group_storage[ALT_FNS]);
>> + collect_operand_groups (generic_exprs, group_storage[ALT_GENERIC_EXPRS]);
>> + collect_operand_groups (generic_fns, group_storage[ALT_GENERIC_FNS]);
>> +
>> + auto_vec<alt_unit> units;
>> + for (unsigned cat = ALT_GIMPLE_EXPRS; cat < ALT_PREDS; ++cat)
>> + collect_alt_units (units, (alt_cat) cat, group_storage[cat]);
>> +
>> + /* Decide which function holds each group. Zero keeps it in the entry,
>> + which is where a group stays when nothing has to be split out. */
>> + for (unsigned cat = ALT_GIMPLE_EXPRS; cat < ALT_PREDS; ++cat)
>> + {
>> + owner[cat].truncate (0);
>> + owner[cat].safe_grow_cleared (group_storage[cat].length ());
>> + }
>> +
>> + const unsigned first_chunk = nchunks + 1;
>> + unsigned have = 0;
>> + bool split_p = false;
>> + for (auto &u : units)
>> + {
>> + if (!split_p || full (have, u.nodes))
>> + {
>> + ++nchunks;
>> + have = 0;
>> + split_p = true;
>> + }
>> + owner[u.cat][u.index] = nchunks;
>> + have += u.nodes;
>> + }
>> +
>> + /* The entry function: the dispatch, and whatever cannot be reached
>> + through it. */
>> + dispatching = true;
>> + node->gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
>> + fns, generic_fns, preds, others, false, this);
>> + dispatching = false;
>> +
>> + /* One function per group of alternatives that was split out. */
>> + for (unsigned n = first_chunk; n <= nchunks; ++n)
>> + {
>> + auto_vec<dt_operand *> sub[ALT_PREDS];
>> + for (unsigned cat = ALT_GIMPLE_EXPRS; cat < ALT_PREDS; ++cat)
>> + for (unsigned i = 0; i < owner[cat].length (); ++i)
>> + if (owner[cat][i] == n)
>> + for (auto op : group_storage[cat][i]->operands)
>> + sub[cat].safe_push (op);
>> +
>> + FILE *cf = open_chunk (n);
>> + auto_vec<dt_operand *> no_preds;
>> + auto_vec<dt_node *> no_others;
>> + node->gen_kids_1 (cf, 2, gimple, depth, sub[ALT_GIMPLE_EXPRS],
>> + sub[ALT_GENERIC_EXPRS], sub[ALT_FNS],
>> + sub[ALT_GENERIC_FNS], no_preds, no_others, false);
>> + close_chunk (cf);
>> + }
>> +}
>> +
>> /* Generate matching code for the children of the decision tree node. */
>>
>> void
>> -dt_node::gen_kids (FILE *f, int indent, bool gimple, int depth, bool
>> in_gcond)
>> +dt_node::gen_kids (FILE *f, int indent, bool gimple, int depth, bool
>> in_gcond,
>> + fn_splitter *sp)
>> {
>> + /* A gcond matcher starts inside an open scope and is not a valid function
>> + boundary. */
>> + gcc_assert (!sp || !in_gcond);
>> +
>> auto_vec<dt_operand *> gimple_exprs;
>> auto_vec<dt_operand *> generic_exprs;
>> auto_vec<dt_operand *> fns;
>> @@ -4332,10 +4649,25 @@ dt_node::gen_kids (FILE *f, int indent, bool gimple,
>> int depth, bool in_gcond)
>> Like DT_TRUE, DT_MATCH serves as a barrier as it can cause
>> dependent matches to get out-of-order. Generate code now
>> for what we have collected sofar. */
>> - gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
>> - fns, generic_fns, preds, others, in_gcond);
>> + if (sp)
>> + {
>> + sp->emit_alternatives (this, indent, depth, gimple_exprs,
>> + generic_exprs, fns, generic_fns, preds,
>> + others);
>> + }
>> + else
>> + gen_kids_1 (f, indent, gimple, depth, gimple_exprs,
>> + generic_exprs, fns, generic_fns, preds, others,
>> + in_gcond);
>> /* And output the true operand itself. */
>> - kids[i]->gen (f, indent, gimple, depth);
>> + if (sp && kids[i]->type == dt_node::DT_TRUE)
>> + {
>> + /* A DT_TRUE node emits no code and opens no scope. Its children
>> + remain valid function-level split points. */
>> + kids[i]->gen_kids (f, indent, gimple, depth, false, sp);
>> + }
>> + else
>> + kids[i]->gen (f, indent, gimple, depth);
>> gimple_exprs.truncate (0);
>> generic_exprs.truncate (0);
>> fns.truncate (0);
>> @@ -4348,8 +4680,12 @@ dt_node::gen_kids (FILE *f, int indent, bool gimple,
>> int depth, bool in_gcond)
>> }
>>
>> /* Generate code for the remains. */
>> - gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
>> - fns, generic_fns, preds, others, in_gcond);
>> + if (sp)
>> + sp->emit_alternatives (this, indent, depth, gimple_exprs, generic_exprs,
>> + fns, generic_fns, preds, others);
>> + else
>> + gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
>> + fns, generic_fns, preds, others, in_gcond);
>> }
>>
>> /* Generate matching code for the children of the decision tree node. */
>> @@ -4362,7 +4698,7 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool gimple,
>> int depth,
>> const vec<dt_operand *> &generic_fns,
>> const vec<dt_operand *> &preds,
>> const vec<dt_node *> &others,
>> - bool in_gcond)
>> + bool in_gcond, fn_splitter *sp)
>> {
>> char buf[128];
>> char *kid_opname = buf;
>> @@ -4430,10 +4766,17 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
>> gimple, int depth,
>> char code[64];
>> snprintf (code, sizeof (code),
>> "gimple_assign_rhs_code (_a%d)", depth);
>> - for (auto group : gimple_expr_groups)
>> + for (unsigned gi = 0; gi < gimple_expr_groups.length (); ++gi)
>> {
>> - emit_group_labels (f, indent, group);
>> - emit_expr_group (f, indent, true, depth, code, group);
>> + emit_group_labels (f, indent, gimple_expr_groups[gi]);
>> + if (sp
>> + && sp->same_chunk_p (fn_splitter::ALT_GIMPLE_EXPRS, gi,
>> + gi + 1))
>> + continue;
>> + if (!sp || !sp->emit_dispatch (f, indent,
>> + fn_splitter::ALT_GIMPLE_EXPRS, gi))
>> + emit_expr_group (f, indent, true, depth, code,
>> + gimple_expr_groups[gi]);
>> }
>> fprintf_indent (f, indent, "default:;\n");
>> fprintf_indent (f, indent, "}\n");
>> @@ -4475,10 +4818,15 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
>> gimple, int depth,
>> char code[64];
>> snprintf (code, sizeof (code),
>> "gimple_call_combined_fn (_c%d)", depth);
>> - for (auto group : fn_groups)
>> + for (unsigned fi = 0; fi < fn_groups.length (); ++fi)
>> {
>> - emit_group_labels (f, indent, group);
>> - emit_fn_group (f, indent, true, depth, NULL, code, group);
>> + emit_group_labels (f, indent, fn_groups[fi]);
>> + if (sp && sp->same_chunk_p (fn_splitter::ALT_FNS, fi, fi + 1))
>> + continue;
>> + if (!sp || !sp->emit_dispatch (f, indent, fn_splitter::ALT_FNS,
>> + fi))
>> + emit_fn_group (f, indent, true, depth, NULL, code,
>> + fn_groups[fi]);
>> }
>> fprintf_indent (f, indent, "default:;\n");
>> fprintf_indent (f, indent, "}\n");
>> @@ -4511,14 +4859,29 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
>> gimple, int depth,
>> char generic_code[160];
>> snprintf (generic_code, sizeof (generic_code), "TREE_CODE (%s)",
>> kid_opname);
>> - for (auto group : generic_expr_groups)
>> + /* The SSA_NAME group was emitted above, so it is not here to fall
>> + through to, and a run of labels must not be merged across it. */
>> + auto emitted_above_p = [&] (unsigned i)
>> + {
>> + return (gimple_match_p
>> + && generic_expr_groups[i]->cases.length () == 1
>> + && *generic_expr_groups[i]->cases[0] == SSA_NAME);
>> + };
>> + for (unsigned gei = 0; gei < generic_expr_groups.length (); ++gei)
>> {
>> - if (gimple_match_p
>> - && group->cases.length () == 1
>> - && *group->cases[0] == SSA_NAME)
>> + dt_operand_group *group = generic_expr_groups[gei];
>> + if (emitted_above_p (gei))
>> continue;
>> emit_group_labels (f, indent, group);
>> - emit_expr_group (f, indent, gimple, depth, generic_code, group);
>> + if (sp
>> + && gei + 1 < generic_expr_groups.length ()
>> + && !emitted_above_p (gei + 1)
>> + && sp->same_chunk_p (fn_splitter::ALT_GENERIC_EXPRS, gei,
>> + gei + 1))
>> + continue;
>> + if (!sp || !sp->emit_dispatch (f, indent,
>> + fn_splitter::ALT_GENERIC_EXPRS, gei))
>> + emit_expr_group (f, indent, gimple, depth, generic_code, group);
>> }
>> }
>>
>> @@ -4536,10 +4899,16 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
>> gimple, int depth,
>> char code[160];
>> snprintf (code, sizeof (code), "get_call_combined_fn (%s)",
>> kid_opname);
>> - for (auto group : generic_fn_groups)
>> + for (unsigned gfi = 0; gfi < generic_fn_groups.length (); ++gfi)
>> {
>> - emit_group_labels (f, indent, group);
>> - emit_fn_group (f, indent, false, depth, kid_opname, code, group);
>> + emit_group_labels (f, indent, generic_fn_groups[gfi]);
>> + if (sp && sp->same_chunk_p (fn_splitter::ALT_GENERIC_FNS, gfi,
>> + gfi + 1))
>> + continue;
>> + if (!sp || !sp->emit_dispatch (f, indent,
>> + fn_splitter::ALT_GENERIC_FNS, gfi))
>> + emit_fn_group (f, indent, false, depth, kid_opname, code,
>> + generic_fn_groups[gfi]);
>> }
>> fprintf_indent (f, indent, "default:;\n");
>>
>> @@ -5324,6 +5693,14 @@ decision_tree::gen (vec <FILE *> &files, bool gimple)
>> }
>> fprintf (stderr, "removed %u duplicate tails\n", rcnt);
>>
>> + /* A file receives about ROOT->TOTAL_SIZE / NFILES decision-tree nodes.
>> + Keep matcher functions small enough to spread them across the files.
>> + Keep single-file output unchanged. */
>> + unsigned nfiles = files.length ();
>> + unsigned budget = (nfiles > 1
>> + ? root->total_size / nfiles / MATCHER_FN_FILE_FRACTION
>> + : 0);
>> +
>> for (unsigned n = 1; n <= 7; ++n)
>> {
>> bool has_kids_p = false;
>> @@ -5366,7 +5743,17 @@ decision_tree::gen (vec <FILE *> &files, bool gimple)
>> fprintf (f, "{\n");
>> fprintf_indent (f, 2, "const bool debug_dump = "
>> "dump_file && (dump_flags & TDF_FOLDING);\n");
>> - dop->gen_kids (f, 2, gimple, 0, false);
>> + if (budget && dop->total_size > budget)
>> + {
>> + char *fname = xasprintf ("genmatch_%s_simplify_%s",
>> + gimple ? "gimple" : "generic",
>> + e->operation->id);
>> + fn_splitter sp (files, f, fname, gimple, n, budget);
>> + dop->gen_kids (f, 2, gimple, 0, false, &sp);
>> + free (fname);
>> + }
>> + else
>> + dop->gen_kids (f, 2, gimple, 0, false);
>> if (gimple)
>> fprintf (f, " return false;\n");
>> else
>>
>
> --
> Richard Biener <[email protected]>
> SUSE Software Solutions Germany GmbH,
> Frankenstrasse 146, 90461 Nuernberg, Germany;
> GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri; (HRB 36809, AG Nuernberg)
