On Thu, 17 Sep 2026, [email protected] wrote:
> From: Kyrylo Tkachov <[email protected]>
>
> This a respin after Richard's feedback, with an attempt to improve the
> locality of the splitted code.
>
> genmatch writes one function for each outer operation. Output partitioning
> places those functions whole, so one operation matcher can dominate an output
> part: on trunk gimple_simplify_COND_EXPR is 58429 lines, and the largest
> GIMPLE part is 2.417 times the smallest. Compiling it with profile
> generation for Armv8-A Thumb also used to fail to assemble due to branch out
> of
> range assembly errors. That is a separate arm bug that can be fixed elsewhere.
>
> Split a large per-operation matcher between its top-level alternatives. The
> entry function keeps the dispatch and calls the function that holds the group
> it selected, so a match enters one group rather than walking all of them.
>
> Before, one function holds the dispatch and every alternative it reaches:
>
> gimple_simplify_COND_EXPR (gimple_match_op *res_op, ...)
> {
> switch (TREE_CODE (_p0))
> {
> case SSA_NAME:
> if (gimple *_d1 = get_def (valueize, _p0))
> if (gassign *_a1 = dyn_cast <gassign *> (_d1))
> switch (gimple_assign_rhs_code (_a1))
> {
> case LT_EXPR:
> {
> tree _q20 = gimple_assign_rhs1 (_a1);
> ... <- 58429 lines in all
> }
> case NE_EXPR:
> { ... }
>
> After, the entry holds the dispatch and nothing else:
>
> gimple_simplify_COND_EXPR (gimple_match_op *res_op, ...)
> {
> switch (TREE_CODE (_p0))
> {
> case SSA_NAME:
> if (gimple *_d1 = get_def (valueize, _p0))
> if (gassign *_a1 = dyn_cast <gassign *> (_d1))
> switch (gimple_assign_rhs_code (_a1))
> {
> case LT_EXPR:
> case GE_EXPR:
> case GT_EXPR:
> case LE_EXPR:
> case EQ_EXPR:
> {
> if (genmatch_gimple_simplify_COND_EXPR_chunk_1 (res_op, seq,
> valueize, code, type, _p0, _p1, _p2))
> return true;
> break;
> }
> case NE_EXPR:
> {
> if (genmatch_gimple_simplify_COND_EXPR_chunk_2 (...))
> return true;
> break;
> }
>
> and each group becomes a function of its own, which output partitioning is
> then free to place in any file:
>
> genmatch_gimple_simplify_COND_EXPR_chunk_1 (gimple_match_op *res_op, ...)
> {
> switch (TREE_CODE (_p0))
> {
> case SSA_NAME:
> if (gimple *_d1 = get_def (valueize, _p0))
> if (gassign *_a1 = dyn_cast <gassign *> (_d1))
> switch (gimple_assign_rhs_code (_a1))
> {
> case LT_EXPR:
> {
> tree _q20 = gimple_assign_rhs1 (_a1);
> ...
>
> The case labels of consecutive groups that land in the same function fall
> through to one call rather than each getting its own, which is what the run of
> labels above is.
>
> The groups are the ones collect_operand_groups already forms, which merges
> any two alternatives whose cases overlap. Distinct groups therefore have
> disjoint cases, and selecting one is what trying them in turn does, so the
> order in which patterns are tried does not change. Alternatives that carry
> no case, the predicates and the remaining decision-tree nodes, stay in the
> entry function. Use one tenth of the average number of decision-tree nodes
> per output part as the target group size. Keep single-file output unchanged
> because it has no shared declaration header.
>
> On aarch64 with the default ten parts the largest-to-smallest GIMPLE part
> ratio goes from 2.417 to 1.116, compiling the twenty generated sources at a
> parallelism of ten takes about a fifth less wall time, and a bootstrap at
> -j110 comes out 10.7% shorter. The generated text
> grows by 2.4%, which is the dispatch in the entry plus the scaffolding each
> group repeats to reach its own alternatives.
>
> The compile time performance is not worse. Over seven large preprocessed
> translation units, against trunk: instructions +0.035%, cycles
> -0.078%, instruction-TLB misses -7.62%, L1I refills -1.18%.
> A match enters the dispatcher and one group, where trunk walks one function
> holding all of them, so the working set of a match is smaller.
>
> Bootstrapped and tested on aarch64-none-linux-gnu.
> Ok for trunk?
Sorry for not jumping to this immediately. Given the numbers I'm
not sure it's worth the increased complexity in the already
complex code generator in genmatch. Instead I see
bool
gimple_simplify_VEC_COND_EXPR (gimple_match_op *res_op, gimple_seq *seq,
...
switch (TREE_CODE (_p0))
{
case SSA_NAME:
...
switch (TREE_CODE (_p1))
{
case SSA_NAME:
...
switch (TREE_CODE (_p2))
{
...
switch (TREE_CODE (_p1))
{
...
switch (TREE_CODE (_p2))
{
case SSA_NAME:
...
which is I think caused by the order-preserving processing of
matches and in the above case toplevel catch-all. In almost all
cases preserving the order of pattern matches is _not_ important.
I do not remember exactly which case prompted me to implement
that in the first place, but I remember there was one.
With -v genmatch prints cases like
/space/rguenther/src/gcc-clean/gcc/match.pd:1599:9 warning: failed to
merge decision tree node
(mult (rdiv@3 REAL_CST@0 @1) REAL_CST@2)
^
/space/rguenther/src/gcc-clean/gcc/match.pd:854:10 warning: with the
following
(mult (rdiv@4 @0 @1) (rdiv@5 @2 @3))
^
/space/rguenther/src/gcc-clean/gcc/match.pd:1450:10 warning: because of
the following which serves as ordering barrier
(mult:c @0 (vec_cond (ne @0 integer_zerop) integer_zerop@2 @1))
^
for true (aka @0) there's 2% reduction in line numbers, for
the 'overlap_node' case another 1% (don't remember exactly what
that barrier case was about). Similarly improving (for ...)
factoring to for example also cover toplevel operations
(cond and vec_cond) would reduce duplication.
-v also prints duplicate patterns that are hard to avoid, like
/space/rguenther/src/gcc-clean/gcc/match.pd:3930:5 warning: duplicate
pattern
(bit_ior:c (code1:c@3 @0 @1) (code2:c@4 (convert?@c0 @0) @2))
^
/space/rguenther/src/gcc-clean/gcc/match.pd:3930:5 warning: previous
pattern defined here
(bit_ior:c (code1:c@3 @0 @1) (code2:c@4 (convert?@c0 @0) @2))
^
diagnosed by
dt_node *
dt_node::append_simplify (simplify *s, unsigned pattern_no,
dt_operand **indexes)
{
dt_simplify *s2;
dt_simplify *n = new dt_simplify (s, pattern_no, indexes);
for (unsigned i = 0; i < kids.length (); ++i)
if ((s2 = dyn_cast <dt_simplify *> (kids[i]))
&& (verbose >= 1
|| s->match->location != s2->s->match->location))
{
/* With a nested patters, it's hard to avoid these in order
to keep match.pd rules relatively small. */
warning_at (s->match->location, "duplicate pattern");
warning_at (s2->s->match->location, "previous pattern defined
here");
print_operand (s->match, stderr);
fprintf (stderr, "\n");
}
return append_node (n);
where we generate code for those duplicates (I think, have to
double-check), rather than optimistically eliding them. The above
example is from
(for code1 (eq ne)
(for code2 (eq ne lt gt le ge)
(simplify
(bit_ior:c (code1:c@3 @0 @1) (code2:c@4 (convert?@c0 @0) @2))
where the bit_ior:c produces duplicates when the conditional convert
is stripped.
Richard.
> Thanks,
> Kyrill
>
> gcc/ChangeLog:
>
> * genmatch.cc (MATCHER_FN_FILE_FRACTION): New constant.
> (class fn_splitter): New class.
> (fn_splitter::full, fn_splitter::same_chunk_p)
> (fn_splitter::print_params, fn_splitter::print_args)
> (fn_splitter::choose_chunk_file, fn_splitter::open_chunk)
> (fn_splitter::close_chunk, fn_splitter::emit_dispatch)
> (collect_alt_units, fn_splitter::emit_alternatives): New functions.
> (dt_node::gen_kids): Add an optional fn_splitter argument and use it
> at valid function-level boundaries.
> (dt_node::gen_kids_1): Add an optional fn_splitter argument and call
> the function holding a group in place of its body.
> (decision_tree::gen): Set the matcher function budget and use
> fn_splitter for large per-operation matchers.
>
> Signed-off-by: Kyrylo Tkachov <[email protected]>
> ---
> gcc/genmatch.cc | 435 +++++++++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 411 insertions(+), 24 deletions(-)
>
> diff --git a/gcc/genmatch.cc b/gcc/genmatch.cc
> index 42a04436c57..7b786057dab 100644
> --- a/gcc/genmatch.cc
> +++ b/gcc/genmatch.cc
> @@ -2516,6 +2516,7 @@ lower (vec<simplify *>& simplifiers, bool gimple)
> simplifies and has those as its leafs. */
>
> class dt_simplify;
> +class fn_splitter;
>
> /* A hash-map collecting semantically equivalent leafs in the decision
> tree for splitting out to separate functions. */
> @@ -2571,11 +2572,12 @@ public:
>
> virtual void gen (FILE *, int, bool, int) {}
>
> - void gen_kids (FILE *, int, bool, int, bool);
> + void gen_kids (FILE *, int, bool, int, bool, fn_splitter * = NULL);
> void gen_kids_1 (FILE *, int, bool, int,
> const vec<dt_operand *> &, const vec<dt_operand *> &,
> const vec<dt_operand *> &, const vec<dt_operand *> &,
> - const vec<dt_operand *> &, const vec<dt_node *> &, bool);
> + const vec<dt_operand *> &, const vec<dt_node *> &, bool,
> + fn_splitter * = NULL);
>
> void analyze (sinfo_map_t &);
> };
> @@ -4259,11 +4261,326 @@ emit_fn_group (FILE *f, int indent, bool gimple, int
> depth,
> fprintf_indent (f, indent, " break;\n");
> }
>
> +/* Target matcher functions at this fraction of the decision-tree nodes that
> + one output file receives on average. */
> +
> +static const unsigned int MATCHER_FN_FILE_FRACTION = 10;
> +
> +/* Split a generated matcher across functions. The entry function keeps the
> + dispatch that selects between the top-level alternatives and hands each
> + group of them to the function that holds it, so that matching walks one
> + group rather than all of them. */
> +
> +class fn_splitter
> +{
> +public:
> + /* These are the alternative vectors accepted by gen_kids_1, in their
> + emission order. ALT_OTHERS identifies the separate dt_node vector. */
> + enum alt_cat
> + {
> + ALT_GIMPLE_EXPRS,
> + ALT_FNS,
> + ALT_GENERIC_EXPRS,
> + ALT_GENERIC_FNS,
> + ALT_PREDS,
> + ALT_OTHERS
> + };
> +
> + /* One complete group from category CAT at INDEX. NODES estimates the
> + amount of generated code for the group. */
> + struct alt_unit
> + {
> + alt_cat cat;
> + unsigned index;
> + unsigned nodes;
> + };
> +
> + fn_splitter (vec<FILE *> &files_, FILE *f_, const char *fname_,
> + bool gimple_, unsigned nops_, unsigned budget_)
> + : f (f_), files (files_), fname (fname_), gimple (gimple_),
> + nops (nops_), budget (budget_), nchunks (0), dispatching (false)
> + {}
> +
> + void emit_alternatives (dt_node *, int, int,
> + const vec<dt_operand *> &,
> + const vec<dt_operand *> &,
> + const vec<dt_operand *> &,
> + const vec<dt_operand *> &,
> + const vec<dt_operand *> &,
> + const vec<dt_node *> &);
> + bool emit_dispatch (FILE *, int, alt_cat, unsigned);
> +
> + /* Return true if groups A and B of category CAT go to the same function,
> + so that the labels of A can fall through to the call emitted for B. */
> + bool same_chunk_p (alt_cat cat, unsigned a, unsigned b) const
> + {
> + return (dispatching
> + && b < owner[cat].length ()
> + && owner[cat][a]
> + && owner[cat][a] == owner[cat][b]);
> + }
> +
> + /* The output file for the entry function. */
> + FILE *f;
> +
> +private:
> + /* Return true if adding MORE nodes to HAVE nodes requires a new function.
> + Always let an empty function accept one complete group. A zero budget
> + disables splitting. */
> + bool full (unsigned have, unsigned more) const
> + {
> + return budget && have && have + more > budget;
> + }
> +
> + void print_params (FILE *);
> + void print_args (FILE *);
> + FILE *open_chunk (unsigned);
> + void close_chunk (FILE *);
> + FILE *choose_chunk_file ();
> +
> + /* All output files used for the generated matcher. */
> + vec<FILE *> &files;
> + /* The base name for new functions. */
> + const char *fname;
> + /* True for GIMPLE matching and false for GENERIC matching. */
> + bool gimple;
> + /* The number of operands passed to each function. */
> + unsigned nops;
> + /* The target maximum number of nodes accumulated in one function. */
> + unsigned budget;
> + /* The number of new functions. */
> + unsigned nchunks;
> + /* True while the entry function is being emitted, when the body of a
> + group is replaced by a call to the function that holds it. */
> + bool dispatching;
> + /* For each category, the function holding each group. */
> + auto_vec<unsigned> owner[ALT_PREDS];
> +};
> +
> +/* Print the parameter list of a new function to TO. Also print its
> + declaration to the generated header. */
> +
> +void
> +fn_splitter::print_params (FILE *to)
> +{
> + if (gimple)
> + fp_decl (to, " (gimple_match_op *ARG_UNUSED (res_op),"
> + " gimple_seq *ARG_UNUSED (seq),\n"
> + "\t\t tree (*valueize)(tree) ATTRIBUTE_UNUSED,\n"
> + "\t\t code_helper ARG_UNUSED (code),"
> + " tree ARG_UNUSED (type)");
> + else
> + fp_decl (to, " (location_t ARG_UNUSED (loc),"
> + " enum tree_code ARG_UNUSED (code),\n"
> + "\t\t const tree ARG_UNUSED (type)");
> + for (unsigned i = 0; i < nops; ++i)
> + fp_decl (to, ", tree ARG_UNUSED (_p%d)", i);
> + fp_decl (to, ")");
> +}
> +
> +/* Print to TO the arguments that forward the current matcher state to a new
> + function. */
> +
> +void
> +fn_splitter::print_args (FILE *to)
> +{
> + fputs (gimple
> + ? " (res_op, seq, valueize, code, type"
> + : " (loc, code, type", to);
> + for (unsigned i = 0; i < nops; ++i)
> + fprintf (to, ", _p%d", i);
> + fputs (")", to);
> +}
> +
> +/* Return the shortest output file that is not the one the entry function is
> + being written to, which must not receive another function while it is
> + open. */
> +
> +FILE *
> +fn_splitter::choose_chunk_file ()
> +{
> + FILE *best = NULL;
> + long best_len = 0;
> + for (FILE *cand : files)
> + {
> + if (cand == f)
> + continue;
> + long len = ftell (cand);
> + if (!best || len < best_len)
> + {
> + best = cand;
> + best_len = len;
> + }
> + }
> + /* Splitting is only enabled with more than one output file. */
> + gcc_assert (best);
> + return best;
> +}
> +
> +/* Open the function holding chunk N and return its output file. */
> +
> +FILE *
> +fn_splitter::open_chunk (unsigned n)
> +{
> + /* Multiple output files require a header, which also carries the
> + declarations for functions emitted in a different file. */
> + gcc_assert (header_file);
> +
> + FILE *cf = choose_chunk_file ();
> + fp_decl (cf, "\n%s\n%s_chunk_%u", gimple ? "bool" : "tree", fname, n);
> + print_params (cf);
> + fp_decl_done (cf, "");
> + fputs ("{\n", cf);
> + fprintf_indent (cf, 2, "const bool debug_dump = "
> + "dump_file && (dump_flags & TDF_FOLDING);\n");
> + return cf;
> +}
> +
> +/* Close the function being written to CF. */
> +
> +void
> +fn_splitter::close_chunk (FILE *cf)
> +{
> + fprintf (cf, gimple ? " return false;\n}\n" : " return NULL_TREE;\n}\n");
> +}
> +
> +/* Emitted from gen_kids_1 in place of the body of group INDEX of category
> + CAT. Hand control to the function holding that group and return true, or
> + return false to have the body emitted here after all. */
> +
> +bool
> +fn_splitter::emit_dispatch (FILE *to, int indent, alt_cat cat, unsigned
> index)
> +{
> + if (!dispatching || index >= owner[cat].length () || !owner[cat][index])
> + return false;
> +
> + fprintf_indent (to, indent, " {\n");
> + if (gimple)
> + {
> + fprintf_indent (to, indent, " if (%s_chunk_%u", fname,
> + owner[cat][index]);
> + print_args (to);
> + fprintf (to, ")\n");
> + fprintf_indent (to, indent, " return true;\n");
> + }
> + else
> + {
> + fprintf_indent (to, indent, " if (tree _chunk_res = %s_chunk_%u",
> + fname, owner[cat][index]);
> + print_args (to);
> + fprintf (to, ")\n");
> + fprintf_indent (to, indent, " return _chunk_res;\n");
> + }
> + fprintf_indent (to, indent, " break;\n");
> + fprintf_indent (to, indent, " }\n");
> + return true;
> +}
> +
> +/* Append the complete operand GROUPS to UNITS as category CAT. */
> +
> +static void
> +collect_alt_units (vec<fn_splitter::alt_unit> &units,
> + fn_splitter::alt_cat cat,
> + const vec<dt_operand_group *> &groups)
> +{
> + for (unsigned i = 0; i < groups.length (); ++i)
> + {
> + /* Only alternatives with a level-one parent inspect a parameter of the
> + matcher entry function. */
> + unsigned nodes = 0;
> + for (auto op : groups[i]->operands)
> + {
> + gcc_assert (op->parent && op->parent->level == 1);
> + nodes += op->total_size;
> + }
> + units.safe_push (fn_splitter::alt_unit { cat, i, nodes });
> + }
> +}
> +
> +/* Emit the alternatives for NODE. The entry function keeps the dispatch and
> + each group of alternatives that does not fit the budget alongside the ones
> + before it goes to a function of its own. Groups reached from one dispatch
> + have disjoint cases, so selecting one is what trying them in turn does.
> */
> +
> +void
> +fn_splitter::emit_alternatives (dt_node *node, int indent, int depth,
> + const vec<dt_operand *> &gimple_exprs,
> + const vec<dt_operand *> &generic_exprs,
> + const vec<dt_operand *> &fns,
> + const vec<dt_operand *> &generic_fns,
> + const vec<dt_operand *> &preds,
> + const vec<dt_node *> &others)
> +{
> + auto_delete_vec<dt_operand_group> group_storage[ALT_PREDS];
> + collect_operand_groups (gimple_exprs, group_storage[ALT_GIMPLE_EXPRS]);
> + collect_operand_groups (fns, group_storage[ALT_FNS]);
> + collect_operand_groups (generic_exprs, group_storage[ALT_GENERIC_EXPRS]);
> + collect_operand_groups (generic_fns, group_storage[ALT_GENERIC_FNS]);
> +
> + auto_vec<alt_unit> units;
> + for (unsigned cat = ALT_GIMPLE_EXPRS; cat < ALT_PREDS; ++cat)
> + collect_alt_units (units, (alt_cat) cat, group_storage[cat]);
> +
> + /* Decide which function holds each group. Zero keeps it in the entry,
> + which is where a group stays when nothing has to be split out. */
> + for (unsigned cat = ALT_GIMPLE_EXPRS; cat < ALT_PREDS; ++cat)
> + {
> + owner[cat].truncate (0);
> + owner[cat].safe_grow_cleared (group_storage[cat].length ());
> + }
> +
> + const unsigned first_chunk = nchunks + 1;
> + unsigned have = 0;
> + bool split_p = false;
> + for (auto &u : units)
> + {
> + if (!split_p || full (have, u.nodes))
> + {
> + ++nchunks;
> + have = 0;
> + split_p = true;
> + }
> + owner[u.cat][u.index] = nchunks;
> + have += u.nodes;
> + }
> +
> + /* The entry function: the dispatch, and whatever cannot be reached
> + through it. */
> + dispatching = true;
> + node->gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
> + fns, generic_fns, preds, others, false, this);
> + dispatching = false;
> +
> + /* One function per group of alternatives that was split out. */
> + for (unsigned n = first_chunk; n <= nchunks; ++n)
> + {
> + auto_vec<dt_operand *> sub[ALT_PREDS];
> + for (unsigned cat = ALT_GIMPLE_EXPRS; cat < ALT_PREDS; ++cat)
> + for (unsigned i = 0; i < owner[cat].length (); ++i)
> + if (owner[cat][i] == n)
> + for (auto op : group_storage[cat][i]->operands)
> + sub[cat].safe_push (op);
> +
> + FILE *cf = open_chunk (n);
> + auto_vec<dt_operand *> no_preds;
> + auto_vec<dt_node *> no_others;
> + node->gen_kids_1 (cf, 2, gimple, depth, sub[ALT_GIMPLE_EXPRS],
> + sub[ALT_GENERIC_EXPRS], sub[ALT_FNS],
> + sub[ALT_GENERIC_FNS], no_preds, no_others, false);
> + close_chunk (cf);
> + }
> +}
> +
> /* Generate matching code for the children of the decision tree node. */
>
> void
> -dt_node::gen_kids (FILE *f, int indent, bool gimple, int depth, bool
> in_gcond)
> +dt_node::gen_kids (FILE *f, int indent, bool gimple, int depth, bool
> in_gcond,
> + fn_splitter *sp)
> {
> + /* A gcond matcher starts inside an open scope and is not a valid function
> + boundary. */
> + gcc_assert (!sp || !in_gcond);
> +
> auto_vec<dt_operand *> gimple_exprs;
> auto_vec<dt_operand *> generic_exprs;
> auto_vec<dt_operand *> fns;
> @@ -4332,10 +4649,25 @@ dt_node::gen_kids (FILE *f, int indent, bool gimple,
> int depth, bool in_gcond)
> Like DT_TRUE, DT_MATCH serves as a barrier as it can cause
> dependent matches to get out-of-order. Generate code now
> for what we have collected sofar. */
> - gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
> - fns, generic_fns, preds, others, in_gcond);
> + if (sp)
> + {
> + sp->emit_alternatives (this, indent, depth, gimple_exprs,
> + generic_exprs, fns, generic_fns, preds,
> + others);
> + }
> + else
> + gen_kids_1 (f, indent, gimple, depth, gimple_exprs,
> + generic_exprs, fns, generic_fns, preds, others,
> + in_gcond);
> /* And output the true operand itself. */
> - kids[i]->gen (f, indent, gimple, depth);
> + if (sp && kids[i]->type == dt_node::DT_TRUE)
> + {
> + /* A DT_TRUE node emits no code and opens no scope. Its children
> + remain valid function-level split points. */
> + kids[i]->gen_kids (f, indent, gimple, depth, false, sp);
> + }
> + else
> + kids[i]->gen (f, indent, gimple, depth);
> gimple_exprs.truncate (0);
> generic_exprs.truncate (0);
> fns.truncate (0);
> @@ -4348,8 +4680,12 @@ dt_node::gen_kids (FILE *f, int indent, bool gimple,
> int depth, bool in_gcond)
> }
>
> /* Generate code for the remains. */
> - gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
> - fns, generic_fns, preds, others, in_gcond);
> + if (sp)
> + sp->emit_alternatives (this, indent, depth, gimple_exprs, generic_exprs,
> + fns, generic_fns, preds, others);
> + else
> + gen_kids_1 (f, indent, gimple, depth, gimple_exprs, generic_exprs,
> + fns, generic_fns, preds, others, in_gcond);
> }
>
> /* Generate matching code for the children of the decision tree node. */
> @@ -4362,7 +4698,7 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool gimple,
> int depth,
> const vec<dt_operand *> &generic_fns,
> const vec<dt_operand *> &preds,
> const vec<dt_node *> &others,
> - bool in_gcond)
> + bool in_gcond, fn_splitter *sp)
> {
> char buf[128];
> char *kid_opname = buf;
> @@ -4430,10 +4766,17 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
> gimple, int depth,
> char code[64];
> snprintf (code, sizeof (code),
> "gimple_assign_rhs_code (_a%d)", depth);
> - for (auto group : gimple_expr_groups)
> + for (unsigned gi = 0; gi < gimple_expr_groups.length (); ++gi)
> {
> - emit_group_labels (f, indent, group);
> - emit_expr_group (f, indent, true, depth, code, group);
> + emit_group_labels (f, indent, gimple_expr_groups[gi]);
> + if (sp
> + && sp->same_chunk_p (fn_splitter::ALT_GIMPLE_EXPRS, gi,
> + gi + 1))
> + continue;
> + if (!sp || !sp->emit_dispatch (f, indent,
> + fn_splitter::ALT_GIMPLE_EXPRS, gi))
> + emit_expr_group (f, indent, true, depth, code,
> + gimple_expr_groups[gi]);
> }
> fprintf_indent (f, indent, "default:;\n");
> fprintf_indent (f, indent, "}\n");
> @@ -4475,10 +4818,15 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
> gimple, int depth,
> char code[64];
> snprintf (code, sizeof (code),
> "gimple_call_combined_fn (_c%d)", depth);
> - for (auto group : fn_groups)
> + for (unsigned fi = 0; fi < fn_groups.length (); ++fi)
> {
> - emit_group_labels (f, indent, group);
> - emit_fn_group (f, indent, true, depth, NULL, code, group);
> + emit_group_labels (f, indent, fn_groups[fi]);
> + if (sp && sp->same_chunk_p (fn_splitter::ALT_FNS, fi, fi + 1))
> + continue;
> + if (!sp || !sp->emit_dispatch (f, indent, fn_splitter::ALT_FNS,
> + fi))
> + emit_fn_group (f, indent, true, depth, NULL, code,
> + fn_groups[fi]);
> }
> fprintf_indent (f, indent, "default:;\n");
> fprintf_indent (f, indent, "}\n");
> @@ -4511,14 +4859,29 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
> gimple, int depth,
> char generic_code[160];
> snprintf (generic_code, sizeof (generic_code), "TREE_CODE (%s)",
> kid_opname);
> - for (auto group : generic_expr_groups)
> + /* The SSA_NAME group was emitted above, so it is not here to fall
> + through to, and a run of labels must not be merged across it. */
> + auto emitted_above_p = [&] (unsigned i)
> + {
> + return (gimple_match_p
> + && generic_expr_groups[i]->cases.length () == 1
> + && *generic_expr_groups[i]->cases[0] == SSA_NAME);
> + };
> + for (unsigned gei = 0; gei < generic_expr_groups.length (); ++gei)
> {
> - if (gimple_match_p
> - && group->cases.length () == 1
> - && *group->cases[0] == SSA_NAME)
> + dt_operand_group *group = generic_expr_groups[gei];
> + if (emitted_above_p (gei))
> continue;
> emit_group_labels (f, indent, group);
> - emit_expr_group (f, indent, gimple, depth, generic_code, group);
> + if (sp
> + && gei + 1 < generic_expr_groups.length ()
> + && !emitted_above_p (gei + 1)
> + && sp->same_chunk_p (fn_splitter::ALT_GENERIC_EXPRS, gei,
> + gei + 1))
> + continue;
> + if (!sp || !sp->emit_dispatch (f, indent,
> + fn_splitter::ALT_GENERIC_EXPRS, gei))
> + emit_expr_group (f, indent, gimple, depth, generic_code, group);
> }
> }
>
> @@ -4536,10 +4899,16 @@ dt_node::gen_kids_1 (FILE *f, int indent, bool
> gimple, int depth,
> char code[160];
> snprintf (code, sizeof (code), "get_call_combined_fn (%s)",
> kid_opname);
> - for (auto group : generic_fn_groups)
> + for (unsigned gfi = 0; gfi < generic_fn_groups.length (); ++gfi)
> {
> - emit_group_labels (f, indent, group);
> - emit_fn_group (f, indent, false, depth, kid_opname, code, group);
> + emit_group_labels (f, indent, generic_fn_groups[gfi]);
> + if (sp && sp->same_chunk_p (fn_splitter::ALT_GENERIC_FNS, gfi,
> + gfi + 1))
> + continue;
> + if (!sp || !sp->emit_dispatch (f, indent,
> + fn_splitter::ALT_GENERIC_FNS, gfi))
> + emit_fn_group (f, indent, false, depth, kid_opname, code,
> + generic_fn_groups[gfi]);
> }
> fprintf_indent (f, indent, "default:;\n");
>
> @@ -5324,6 +5693,14 @@ decision_tree::gen (vec <FILE *> &files, bool gimple)
> }
> fprintf (stderr, "removed %u duplicate tails\n", rcnt);
>
> + /* A file receives about ROOT->TOTAL_SIZE / NFILES decision-tree nodes.
> + Keep matcher functions small enough to spread them across the files.
> + Keep single-file output unchanged. */
> + unsigned nfiles = files.length ();
> + unsigned budget = (nfiles > 1
> + ? root->total_size / nfiles / MATCHER_FN_FILE_FRACTION
> + : 0);
> +
> for (unsigned n = 1; n <= 7; ++n)
> {
> bool has_kids_p = false;
> @@ -5366,7 +5743,17 @@ decision_tree::gen (vec <FILE *> &files, bool gimple)
> fprintf (f, "{\n");
> fprintf_indent (f, 2, "const bool debug_dump = "
> "dump_file && (dump_flags & TDF_FOLDING);\n");
> - dop->gen_kids (f, 2, gimple, 0, false);
> + if (budget && dop->total_size > budget)
> + {
> + char *fname = xasprintf ("genmatch_%s_simplify_%s",
> + gimple ? "gimple" : "generic",
> + e->operation->id);
> + fn_splitter sp (files, f, fname, gimple, n, budget);
> + dop->gen_kids (f, 2, gimple, 0, false, &sp);
> + free (fname);
> + }
> + else
> + dop->gen_kids (f, 2, gimple, 0, false);
> if (gimple)
> fprintf (f, " return false;\n");
> else
>
--
Richard Biener <[email protected]>
SUSE Software Solutions Germany GmbH,
Frankenstrasse 146, 90461 Nuernberg, Germany;
GF: Stefan Gaiser, Jochen Jaser, Abhinav Puri; (HRB 36809, AG Nuernberg)
