On Mon, Jun 8, 2026 at 12:38 AM Kyrylo Tkachov <[email protected]> wrote:
>
> Hi all,
>
> In the sink pass, if-convert a diamond that selects between two loads at
> data-dependent addresses (PHI <*P, *Q>) into a single reg-offset load behind
> branchless selects, reusing phi-opt's factoring helpers (now exported) to
> common
> the load and its address. Guarded so it only fires on non-vectorisable
> in-loop
> load-address recurrences, leaving affine vectorisable loops alone. This was
> needed
> to avoid regressing the marian benchmark in SPEC2026 that otherwise ended up
> using
> expensive gathers to vectorise. We can look to relax this condition in the
> future
> for cases where we think it's beneficial.
>
> The load is commoned only when both arms are guaranteed to reduce to pure
> speculatable scalar (scc_arms_speculatable_p), so the branchless finish always
> succeeds and no commoned-but-still-branching diamond is ever left behind. The
> recurrence walk is bounded by a new --param=sink-diamond-recurrence-limit
> (default 256). factor_out_conditional_load checks the MEM_REF operand-1
> types with
> alias_ptr_types_compatible_p so loads with incompatible TBAA are never
> commoned.
>
> With this patch the Snappy hot loop gets if-converted optimally and
> performance
> improves by 25% on my aarch64 machine, making it a bit better than what LLVM
> gets
> today.
>
> It depends on Andrea's phiopt patch [1] that generalizes commoning out
> operations beyond unary so they need to go in one after the other.
>
> The code goes from:
> .L8: ; ---type==0 arm
> lsr x3, x1, 2 ; tag >> 2
> add x1, x0, x3
> add x3, x3, 2
> add x0, x0, x3 ; ip += (tag>>2)+2
> ldrb w1, [x1, 1] ; LOAD #1: tag = ip[(tag>>2)+1]
> cmp x2, x0
> bls .L2
> .L5: ; loop head / else arm
> ands x3, x1, 3 ; type = tag & 3
> beq .L8 ; <-- DATA-DEPENDENT BRANCH (mispredicts)
> ldrb w1, [x0, x3] ; LOAD #2: tag = ip[type]
> add w3, w3, 1
> add x0, x0, x3 ; ip += type+1
> cmp x2, x0
> bhi .L5
>
> to:
> .L3: ; --- single loop body, no diamond branch
> lsr x4, x1, 2 ; tag >> 2
> ands x3, x1, 3 ; type = tag & 3 (Z = type==0)
> csinc x1, x3, x4, ne ; offset = (type!=0) ? type : (tag>>2)+1
> ldrb w1, [x0, x1] ; ONE reg-offset LOAD: tag = ip[offset]
> add x4, x4, 2
> csinc x3, x4, x3, eq ; advance = (type==0) ? (tag>>2)+2 : type+1
> add x0, x0, x3 ; ip += advance
> cmp x2, x0
> bhi .L3
>
> crucially avoiding the badly-predicted branch.
>
> Bootstrapped and tested on aarch64-none-linux-gnu.
>
> Signed-off-by: Kyrylo Tkachov <[email protected]>
>
> gcc/
>
> PR tree-optimization/125557
> * params.opt (sink-diamond-recurrence-limit): New param.
Just some quick review notes:
params should be documented in doc/params.texi .
> * tree-ssa-phiopt.h (factor_out_conditional_load): Declare.
> (factor_out_conditional_operation): Declare.
> * tree-ssa-phiopt.cc: Include "alias.h".
> (remove_factored_arm_defs): New.
> (factor_out_conditional_load): New.
> (factor_out_conditional_operation): No longer static.
> * tree-ssa-sink.cc (scc_recurrence_p, scc_arms_speculatable_p)
> (scc_try_ifconvert, sink_common_computations_to_bb): New.
> (pass_sink_code::execute): Call it at the early sink.
I have an already approved patch which handles:
```
if (a) {
if (c) t = abs<d>
else t = abs<e>
}
```
which seemingly will break the sink code here. The only reason why it
was not applied yet is because I am still convincing myself that the
order of bb processing in phiopt will work.
That is it handles number of operands to PHI != 2.
So can you split this into 2 patches. One for the phiopt piece and one
for the sink code?
>
> gcc/testsuite/
>
> PR tree-optimization/125557
> * gcc.dg/tree-ssa/scc-diamond-1.c: New test.
> * gcc.dg/tree-ssa/scc-diamond-3.c: New test.
> * gcc.target/aarch64/scc-diamond-2.c: New test.
>
> [1] https://gcc.gnu.org/pipermail/gcc-patches/2026-June/719612.html
>
