On 10/7/26 7:21 AM, [email protected] wrote:
From: Kyrylo Tkachov <[email protected]> Postreload GCSE looks for an earlier load or store of the same memory in each predecessor of the block of a load, or along a chain of single predecessors. It then adds copies or loads on the incoming edges and deletes the load. It does not see occurrences before a join, and it gives up if the block uses the destination of the load before the load. For example: bb2: (set (mem:V4SI (reg:DI sp)) (reg:V4SI v0)) bb3, bb4: no change to v0 or to the slot bb5: (set (reg:V4SI v2) (plus:V4SI (reg:V4SI v1) (reg:V4SI v3))) (set (reg:V4SI v1) (mem:V4SI (reg:DI sp))) The load in bb5 can become: (set (reg:V4SI v1) (reg:V4SI v0)) Compute which pairs of a memory expression and a hard register that holds its value are available at the start of each block, with compute_available. An available occurrence whose register does not change later in its block generates its pair. A block kills a pair if the expression is not transparent through the block or if the block sets the register. compute_hash_table records the registers that each block sets, and treats partial call clobbers as full clobbers. Blocks with abnormal predecessors start with no pairs. In the scan for partially redundant loads, first try to replace each load with a copy from a register that holds the value at the start of the block and that the block does not change before the load. The load keeps its destination, so the occurrences in the hash table stay valid. Also scan cold blocks, but there only replace loads, and only if the copy is no larger than the load. The pairs are only computed when transparency is. The bitmaps take (4 * blocks + FIRST_PSEUDO_REGISTER) * pairs bits. If that exceeds max-gcse-memory, the analysis is skipped with a note in the dump file. Building the kill sets takes time linear in the size of the transparency bitmaps and the new bitmaps. compute_available needs all blocks to be reachable, so delete unreachable blocks first, as PRE and cprop do. On the motivating testcase from dav1d at -O3 -fno-ivopts, this patch removes 27% of the SVE vector loads (ldr z) that remain after patch 2. With patch 2, that is 33% of the loads in base, all of them reloads of spill slots. At -O3 it forwards two scalar reloads. Compile time, as instruction counts of the aarch64 cc1 on an x86_64 host: - diamonds-N, generated RTL with one stack slot that is reloaded after each of N joins: v1 of this patch walked the CFG for each load, and the count grew from +48% over base at N = 1024 to +804% at N = 12288. This version stays within 1% of base. - slots-N, with about N pairs in 3N blocks: within 0.3% of base up to N = 4096, and +1.4% at N = 8192, where the new bitmaps need 97 MB, less than the default max-gcse-memory. The existing transparency computation takes most of the time for these inputs. - At -O3 the series adds 0.05% to 0.08% for combine.ii, dwarf2out.ii, fold-const.ii and sqlite3.i. Bootstrapped and tested on aarch64-none-linux-gnu and x86_64-linux. Ok for trunk? Thanks, Kyrill Cc: Jeff Law <[email protected]> gcc/ChangeLog: * postreload-gcse.cc: Include cfgcleanup.h and lcm.h. Update the overview comment. (stats): Add loads_forwarded. (struct expr): Add first_fact and num_facts. (struct occr): Add reg_available_p. (regs_set_in_block, modified_regs, fact_regs, fact_avin): New variables. (reg_changed_after_insn_p): Declare. (alloc_mem, free_mem): Allocate and free modified_regs. (insert_expr_in_table): Set reg_available_p. (record_last_reg_set_info, record_last_reg_set_info_regno): Add the register to regs_set_in_block. (reset_opr_set_tables): Clear regs_set_in_block. (compute_hash_table): Record regs_set_in_block in modified_regs. (lookup_load_in_table): New function, split out of... (eliminate_partially_redundant_loads): ...here. Try replace_fully_redundant_load first. In cold blocks, only do that. (get_fact, compute_fact_availability): New functions. (replace_fully_redundant_load): Likewise. (gcse_after_reload_main): Delete unreachable blocks. Call compute_fact_availability. Free fact_avin and fact_regs. Report the number of forwarded loads. gcc/testsuite/ChangeLog: * gcc.dg/rtl/aarch64/postreload-gcse-available.c: New test. * gcc.dg/rtl/aarch64/postreload-gcse-clobber.c: Likewise. * gcc.dg/rtl/aarch64/postreload-gcse-forward.c: Likewise. * gcc.dg/rtl/aarch64/postreload-gcse-kill.c: Likewise. * gcc.dg/rtl/aarch64/postreload-gcse-memory.c: Likewise. * gcc.target/aarch64/sve/spill-gcse.c: Likewise. * gcc.target/aarch64/sve/spill-gcse-cold.c: Likewise. Signed-off-by: Kyrylo Tkachov <[email protected]>
OK when prereqs are in. jeff
