Hi, This is V2 of
https://gcc.gnu.org/pipermail/gcc-patches/2026-September/729527.html adding in-tree AutoFDO tools (--enable-afdo_tools) so GCC no longer depends on out-of-tree create_gcov / profile_merger for profile creation, merge, bootstrap and testing, and teaching cc1 to read the profiles those tools produce. afdo_create - perf samples -> GCOV AutoFDO profile afdo_merge - merge GCOV AutoFDO profiles afdo_dump - dump a profile It implements the design discussed in two earlier RFCs on gcc@: https://gcc.gnu.org/pipermail/gcc/2026-April/247990.html https://gcc.gnu.org/pipermail/gcc/2026-April/247992.html Thanks for the review of V1. Changes since V1 ================ Patch split The two very large patches are gone. The series is now 37 patches split by module: profile model, GCOV v3 I/O, DWARF reader, perf ABI and readers, branch-stack ingestion, ARM SPE, correlation, GCOV export, then the tool front ends, build wiring and documentation. Tool conventions - The tools run on the host and do not depend on the configured target, so they install unprefixed. The profile format depends on the target only through the GCOV version, never on the host. - --help and --version follow the GNU Coding Standards and report the --with-pkgversion and --with-bugurl strings. - Diagnostics use the gcc translation domain, and gcc/po/exgettext scans afdo_tools/src so they reach gcc.pot. - configure disables afdo_tools on non-Linux hosts and skips its checks for cross builds. The offline path has no host-architecture dependence; only live capture needs perf_event_open locally. - --enable-afdo_tools is documented in install.texi (patch 37). - The output format is selected with -gcov_version, so a newer tool can write profiles an older compiler reads. Bootstrap - CREATE_GCOV and PROFILE_MERGER keep their names and only change default value. The out-of-tree tools therefore still work: set those variables and the new AFDO_GCOV_VERSION=3. - When AFDO_GCOV_VERSION is unset the stage1 compiler is queried via the new -print-autofdo-gcov-version. - The compare_exclusions hunks are dropped. They should not be needed; afdo_tools compares cleanly because every stage builds it from the same sources. - profopt.exp no longer searches for the binary, it resolves a relative CREATE_GCOV against the GCC build directory. PATH alone does not work, because the top level exports a build-dir relative path while the testsuite runs from elsewhere. Collection Offline perf.data remains the primary path. Live capture (patch 18) is optional and can be dropped without affecting the rest of the series. perf pipe mode is worth a look for system-wide support. Fixes - Between-branch ranges up to 1 MB are now attributed rather than dropped at a much smaller cap, matching create_gcov. V1 lost mass for functions with large straight-line regions, which is the most likely reason for the large SPEC deltas seen in V1. - SPE AUX gaps and overlaps are counted and reported once instead of per record, which previously flooded stderr on large traces. - Concurrent libdw sessions are capped; each maps the whole binary and resident pages are charged per mapping. For some big binaries, around 6 GB, this cuts peak RSS by 38%, with no loss in wall time and identical output. - afdo_tools generates its own version.h and builds standalone. - Duplicate types, redundant work and dead code removed across the DWARF reader, symbol helpers and GCOV layers. Testing ======= Configure with --enable-afdo_tools. Run the tool tests from the afdo_tools build directory with: make check Regression tested on aarch64-linux-gnu with no new regressions. TODO / follow-ups: - Support for system-wide profiles. - Further performance improvements for afdo_create. Does this require Steering Committee approval in addition to the normal patch review? (cc Jeff Law) Thanks, Kugan --- [PATCH v2 01/37] [AFDO-TOOLS] Register the afdo_tools directory with the contrib scripts [PATCH v2 02/37] [AFDO-TOOLS] Add a ChangeLog file in the afdo_tools directory [PATCH v2 03/37] [AFDO-TOOLS] Add common file and profiling helpers [PATCH v2 04/37] [AFDO-TOOLS] Add the in-memory GCOV v3 profile model [PATCH v2 05/37] [AFDO-TOOLS] Add GCOV v3 byte I/O and profile summaries [PATCH v2 06/37] [AFDO-TOOLS] Add the GCOV v3 profile reader and writer [PATCH v2 07/37] [AFDO-TOOLS] Add the afdo_dump command [PATCH v2 08/37] [AFDO-TOOLS] Add GCOV profile merge support [PATCH v2 09/37] [AFDO-TOOLS] Define the sample-scoped DWARF reader API [PATCH v2 10/37] [AFDO-TOOLS] Add symbol, source-path and function lookup helpers [PATCH v2 11/37] [AFDO-TOOLS] Implement sample-scoped DWARF indexing [PATCH v2 12/37] [AFDO-TOOLS] Describe the Linux perf data ABI [PATCH v2 13/37] [AFDO-TOOLS] Define the profile reader interface [PATCH v2 14/37] [AFDO-TOOLS] Add offline perf.data parsing [PATCH v2 15/37] [AFDO-TOOLS] Decode perf event samples [PATCH v2 16/37] [AFDO-TOOLS] Add common branch-stack ingestion [PATCH v2 17/37] [AFDO-TOOLS] Read BRBE and LBR branch stacks [PATCH v2 18/37] [AFDO-TOOLS] Add live LBR profile capture [PATCH v2 19/37] [AFDO-TOOLS] Decode ARM SPE auxiliary packets [PATCH v2 20/37] [AFDO-TOOLS] Read ARM SPE profiles [PATCH v2 21/37] [AFDO-TOOLS] Add the offline profile reader [PATCH v2 22/37] [AFDO-TOOLS] Add the AutoFDO profile model [PATCH v2 23/37] [AFDO-TOOLS] Define the AutoFDO run options [PATCH v2 24/37] [AFDO-TOOLS] Bucket branch mass onto ELF functions [PATCH v2 25/37] [AFDO-TOOLS] Deposit source and call-edge profile mass [PATCH v2 26/37] [AFDO-TOOLS] Add correlation debug dumps [PATCH v2 27/37] [AFDO-TOOLS] Add the profile correlation driver [PATCH v2 28/37] [AFDO-TOOLS] Export correlated profiles as GCOV [PATCH v2 29/37] [AFDO-TOOLS] Add AutoFDO profiler orchestration [PATCH v2 30/37] [AFDO-TOOLS] Add the afdo_create command [PATCH v2 31/37] [AFDO-TOOLS] Configure and build the afdo_tools module [PATCH v2 32/37] [AFDO-TOOLS] Wire afdo_tools into the GCC build [PATCH v2 33/37] [AFDO-TOOLS] Add afdo-tools test support and tests [PATCH v2 34/37] [AFDO-TOOLS] Use in-tree tools for profile bootstrap [PATCH v2 35/37] [AFDO-TOOLS] Add GCOV v4 read/write machinery [PATCH v2 36/37] [AFDO-TOOLS] Teach cc1 to read GCOV AutoFDO profiles [PATCH v2 37/37] [AFDO-TOOLS] Document afdo_tools in install.texi Makefile.def | 7 + Makefile.in | 1317 ++- Makefile.tpl | 11 +- afdo_tools/ChangeLog | 6 + afdo_tools/Makefile.in | 223 + afdo_tools/aclocal.m4 | 19 + afdo_tools/config.h.in | 143 + afdo_tools/configure | 7744 +++++++++++++++++ afdo_tools/configure.ac | 93 + afdo_tools/src/afdo_assert.h | 37 + afdo_tools/src/afdo_create.cc | 519 ++ afdo_tools/src/afdo_dump.cc | 287 + afdo_tools/src/afdo_merge.cc | 161 + afdo_tools/src/arm_spe_aux_decode.cc | 531 ++ afdo_tools/src/arm_spe_aux_decode.h | 100 + afdo_tools/src/autofdo_correlate_bucket.cc | 476 + afdo_tools/src/autofdo_correlate_deposit.cc | 801 ++ afdo_tools/src/autofdo_correlate_detail.h | 94 + afdo_tools/src/autofdo_correlator.cc | 93 + afdo_tools/src/autofdo_correlator.h | 48 + afdo_tools/src/autofdo_debug_dump.cc | 249 + afdo_tools/src/autofdo_debug_dump.h | 41 + afdo_tools/src/autofdo_gcov_export.cc | 343 + afdo_tools/src/autofdo_gcov_export.h | 46 + afdo_tools/src/autofdo_model.cc | 324 + afdo_tools/src/autofdo_model.h | 190 + afdo_tools/src/autofdo_options.h | 42 + afdo_tools/src/autofdo_profiler.cc | 141 + afdo_tools/src/autofdo_profiler.h | 76 + afdo_tools/src/branch_stack_reader_base.cc | 221 + afdo_tools/src/branch_stack_reader_base.h | 96 + afdo_tools/src/dwarf_reader.cc | 2545 ++++++ afdo_tools/src/dwarf_reader.h | 167 + afdo_tools/src/dwarf_reader_impl.h | 359 + afdo_tools/src/dwarf_types.cc | 82 + afdo_tools/src/dwarf_types.h | 141 + afdo_tools/src/file_mmap.cc | 82 + afdo_tools/src/file_mmap.h | 56 + afdo_tools/src/gcov.h | 318 + afdo_tools/src/gcov_discriminator_encoding.h | 106 + afdo_tools/src/gcov_io.cc | 406 + afdo_tools/src/gcov_io.h | 89 + afdo_tools/src/gcov_merge.cc | 458 + afdo_tools/src/gcov_merge.h | 79 + afdo_tools/src/gcov_reader.cc | 97 + afdo_tools/src/gcov_reader.h | 165 + afdo_tools/src/gcov_reader_v3.cc | 451 + afdo_tools/src/gcov_reader_v4.cc | 952 ++ afdo_tools/src/gcov_summary.cc | 124 + afdo_tools/src/gcov_summary.h | 36 + afdo_tools/src/gcov_writer.cc | 75 + afdo_tools/src/gcov_writer.h | 200 + afdo_tools/src/gcov_writer_v3.cc | 262 + afdo_tools/src/gcov_writer_v4.cc | 802 ++ afdo_tools/src/hash_util.h | 38 + afdo_tools/src/live_branch_stack_reader.cc | 844 ++ afdo_tools/src/live_branch_stack_reader.h | 94 + afdo_tools/src/perf.h | 406 + afdo_tools/src/perf_branch_stack_reader.cc | 289 + afdo_tools/src/perf_branch_stack_reader.h | 69 + afdo_tools/src/perf_event_sample_util.cc | 381 + afdo_tools/src/perf_event_sample_util.h | 59 + afdo_tools/src/perf_file_reader.cc | 1101 +++ afdo_tools/src/perf_file_reader.h | 176 + afdo_tools/src/perf_spe_reader.cc | 488 ++ afdo_tools/src/perf_spe_reader.h | 68 + afdo_tools/src/profile_reader.cc | 230 + afdo_tools/src/profile_reader.h | 126 + afdo_tools/src/profile_timer.cc | 90 + afdo_tools/src/profile_timer.h | 40 + afdo_tools/src/symbol_util.cc | 260 + afdo_tools/src/symbol_util.h | 69 + .../testsuite/afdo-test/addr-mapping-basic.c | 52 + .../testsuite/afdo-test/afdo-dump-tree.c | 55 + .../afdo-test/afdo-fauto-profile-opt-dump.c | 56 + .../testsuite/afdo-test/afdo-fauto-profile.c | 53 + .../afdo-test/afdo-live-offline-parity.c | 32 + .../afdo-test/afdo-merge-dso-a-main.c | 9 + .../testsuite/afdo-test/afdo-merge-dso-a.c | 10 + .../afdo-test/afdo-merge-dso-b-main.c | 9 + .../testsuite/afdo-test/afdo-merge-dso-b.c | 10 + .../afdo-test/afdo-merge-multi-dso-profiles.c | 16 + .../afdo-test/afdo-merge-multi-workload.c | 60 + afdo_tools/testsuite/afdo-test/afdo-test.exp | 64 + .../testsuite/afdo-test/cse-inline-stacks.c | 58 + .../afdo-test/cse-source-duplication.c | 32 + .../testsuite/afdo-test/deep-inline-chain.c | 55 + .../duplicate-static-symbol-names-b.c | 17 + .../afdo-test/duplicate-static-symbol-names.c | 38 + .../afdo-test/dwarf-sym-perf-large.c | 48 + .../afdo-test/function-cloning-attribution.c | 58 + .../afdo-test/icall-target-addr-map.c | 26 + .../afdo-test/invalid-perf-data-corrupt.c | 8 + .../afdo-test/invalid-perf-data-empty.c | 8 + .../afdo-test/ip-only-gcov-generation.c | 35 + .../testsuite/afdo-test/ip-only-gcov-smoke.c | 31 + .../afdo-test/ip-only-noninline-hotfunc.c | 34 + .../afdo-test/ip-only-profile-driven-inline.c | 57 + .../afdo-test/ipa-icf-profile-alias.c | 46 + .../afdo-test/is-stmt-callsite-routing.c | 44 + .../afdo-test/lto-profile-application-tu2.c | 8 + .../afdo-test/lto-profile-application.c | 32 + .../afdo-test/overloaded-cpp-mangled-names.cc | 30 + .../afdo-test/pie-aslr-addr-normalization.c | 53 + .../profile-generation-determinism.c | 45 + .../afdo-test/recursive-fn-attribution.c | 26 + .../afdo-test/recursive-fn-self-edge.c | 32 + .../afdo-test/stripped-binary-rejection.c | 35 + .../afdo-test/system-wide-branch-stack.c | 35 + .../afdo-test/system-wide-perf-pipe.c | 29 + .../afdo-test/validate_addr_mapping.py | 310 + .../testsuite/afdo-test/validate_aslr_mmap.py | 101 + .../afdo-test/validate_determinism.py | 97 + .../testsuite/afdo-test/validate_dump_tree.py | 183 + .../testsuite/afdo-test/validate_merge.py | 47 + .../afdo-test/validate_system_wide.py | 87 + afdo_tools/testsuite/config/default.exp | 17 + afdo_tools/testsuite/lib/afdo.exp | 1606 ++++ afdo_tools/testsuite/lib/afdo_tools.exp | 40 + afdo_tools/version.h.in | 10 + configure | 26 + configure.ac | 20 + contrib/gcc-changelog/git_commit.py | 1 + contrib/update-copyright.py | 2 + gcc/Makefile.in | 38 +- gcc/auto-profile.cc | 453 +- gcc/auto-profile.h | 2 +- gcc/c/Make-lang.in | 6 +- gcc/common.opt | 24 + gcc/cp/Make-lang.in | 6 +- gcc/doc/install.texi | 22 +- gcc/doc/invoke.texi | 29 +- gcc/flag-types.h | 9 + gcc/lto/Make-lang.in | 6 +- gcc/po/exgettext | 4 +- gcc/testsuite/lib/profopt.exp | 17 +- 136 files changed, 31562 insertions(+), 306 deletions(-) create mode 100644 afdo_tools/ChangeLog create mode 100644 afdo_tools/Makefile.in create mode 100644 afdo_tools/aclocal.m4 create mode 100644 afdo_tools/config.h.in create mode 100755 afdo_tools/configure create mode 100644 afdo_tools/configure.ac create mode 100644 afdo_tools/src/afdo_assert.h create mode 100644 afdo_tools/src/afdo_create.cc create mode 100644 afdo_tools/src/afdo_dump.cc create mode 100644 afdo_tools/src/afdo_merge.cc create mode 100644 afdo_tools/src/arm_spe_aux_decode.cc create mode 100644 afdo_tools/src/arm_spe_aux_decode.h create mode 100644 afdo_tools/src/autofdo_correlate_bucket.cc create mode 100644 afdo_tools/src/autofdo_correlate_deposit.cc create mode 100644 afdo_tools/src/autofdo_correlate_detail.h create mode 100644 afdo_tools/src/autofdo_correlator.cc create mode 100644 afdo_tools/src/autofdo_correlator.h create mode 100644 afdo_tools/src/autofdo_debug_dump.cc create mode 100644 afdo_tools/src/autofdo_debug_dump.h create mode 100644 afdo_tools/src/autofdo_gcov_export.cc create mode 100644 afdo_tools/src/autofdo_gcov_export.h create mode 100644 afdo_tools/src/autofdo_model.cc create mode 100644 afdo_tools/src/autofdo_model.h create mode 100644 afdo_tools/src/autofdo_options.h create mode 100644 afdo_tools/src/autofdo_profiler.cc create mode 100644 afdo_tools/src/autofdo_profiler.h create mode 100644 afdo_tools/src/branch_stack_reader_base.cc create mode 100644 afdo_tools/src/branch_stack_reader_base.h create mode 100644 afdo_tools/src/dwarf_reader.cc create mode 100644 afdo_tools/src/dwarf_reader.h create mode 100644 afdo_tools/src/dwarf_reader_impl.h create mode 100644 afdo_tools/src/dwarf_types.cc create mode 100644 afdo_tools/src/dwarf_types.h create mode 100644 afdo_tools/src/file_mmap.cc create mode 100644 afdo_tools/src/file_mmap.h create mode 100644 afdo_tools/src/gcov.h create mode 100644 afdo_tools/src/gcov_discriminator_encoding.h create mode 100644 afdo_tools/src/gcov_io.cc create mode 100644 afdo_tools/src/gcov_io.h create mode 100644 afdo_tools/src/gcov_merge.cc create mode 100644 afdo_tools/src/gcov_merge.h create mode 100644 afdo_tools/src/gcov_reader.cc create mode 100644 afdo_tools/src/gcov_reader.h create mode 100644 afdo_tools/src/gcov_reader_v3.cc create mode 100644 afdo_tools/src/gcov_reader_v4.cc create mode 100644 afdo_tools/src/gcov_summary.cc create mode 100644 afdo_tools/src/gcov_summary.h create mode 100644 afdo_tools/src/gcov_writer.cc create mode 100644 afdo_tools/src/gcov_writer.h create mode 100644 afdo_tools/src/gcov_writer_v3.cc create mode 100644 afdo_tools/src/gcov_writer_v4.cc create mode 100644 afdo_tools/src/hash_util.h create mode 100644 afdo_tools/src/live_branch_stack_reader.cc create mode 100644 afdo_tools/src/live_branch_stack_reader.h create mode 100644 afdo_tools/src/perf.h create mode 100644 afdo_tools/src/perf_branch_stack_reader.cc create mode 100644 afdo_tools/src/perf_branch_stack_reader.h create mode 100644 afdo_tools/src/perf_event_sample_util.cc create mode 100644 afdo_tools/src/perf_event_sample_util.h create mode 100644 afdo_tools/src/perf_file_reader.cc create mode 100644 afdo_tools/src/perf_file_reader.h create mode 100644 afdo_tools/src/perf_spe_reader.cc create mode 100644 afdo_tools/src/perf_spe_reader.h create mode 100644 afdo_tools/src/profile_reader.cc create mode 100644 afdo_tools/src/profile_reader.h create mode 100644 afdo_tools/src/profile_timer.cc create mode 100644 afdo_tools/src/profile_timer.h create mode 100644 afdo_tools/src/symbol_util.cc create mode 100644 afdo_tools/src/symbol_util.h create mode 100644 afdo_tools/testsuite/afdo-test/addr-mapping-basic.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-dump-tree.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-fauto-profile-opt-dump.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-fauto-profile.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-live-offline-parity.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-merge-dso-a-main.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-merge-dso-a.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-merge-dso-b-main.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-merge-dso-b.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-merge-multi-dso-profiles.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-merge-multi-workload.c create mode 100644 afdo_tools/testsuite/afdo-test/afdo-test.exp create mode 100644 afdo_tools/testsuite/afdo-test/cse-inline-stacks.c create mode 100644 afdo_tools/testsuite/afdo-test/cse-source-duplication.c create mode 100644 afdo_tools/testsuite/afdo-test/deep-inline-chain.c create mode 100644 afdo_tools/testsuite/afdo-test/duplicate-static-symbol-names-b.c create mode 100644 afdo_tools/testsuite/afdo-test/duplicate-static-symbol-names.c create mode 100644 afdo_tools/testsuite/afdo-test/dwarf-sym-perf-large.c create mode 100644 afdo_tools/testsuite/afdo-test/function-cloning-attribution.c create mode 100644 afdo_tools/testsuite/afdo-test/icall-target-addr-map.c create mode 100644 afdo_tools/testsuite/afdo-test/invalid-perf-data-corrupt.c create mode 100644 afdo_tools/testsuite/afdo-test/invalid-perf-data-empty.c create mode 100644 afdo_tools/testsuite/afdo-test/ip-only-gcov-generation.c create mode 100644 afdo_tools/testsuite/afdo-test/ip-only-gcov-smoke.c create mode 100644 afdo_tools/testsuite/afdo-test/ip-only-noninline-hotfunc.c create mode 100644 afdo_tools/testsuite/afdo-test/ip-only-profile-driven-inline.c create mode 100644 afdo_tools/testsuite/afdo-test/ipa-icf-profile-alias.c create mode 100644 afdo_tools/testsuite/afdo-test/is-stmt-callsite-routing.c create mode 100644 afdo_tools/testsuite/afdo-test/lto-profile-application-tu2.c create mode 100644 afdo_tools/testsuite/afdo-test/lto-profile-application.c create mode 100644 afdo_tools/testsuite/afdo-test/overloaded-cpp-mangled-names.cc create mode 100644 afdo_tools/testsuite/afdo-test/pie-aslr-addr-normalization.c create mode 100644 afdo_tools/testsuite/afdo-test/profile-generation-determinism.c create mode 100644 afdo_tools/testsuite/afdo-test/recursive-fn-attribution.c create mode 100644 afdo_tools/testsuite/afdo-test/recursive-fn-self-edge.c create mode 100644 afdo_tools/testsuite/afdo-test/stripped-binary-rejection.c create mode 100644 afdo_tools/testsuite/afdo-test/system-wide-branch-stack.c create mode 100644 afdo_tools/testsuite/afdo-test/system-wide-perf-pipe.c create mode 100755 afdo_tools/testsuite/afdo-test/validate_addr_mapping.py create mode 100644 afdo_tools/testsuite/afdo-test/validate_aslr_mmap.py create mode 100644 afdo_tools/testsuite/afdo-test/validate_determinism.py create mode 100644 afdo_tools/testsuite/afdo-test/validate_dump_tree.py create mode 100644 afdo_tools/testsuite/afdo-test/validate_merge.py create mode 100644 afdo_tools/testsuite/afdo-test/validate_system_wide.py create mode 100644 afdo_tools/testsuite/config/default.exp create mode 100644 afdo_tools/testsuite/lib/afdo.exp create mode 100644 afdo_tools/testsuite/lib/afdo_tools.exp create mode 100644 afdo_tools/version.h.in -- 2.34.1
