[compiler-rt] [clang] [clang-tools-extra] [PGO] Reland PGO's Counter Reset and File Dumping APIs #76471 (PR #78285)
qiongsiwu wrote: Thanks for to comments/suggestions @w2yehia ! The test is modified so that 1. The profile files are generated. 2. The test does not support Windows to avoid known limitations on Windows. https://github.com/llvm/llvm-project/pull/78285 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[compiler-rt] [clang] [clang-tools-extra] [PGO] Reland PGO's Counter Reset and File Dumping APIs #76471 (PR #78285)
https://github.com/qiongsiwu updated
https://github.com/llvm/llvm-project/pull/78285
>From ebae7155814ad83ebd1a0159b86550c14c72b2b6 Mon Sep 17 00:00:00 2001
From: Qiongsi Wu
Date: Fri, 12 Jan 2024 11:45:50 -0500
Subject: [PATCH 1/2] Reland https://github.com/llvm/llvm-project/pull/76471
---
.../ExpandModularHeadersPPCallbacks.cpp | 2 +-
clang/docs/UsersManual.rst| 104 ++
clang/include/clang/Basic/CodeGenOptions.h| 3 +
clang/include/clang/Frontend/Utils.h | 4 +-
clang/lib/Frontend/CompilerInstance.cpp | 2 +-
clang/lib/Frontend/InitPreprocessor.cpp | 23 +++-
clang/test/Profile/c-general.c| 10 ++
compiler-rt/include/CMakeLists.txt| 1 +
.../include/profile/instr_prof_interface.h| 92
compiler-rt/lib/profile/InstrProfiling.h | 61 ++
.../profile/Linux/instrprof-weak-symbol.c | 16 +++
compiler-rt/test/profile/instrprof-api.c | 46
12 files changed, 307 insertions(+), 57 deletions(-)
create mode 100644 compiler-rt/include/profile/instr_prof_interface.h
create mode 100644 compiler-rt/test/profile/Linux/instrprof-weak-symbol.c
create mode 100644 compiler-rt/test/profile/instrprof-api.c
diff --git a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
index e414ac8c770508f..5ecd4fb19131e43 100644
--- a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -100,7 +100,7 @@
ExpandModularHeadersPPCallbacks::ExpandModularHeadersPPCallbacks(
/*OwnsHeaderSearch=*/false);
PP->Initialize(Compiler.getTarget(), Compiler.getAuxTarget());
InitializePreprocessor(*PP, *PO, Compiler.getPCHContainerReader(),
- Compiler.getFrontendOpts());
+ Compiler.getFrontendOpts(),
Compiler.getCodeGenOpts());
ApplyHeaderSearchOptions(*HeaderInfo, *HSO, LangOpts,
Compiler.getTarget().getTriple());
}
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 881d903d91a7ea4..ff2d4a68b8e55a7 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2809,6 +2809,110 @@ indexed format, regardeless whether it is produced by
frontend or the IR pass.
overhead. ``prefer-atomic`` will be transformed to ``atomic`` when supported
by the target, or ``single`` otherwise.
+Fine Tuning Profile Collection
+^^
+
+The PGO infrastructure provides user program knobs to fine tune profile
+collection. Specifically, the PGO runtime provides the following functions
+that can be used to control the regions in the program where profiles should
+be collected.
+
+ * ``void __llvm_profile_set_filename(const char *Name)``: changes the name of
+ the profile file to ``Name``.
+ * ``void __llvm_profile_reset_counters(void)``: resets all counters to zero.
+ * ``int __llvm_profile_dump(void)``: write the profile data to disk.
+ * ``int __llvm_orderfile_dump(void)``: write the order file to disk.
+
+For example, the following pattern can be used to skip profiling program
+initialization, profile two specific hot regions, and skip profiling program
+cleanup:
+
+.. code-block:: c
+
+int main() {
+ initialize();
+
+ // Reset all profile counters to 0 to omit profile collected during
+ // initialize()'s execution.
+ __llvm_profile_reset_counters();
+ ... hot region 1
+ // Dump the profile for hot region 1.
+ __llvm_profile_set_filename("region1.profraw");
+ __llvm_profile_dump();
+
+ // Reset counters before proceeding to hot region 2.
+ __llvm_profile_reset_counters();
+ ... hot region 2
+ // Dump the profile for hot region 2.
+ __llvm_profile_set_filename("region2.profraw");
+ __llvm_profile_dump();
+
+ // Since the profile has been dumped, no further profile data
+ // will be collected beyond the above __llvm_profile_dump().
+ cleanup();
+ return 0;
+}
+
+These APIs' names can be introduced to user programs in two ways.
+They can be declared as weak symbols on platforms which support
+treating weak symbols as ``null`` during linking. For example, the user can
+have
+
+.. code-block:: c
+
+__attribute__((weak)) int __llvm_profile_dump(void);
+
+// Then later in the same source file
+if (__llvm_profile_dump)
+ if (__llvm_profile_dump() != 0) { ... }
+// The first if condition tests if the symbol is actually defined.
+// Profile dumping only happens if the symbol is defined. Hence,
+// the user program works correctly during normal (not profile-generate)
+// executions.
+
+Alternatively, the user program can include the header
+``profile/instr_prof_interface.h``, which contains the API names. For example,
+
+..
[compiler-rt] [clang] [clang-tools-extra] [PGO] Reland PGO's Counter Reset and File Dumping APIs #76471 (PR #78285)
llvmbot wrote:
@llvm/pr-subscribers-pgo
Author: Qiongsi Wu (qiongsiwu)
Changes
https://github.com/llvm/llvm-project/pull/76471 caused buildbot failure on
Windows. For more details, see
https://github.com/llvm/llvm-project/issues/77546.
This PR revises the test and relands
https://github.com/llvm/llvm-project/pull/76471.
---
Patch is 21.08 KiB, truncated to 20.00 KiB below, full version:
https://github.com/llvm/llvm-project/pull/78285.diff
13 Files Affected:
- (modified) clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
(+1-1)
- (modified) clang/docs/UsersManual.rst (+104)
- (modified) clang/include/clang/Basic/CodeGenOptions.h (+3)
- (modified) clang/include/clang/Frontend/Utils.h (+3-1)
- (modified) clang/lib/Frontend/CompilerInstance.cpp (+1-1)
- (modified) clang/lib/Frontend/InitPreprocessor.cpp (+19-4)
- (modified) clang/test/Profile/c-general.c (+10)
- (modified) compiler-rt/include/CMakeLists.txt (+1)
- (added) compiler-rt/include/profile/instr_prof_interface.h (+92)
- (modified) compiler-rt/lib/profile/InstrProfiling.h (+11-50)
- (added) compiler-rt/test/profile/Inputs/instrprof-api.c.profdata ()
- (added) compiler-rt/test/profile/Linux/instrprof-weak-symbol.c (+16)
- (added) compiler-rt/test/profile/instrprof-api.c (+39)
``diff
diff --git a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
index e414ac8c770508f..5ecd4fb19131e43 100644
--- a/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
+++ b/clang-tools-extra/clang-tidy/ExpandModularHeadersPPCallbacks.cpp
@@ -100,7 +100,7 @@
ExpandModularHeadersPPCallbacks::ExpandModularHeadersPPCallbacks(
/*OwnsHeaderSearch=*/false);
PP->Initialize(Compiler.getTarget(), Compiler.getAuxTarget());
InitializePreprocessor(*PP, *PO, Compiler.getPCHContainerReader(),
- Compiler.getFrontendOpts());
+ Compiler.getFrontendOpts(),
Compiler.getCodeGenOpts());
ApplyHeaderSearchOptions(*HeaderInfo, *HSO, LangOpts,
Compiler.getTarget().getTriple());
}
diff --git a/clang/docs/UsersManual.rst b/clang/docs/UsersManual.rst
index 7c30570437e8b01..27c629a1ffc6576 100644
--- a/clang/docs/UsersManual.rst
+++ b/clang/docs/UsersManual.rst
@@ -2809,6 +2809,110 @@ indexed format, regardeless whether it is produced by
frontend or the IR pass.
overhead. ``prefer-atomic`` will be transformed to ``atomic`` when supported
by the target, or ``single`` otherwise.
+Fine Tuning Profile Collection
+^^
+
+The PGO infrastructure provides user program knobs to fine tune profile
+collection. Specifically, the PGO runtime provides the following functions
+that can be used to control the regions in the program where profiles should
+be collected.
+
+ * ``void __llvm_profile_set_filename(const char *Name)``: changes the name of
+ the profile file to ``Name``.
+ * ``void __llvm_profile_reset_counters(void)``: resets all counters to zero.
+ * ``int __llvm_profile_dump(void)``: write the profile data to disk.
+ * ``int __llvm_orderfile_dump(void)``: write the order file to disk.
+
+For example, the following pattern can be used to skip profiling program
+initialization, profile two specific hot regions, and skip profiling program
+cleanup:
+
+.. code-block:: c
+
+int main() {
+ initialize();
+
+ // Reset all profile counters to 0 to omit profile collected during
+ // initialize()'s execution.
+ __llvm_profile_reset_counters();
+ ... hot region 1
+ // Dump the profile for hot region 1.
+ __llvm_profile_set_filename("region1.profraw");
+ __llvm_profile_dump();
+
+ // Reset counters before proceeding to hot region 2.
+ __llvm_profile_reset_counters();
+ ... hot region 2
+ // Dump the profile for hot region 2.
+ __llvm_profile_set_filename("region2.profraw");
+ __llvm_profile_dump();
+
+ // Since the profile has been dumped, no further profile data
+ // will be collected beyond the above __llvm_profile_dump().
+ cleanup();
+ return 0;
+}
+
+These APIs' names can be introduced to user programs in two ways.
+They can be declared as weak symbols on platforms which support
+treating weak symbols as ``null`` during linking. For example, the user can
+have
+
+.. code-block:: c
+
+__attribute__((weak)) int __llvm_profile_dump(void);
+
+// Then later in the same source file
+if (__llvm_profile_dump)
+ if (__llvm_profile_dump() != 0) { ... }
+// The first if condition tests if the symbol is actually defined.
+// Profile dumping only happens if the symbol is defined. Hence,
+// the user program works correctly during normal (not profile-generate)
+// executions.
+
+Alternatively, the user program can include the header
+``profile/instr_prof_interface.h``, which
