[PATCH] D84246: [clang][LTO] Pass-through remarks options and set auto hotness threshold

2020-08-11 Thread Wei Wang via Phabricator via cfe-commits 
weiwang abandoned this revision.
weiwang added a comment.

Diff has been split into 3 smaller ones:

1. [lld] Enable remarks hotness filtering in lld: 
https://reviews.llvm.org/D85809
2. [clang] Pass-through remarks options to lld: https://reviews.llvm.org/D85810
3. [remarks] Optimization remarks hotness filtering from profile summary: 
https://reviews.llvm.org/D85808


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84246/new/

https://reviews.llvm.org/D84246

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84246: [clang][LTO] Pass-through remarks options and set auto hotness threshold

2020-08-03 Thread Wei Wang via Phabricator via cfe-commits 
weiwang added a comment.

still waiting for inputs. Thanks.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84246/new/

https://reviews.llvm.org/D84246

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84246: [clang][LTO] Pass-through remarks options and set auto hotness threshold

2020-07-27 Thread Wei Wang via Phabricator via cfe-commits 
weiwang updated this revision to Diff 281077.
weiwang added a comment.

Fix msvc build failure


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84246/new/

https://reviews.llvm.org/D84246

Files:
  clang/include/clang/Basic/CodeGenOptions.def
  clang/include/clang/Basic/CodeGenOptions.h
  clang/include/clang/Basic/DiagnosticDriverKinds.td
  clang/include/clang/Driver/Driver.h
  clang/include/clang/Driver/Options.td
  clang/lib/Driver/Driver.cpp
  clang/lib/Driver/ToolChains/CommonArgs.cpp
  clang/lib/Frontend/CompilerInvocation.cpp
  clang/test/Driver/remarks-pass-through.c
  lld/ELF/Config.h
  lld/ELF/Driver.cpp
  lld/ELF/LTO.cpp
  lld/ELF/Options.td
  lld/test/ELF/lto/opt-remarks.ll
  llvm/include/llvm/Analysis/ProfileSummaryInfo.h
  llvm/include/llvm/IR/LLVMContext.h
  llvm/include/llvm/IR/LLVMRemarkStreamer.h
  llvm/include/llvm/IR/Module.h
  llvm/include/llvm/LTO/Config.h
  llvm/include/llvm/LTO/LTO.h
  llvm/include/llvm/Remarks/HotnessThresholdParser.h
  llvm/lib/Analysis/OptimizationRemarkEmitter.cpp
  llvm/lib/IR/LLVMContext.cpp
  llvm/lib/IR/LLVMContextImpl.h
  llvm/lib/IR/LLVMRemarkStreamer.cpp
  llvm/lib/IR/Module.cpp
  llvm/lib/LTO/LTO.cpp
  llvm/lib/LTO/LTOBackend.cpp
  llvm/lib/LTO/LTOCodeGenerator.cpp
  llvm/lib/LTO/ThinLTOCodeGenerator.cpp
  llvm/test/LTO/X86/diagnostic-handler-remarks-with-hotness.ll
  llvm/test/Other/optimization-remarks-auto.ll
  llvm/test/Transforms/SampleProfile/Inputs/remarks-hotness.prof
  llvm/test/Transforms/SampleProfile/remarks-hotness.ll
  llvm/tools/llc/llc.cpp
  llvm/tools/opt/opt.cpp

Index: llvm/tools/opt/opt.cpp
===
--- llvm/tools/opt/opt.cpp
+++ llvm/tools/opt/opt.cpp
@@ -38,6 +38,7 @@
 #include "llvm/LinkAllIR.h"
 #include "llvm/LinkAllPasses.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
@@ -272,11 +273,13 @@
 cl::desc("With PGO, include profile count in optimization remarks"),
 cl::Hidden);
 
-static cl::opt
-RemarksHotnessThreshold("pass-remarks-hotness-threshold",
-cl::desc("Minimum profile count required for "
- "an optimization remark to be output"),
-cl::Hidden);
+static cl::opt, false, remarks::HotnessThresholdParser>
+RemarksHotnessThreshold(
+"pass-remarks-hotness-threshold",
+cl::desc("Minimum profile count required for "
+ "an optimization remark to be output. "
+ "Use 'auto' to apply the threshold from profile summary."),
+cl::value_desc("N or 'auto'"), cl::init(0), cl::Hidden);
 
 static cl::opt
 RemarksFilename("pass-remarks-output",
Index: llvm/tools/llc/llc.cpp
===
--- llvm/tools/llc/llc.cpp
+++ llvm/tools/llc/llc.cpp
@@ -37,6 +37,7 @@
 #include "llvm/InitializePasses.h"
 #include "llvm/MC/SubtargetFeature.h"
 #include "llvm/Pass.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/CommandLine.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
@@ -142,11 +143,13 @@
 cl::desc("With PGO, include profile count in optimization remarks"),
 cl::Hidden);
 
-static cl::opt
-RemarksHotnessThreshold("pass-remarks-hotness-threshold",
-cl::desc("Minimum profile count required for "
- "an optimization remark to be output"),
-cl::Hidden);
+static cl::opt, false, remarks::HotnessThresholdParser>
+RemarksHotnessThreshold(
+"pass-remarks-hotness-threshold",
+cl::desc("Minimum profile count required for "
+ "an optimization remark to be output. "
+ "Use 'auto' to apply the threshold from profile summary."),
+cl::value_desc("N or 'auto'"), cl::init(0), cl::Hidden);
 
 static cl::opt
 RemarksFilename("pass-remarks-output",
Index: llvm/test/Transforms/SampleProfile/remarks-hotness.ll
===
--- /dev/null
+++ llvm/test/Transforms/SampleProfile/remarks-hotness.ll
@@ -0,0 +1,96 @@
+;; This test verifies 'auto' hotness threshold when profile file is provided.
+;;
+;; new PM
+; RUN: rm -f %t.yaml %t.hot.yaml
+; RUN: opt %s --enable-new-pm --passes='sample-profile,cgscc(inline)' \
+; RUN: --sample-profile-file=%S/Inputs/remarks-hotness.prof \
+; RUN: -S --pass-remarks-filter=inline --pass-remarks-output=%t.yaml \
+; RUN: -pass-remarks-with-hotness --disable-output
+; RUN: FileCheck %s -check-prefix=YAML-PASS < %t.yaml
+; RUN: FileCheck %s -check-prefix=YAML-MISS < %t.yaml
+
+;; test 'auto' threshold
+; RUN: opt %s --enable-new-pm --passes='sample-profile,cgscc(inline)' \
+; RUN:

[PATCH] D84246: [clang][LTO] Pass-through remarks options and set auto hotness threshold

2020-07-27 Thread Wei Wang via Phabricator via cfe-commits 
weiwang added a comment.

I put everything together in a single diff so that it is easier to get the 
whole idea. The change itself is too big to go in as a single diff, and I'd 
like to get inputs on how to split it.


Repository:
  rG LLVM Github Monorepo

CHANGES SINCE LAST ACTION
  https://reviews.llvm.org/D84246/new/

https://reviews.llvm.org/D84246



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[PATCH] D84246: [clang][LTO] Pass-through remarks options and set auto hotness threshold

2020-07-21 Thread Wei Wang via Phabricator via cfe-commits 
weiwang created this revision.
Herald added subscribers: llvm-commits, cfe-commits, dang, dexonsmith, 
steven_wu, MaskRay, hiraditya, eraman, arichardson, inglorion, emaste.
Herald added a reviewer: espindola.
Herald added a reviewer: MaskRay.
Herald added projects: clang, LLVM.

Expand remarks hotness threshold option -fdiagnostics-hotness-threshold in clang
driver command-line to both filtering remarks by hotness threshold from profile
summary and automatic remarks options pass-through to linker.

Remarks hotness filtering relies on several driver options. Table below lists
how different options are correlated and affect final remarks outputs:

| profile | hotness | threshold | remarks printed |
| --- | --- | - | --- |
| No  | No  | No| All |
| No  | No  | Yes   | None|
| No  | Yes | No| All |
| No  | Yes | Yes   | None|
| Yes | No  | No| All |
| Yes | No  | Yes   | None|
| Yes | Yes | No| All |
| Yes | Yes | Yes   | >=threshold |
|

The new argument value -fdiagnostics-hotness-threshold=auto indicates threshold
will be synced with hotness threshold from profile summary during compilation.
In addition, when the following conditions are met, remarks related options are
passed into linker as well:

1. LTO is enabled;
2. Single arch target is specified;
3. The linker is lld;

The "auto" threshold relies on the availability of profile summary. In case of
missing such information, no remarks will be generated.

This gives novice user a convenient way to collect and filter remarks throughout
a typical toolchain invocation with sample profile and LTO using single switch
from the clang driver.

A typical use of this option from clang command-line:

- Using -Rpass* options to print remarks to screen:

clang -fuse-ld=lld -flto=thin

  -fprofile-sample-use=foo_sample.txt
  -Rpass=inline
  -Rpass-missed=inline -Rpass-analysis=inline
  -fdiagnostics-show-hotness -fdiagnostics-hotness-threshold=auto
  -o foo foo.cpp

Remarks will be dumped to screen from both pre-lto and lto compilation.

- Using serialized remarks options:

clang -fuse-ld=lld -flto=thin

  -fprofile-sample-use=foo_sample.txt -fsave-optimization-record
  -fdiagnostics-show-hotness -fdiagnostics-hotness-threshold=auto
  -o foo foo.cpp

This will produce multiple yaml files containing optimization remarks:

- foo.opt.yaml : remarks from pre-lto
- foo.opt.ld.yaml.thin.1.yaml: remark during lto

Both types of options can be used together.

Given its restricted use cases, this shouldn't be viewed as a complete
replacement of current remarks option handling for the linker.

In order to make the option consistent across various tools, the support for the
new 'auto' argument is also available in the following tools:

- lld:  -opt-remarks-with-hotness=auto
- llvm-lto: -lto-pass-remarks-hotness-threshold=auto
- opt/llc:  --pass-remarks-hotness-threshold=auto


Repository:
  rG LLVM Github Monorepo

https://reviews.llvm.org/D84246

Files:
  clang/include/clang/Basic/CodeGenOptions.def
  clang/include/clang/Basic/CodeGenOptions.h
  clang/include/clang/Basic/DiagnosticDriverKinds.td
  clang/include/clang/Driver/Driver.h
  clang/include/clang/Driver/Options.td
  clang/lib/Driver/Driver.cpp
  clang/lib/Driver/ToolChains/CommonArgs.cpp
  clang/lib/Frontend/CompilerInvocation.cpp
  clang/test/Driver/remarks-pass-through.c
  lld/ELF/Config.h
  lld/ELF/Driver.cpp
  lld/ELF/LTO.cpp
  lld/ELF/Options.td
  lld/test/ELF/lto/opt-remarks.ll
  llvm/include/llvm/Analysis/ProfileSummaryInfo.h
  llvm/include/llvm/IR/LLVMContext.h
  llvm/include/llvm/IR/LLVMRemarkStreamer.h
  llvm/include/llvm/IR/Module.h
  llvm/include/llvm/LTO/Config.h
  llvm/include/llvm/LTO/LTO.h
  llvm/include/llvm/Remarks/HotnessThresholdParser.h
  llvm/lib/Analysis/OptimizationRemarkEmitter.cpp
  llvm/lib/IR/LLVMContext.cpp
  llvm/lib/IR/LLVMContextImpl.h
  llvm/lib/IR/LLVMRemarkStreamer.cpp
  llvm/lib/IR/Module.cpp
  llvm/lib/LTO/LTO.cpp
  llvm/lib/LTO/LTOBackend.cpp
  llvm/lib/LTO/LTOCodeGenerator.cpp
  llvm/lib/LTO/ThinLTOCodeGenerator.cpp
  llvm/test/LTO/X86/diagnostic-handler-remarks-with-hotness.ll
  llvm/test/Other/optimization-remarks-auto.ll
  llvm/test/Transforms/SampleProfile/Inputs/remarks-hotness.prof
  llvm/test/Transforms/SampleProfile/remarks-hotness.ll
  llvm/tools/llc/llc.cpp
  llvm/tools/opt/opt.cpp

Index: llvm/tools/opt/opt.cpp
===
--- llvm/tools/opt/opt.cpp
+++ llvm/tools/opt/opt.cpp
@@ -40,6 +40,7 @@
 #include "llvm/LinkAllIR.h"
 #include "llvm/LinkAllPasses.h"
 #include "llvm/MC/SubtargetFeature.h"
+#include "llvm/Remarks/HotnessThresholdParser.h"
 #include "llvm/Support/Debug.h"
 #include "llvm/Support/FileSystem.h"
 #include "llvm/Support/Host.h"
@@ -274,11 +275,13