[clang] [clang-tools-extra] [flang] [llvm] [mlir] [polly] [test]: fix filecheck annotation typos (PR #91854)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

Were these found manually or using some automated tooling? Based on the Rust 
PR, it seems like these were found by just manually going through the results 
from a regex?

https://github.com/llvm/llvm-project/pull/91854
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [CMake][Release] Disable PGO (PR #88465)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 approved this pull request.

LGTM.

I've run into the issue where profile data is generated again where it 
shouldn't be. I've meant to open up an issue and look into it at some point, 
but haven't had time to do so. I haven't previously run into this causing 
recompilations, but the outlined flow makes sense.

I'm assuming this is a stop-gap, and the plan is to eventually fix the issue 
and reenable PGO in the release cache?

https://github.com/llvm/llvm-project/pull/88465
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Github] Use building LLVM as perf-training for CI container (PR #80713)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

Looks like there was a transient failure in the stage2 build while loading the 
container image. I've restarted it and everything seems to be going fine. 
Hopefully that's not an issue going forward.

https://github.com/llvm/llvm-project/pull/80713
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [Github] Use building LLVM as perf-training for CI container (PR #80713)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,13 @@
+diff --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
+index 1a04ca9a74e5..d092820e4115 100644
+--- a/clang/cmake/caches/BOLT-PGO.cmake
 b/clang/cmake/caches/BOLT-PGO.cmake
+@@ -4,6 +4,8 @@ set(CLANG_BOOTSTRAP_TARGETS
+   stage2-clang-bolt
+   stage2-distribution
+   stage2-install-distribution
++  clang
++  lld

boomanaiden154 wrote:

Yes. `stage1` takes about an hour to build `stage2-instrumented takes about an 
hour and a half, generating the profile data takes about four hours, and 
building the final compiler takes about an hour. I've split the build so that 
stage 1 and stage2-instrumented are built in the first stage and the profile 
collection and stage2 are built afterwards. There's no explicit build target 
(from what I can gather) for building the `stage2-instrumented` compiler 
without doing profile data collection, so I've just added the targets 
explicitly here.

It's a hack and I don't think there's that much utility in doing this more 
broadly. If we wanted to make it less of  hack, we could add an explicit target 
to build the `stage2-instrumented` dependencies without doing the profile 
collection, but I think this hack works well enough for now and we should be 
able to move away from this once we have self-hosted runners setup.

https://github.com/llvm/llvm-project/pull/80713
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Github] Use building LLVM as perf-training for CI container (PR #80713)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,13 @@
+diff --git a/clang/cmake/caches/BOLT-PGO.cmake 
b/clang/cmake/caches/BOLT-PGO.cmake
+index 1a04ca9a74e5..d092820e4115 100644
+--- a/clang/cmake/caches/BOLT-PGO.cmake
 b/clang/cmake/caches/BOLT-PGO.cmake
+@@ -4,6 +4,8 @@ set(CLANG_BOOTSTRAP_TARGETS
+   stage2-clang-bolt
+   stage2-distribution
+   stage2-install-distribution
++  clang
++  lld

boomanaiden154 wrote:

It's needed so that we don't build more than we need to in the 
stage2-instrumented build. The default targets build way more in that stage and 
the generate-profdata target only depends on having clang and lld built.

https://github.com/llvm/llvm-project/pull/80713
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Github] Use building LLVM as perf-training for CI container (PR #80713)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/80713

>From bef28f6d909060aeb4993866fad52dbe8a897d20 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 3 Feb 2024 20:46:57 -0800
Subject: [PATCH 1/2] [Github] Use building LLVM as perf-training for CI
 container

This patch adjusts the build process for building the toolchain for the
CI container to perform more rigorous perf-training for PGO,
particularly building the entirety of LLVM as that is what showed the
best results while benchmarking. This patch also splits the job into two
stages to avoid timeouts due to the large increase in buildtime. There
are a couple other hacks added in here to make things work that we can
do away with eventually once we're able to run jobs like this on more
powerful self-hosted runners.
---
 .github/workflows/build-ci-container.yml  | 54 +-
 .../containers/github-action-ci/Dockerfile| 55 ---
 .../github-action-ci/bootstrap.patch  | 13 +
 .../github-action-ci/stage1.Dockerfile| 44 +++
 .../github-action-ci/stage2.Dockerfile| 27 +
 .../containers/github-action-ci/storage.conf  |  4 ++
 clang/cmake/caches/BOLT-PGO.cmake |  2 +
 7 files changed, 141 insertions(+), 58 deletions(-)
 delete mode 100644 .github/workflows/containers/github-action-ci/Dockerfile
 create mode 100644 
.github/workflows/containers/github-action-ci/bootstrap.patch
 create mode 100644 
.github/workflows/containers/github-action-ci/stage1.Dockerfile
 create mode 100644 
.github/workflows/containers/github-action-ci/stage2.Dockerfile
 create mode 100644 .github/workflows/containers/github-action-ci/storage.conf

diff --git a/.github/workflows/build-ci-container.yml 
b/.github/workflows/build-ci-container.yml
index ad3d50d4d578a..3f2bf57eb8508 100644
--- a/.github/workflows/build-ci-container.yml
+++ b/.github/workflows/build-ci-container.yml
@@ -1,4 +1,3 @@
-
 name: Build CI Container
 
 permissions:
@@ -19,9 +18,41 @@ on:
   - '.github/workflows/containers/github-action-ci/**'
 
 jobs:
-  build-ci-container:
+  # TODO(boomanaiden154): Switch this back to a single stage build when we can
+  # run this on the self-hosted runners and don't have to do it this way to
+  # avoid timeouts.
+  build-ci-container-stage1:
 if: github.repository_owner == 'llvm'
 runs-on: ubuntu-latest
+steps:
+  - name: Checkout LLVM
+uses: actions/checkout@v4
+with:
+  sparse-checkout: .github/workflows/containers/github-action-ci/
+  - name: Change podman Root Direcotry
+run: |
+  mkdir -p ~/.config/containers
+  sudo mkdir -p /mnt/podman
+  sudo chown `whoami`:`whoami` /mnt/podman
+  cp ./.github/workflows/containers/github-action-ci/storage.conf 
~/.config/containers/storage.conf
+  podman info
+  - name: Build container stage1
+working-directory: ./.github/workflows/containers/github-action-ci/
+run: |
+  podman build -t stage1-toolchain --target stage1-toolchain -f 
stage1.Dockerfile .
+  - name: Save container image
+run: |
+  podman save stage1-toolchain > stage1-toolchain.tar
+  - name: Upload container image
+uses: actions/upload-artifact@v4
+with:
+  name: stage1-toolchain
+  path: stage1-toolchain.tar
+  retention-days: 1
+  build-ci-container-stage2:
+if: github.repository_owner == 'llvm'
+runs-on: ubuntu-latest
+needs: build-ci-container-stage1
 permissions:
   packages: write
 steps:
@@ -38,10 +69,27 @@ jobs:
 with:
   sparse-checkout: .github/workflows/containers/github-action-ci/
 
+  - name: Change podman Root Direcotry
+run: |
+  mkdir -p ~/.config/containers
+  sudo mkdir -p /mnt/podman
+  sudo chown `whoami`:`whoami` /mnt/podman
+  cp ./.github/workflows/containers/github-action-ci/storage.conf 
~/.config/containers/storage.conf
+  podman info
+
+  - name: Download stage1-toolchain
+uses: actions/download-artifact@v4
+with:
+  name: stage1-toolchain
+
+  - name: Load stage1-toolchain
+run: |
+  podman load -i stage1-toolchain.tar
+
   - name: Build Container
 working-directory: ./.github/workflows/containers/github-action-ci/
 run: |
-  podman build -t ${{ steps.vars.outputs.container-name-tag }} .
+  podman build -t ${{ steps.vars.outputs.container-name-tag }} -f 
stage2.Dockerfile .
   podman tag ${{ steps.vars.outputs.container-name-tag }} ${{ 
steps.vars.outputs.container-name }}:latest
 
   - name: Test Container
diff --git a/.github/workflows/containers/github-action-ci/Dockerfile 
b/.github/workflows/containers/github-action-ci/Dockerfile
deleted file mode 100644
index 66fa81d5a10ae..0
--- 

[clang] [llvm] [Github] Use building LLVM as perf-training for CI container (PR #80713)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 created 
https://github.com/llvm/llvm-project/pull/80713

This patch adjusts the build process for building the toolchain for the CI 
container to perform more rigorous perf-training for PGO, particularly building 
the entirety of LLVM as that is what showed the best results while 
benchmarking. This patch also splits the job into two stages to avoid timeouts 
due to the large increase in buildtime. There are a couple other hacks added in 
here to make things work that we can do away with eventually once we're able to 
run jobs like this on more powerful self-hosted runners.

>From bef28f6d909060aeb4993866fad52dbe8a897d20 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 3 Feb 2024 20:46:57 -0800
Subject: [PATCH] [Github] Use building LLVM as perf-training for CI container

This patch adjusts the build process for building the toolchain for the
CI container to perform more rigorous perf-training for PGO,
particularly building the entirety of LLVM as that is what showed the
best results while benchmarking. This patch also splits the job into two
stages to avoid timeouts due to the large increase in buildtime. There
are a couple other hacks added in here to make things work that we can
do away with eventually once we're able to run jobs like this on more
powerful self-hosted runners.
---
 .github/workflows/build-ci-container.yml  | 54 +-
 .../containers/github-action-ci/Dockerfile| 55 ---
 .../github-action-ci/bootstrap.patch  | 13 +
 .../github-action-ci/stage1.Dockerfile| 44 +++
 .../github-action-ci/stage2.Dockerfile| 27 +
 .../containers/github-action-ci/storage.conf  |  4 ++
 clang/cmake/caches/BOLT-PGO.cmake |  2 +
 7 files changed, 141 insertions(+), 58 deletions(-)
 delete mode 100644 .github/workflows/containers/github-action-ci/Dockerfile
 create mode 100644 
.github/workflows/containers/github-action-ci/bootstrap.patch
 create mode 100644 
.github/workflows/containers/github-action-ci/stage1.Dockerfile
 create mode 100644 
.github/workflows/containers/github-action-ci/stage2.Dockerfile
 create mode 100644 .github/workflows/containers/github-action-ci/storage.conf

diff --git a/.github/workflows/build-ci-container.yml 
b/.github/workflows/build-ci-container.yml
index ad3d50d4d578a..3f2bf57eb8508 100644
--- a/.github/workflows/build-ci-container.yml
+++ b/.github/workflows/build-ci-container.yml
@@ -1,4 +1,3 @@
-
 name: Build CI Container
 
 permissions:
@@ -19,9 +18,41 @@ on:
   - '.github/workflows/containers/github-action-ci/**'
 
 jobs:
-  build-ci-container:
+  # TODO(boomanaiden154): Switch this back to a single stage build when we can
+  # run this on the self-hosted runners and don't have to do it this way to
+  # avoid timeouts.
+  build-ci-container-stage1:
 if: github.repository_owner == 'llvm'
 runs-on: ubuntu-latest
+steps:
+  - name: Checkout LLVM
+uses: actions/checkout@v4
+with:
+  sparse-checkout: .github/workflows/containers/github-action-ci/
+  - name: Change podman Root Direcotry
+run: |
+  mkdir -p ~/.config/containers
+  sudo mkdir -p /mnt/podman
+  sudo chown `whoami`:`whoami` /mnt/podman
+  cp ./.github/workflows/containers/github-action-ci/storage.conf 
~/.config/containers/storage.conf
+  podman info
+  - name: Build container stage1
+working-directory: ./.github/workflows/containers/github-action-ci/
+run: |
+  podman build -t stage1-toolchain --target stage1-toolchain -f 
stage1.Dockerfile .
+  - name: Save container image
+run: |
+  podman save stage1-toolchain > stage1-toolchain.tar
+  - name: Upload container image
+uses: actions/upload-artifact@v4
+with:
+  name: stage1-toolchain
+  path: stage1-toolchain.tar
+  retention-days: 1
+  build-ci-container-stage2:
+if: github.repository_owner == 'llvm'
+runs-on: ubuntu-latest
+needs: build-ci-container-stage1
 permissions:
   packages: write
 steps:
@@ -38,10 +69,27 @@ jobs:
 with:
   sparse-checkout: .github/workflows/containers/github-action-ci/
 
+  - name: Change podman Root Direcotry
+run: |
+  mkdir -p ~/.config/containers
+  sudo mkdir -p /mnt/podman
+  sudo chown `whoami`:`whoami` /mnt/podman
+  cp ./.github/workflows/containers/github-action-ci/storage.conf 
~/.config/containers/storage.conf
+  podman info
+
+  - name: Download stage1-toolchain
+uses: actions/download-artifact@v4
+with:
+  name: stage1-toolchain
+
+  - name: Load stage1-toolchain
+run: |
+  podman load -i stage1-toolchain.tar
+
   - name: Build Container
 working-directory: ./.github/workflows/containers/github-action-ci/
 run: |
-  podman build -t ${{ 

[llvm] [lld] [clang] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

@@ -73,68 +83,89 @@ FileHeader:
 Sections:
   - Name:.text.foo
 Type:SHT_PROGBITS
-Address: [[FOO_ADDR]]
+Address: 0x4000
 Flags:   [SHF_ALLOC, SHF_EXECINSTR]
-Content: '503b050520907d02ebf5c3'
+Content: '503b050530907d08ebf50f8dee1fc3'
   - Name:.text.bar
 Type:SHT_PROGBITS
-Address: [[BAR_ADDR]]
+Address: 0x5000
 Flags:   [SHF_ALLOC, SHF_EXECINSTR]
 Content: '5089d0740231f6e8f4ffc3'
+  - Name:.text.split
+Type:SHT_PROGBITS
+Address: 0x6000
+Flags:   [SHF_ALLOC, SHF_EXECINSTR]
+Content: 'c3'
   - Name:.data
 Type:SHT_PROGBITS
 Flags:   [SHF_ALLOC, SHF_WRITE]
-Address: 0x6000
+Address: 0x7000
   - Name:   .llvm_bb_addr_map.foo
 Type:   SHT_LLVM_BB_ADDR_MAP
 Link:   .text.foo
 Entries:
   - Version: 2
-Address: [[FOO_ADDR]]
-BBEntries:
-  - ID:3
-AddressOffset: 0x0
-Size:  0x1
-Metadata:  0x1
-  - ID:1
-AddressOffset: 0x0
-Size:  0x6
-Metadata:  0x0
-  - ID:2
-AddressOffset: 0x1
-Size:  0x4
-Metadata:  0x0
-  - ID:5
-AddressOffset: 0x0
-Size:  0x1
-Metadata:  0x2
+Feature: 0x8
+BBRanges:
+  - BaseAddress: 0x4000
+BBEntries:
+ - ID:3
+   AddressOffset: 0x0

boomanaiden154 wrote:

Makes sense. The test case was passing, so everything was getting decoded 
correctly.  For some reason never made the connection that the address offset 
was relative.

https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [clang-tools-extra] [llvm-exegesis] Replace --num-repetitions with --min-instructions (PR #77153)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/77153
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Replace --num-repetitions with --min-instructions (PR #77153)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

> It's a single line of code so it probably can stay there for backwards 
> compatibility :)

Sounds good.

https://github.com/llvm/llvm-project/pull/77153
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Replace --num-repetitions with --min-instructions (PR #77153)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77153

>From 1a86a03fdc9d6eee08830ff2f113c39e6096d564 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Fri, 5 Jan 2024 14:40:26 -0800
Subject: [PATCH 1/5] [llvm-exegesis] Replace --num-repetitions with
 --min-instructions

This patch replaces --num-repetitions with --min-instructions to make it
more clear that the value refers to the minimum number of instructions
in the final assembled snippet rather than the number of repetitions  of
the snippet. This patch also refactors some llvm-exegesis internal
variable names to reflect the name change.

Fixes #76890.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp |  8 +++-
 llvm/tools/llvm-exegesis/lib/BenchmarkResult.h   |  2 +-
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp | 16 
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h   |  2 +-
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp   | 13 +++--
 .../llvm-exegesis/Mips/BenchmarkResultTest.cpp   |  6 +++---
 .../llvm-exegesis/X86/BenchmarkResultTest.cpp|  6 +++---
 7 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
index 02c4da11e032d..d13409fe4738a 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
@@ -293,7 +293,6 @@ struct MappingContextTraits {
 Io.mapRequired("key", Obj.Key, Context);
 Io.mapRequired("cpu_name", Obj.CpuName);
 Io.mapRequired("llvm_triple", Obj.LLVMTriple);
-Io.mapRequired("num_repetitions", Obj.NumRepetitions);
 Io.mapRequired("measurements", Obj.Measurements);
 Io.mapRequired("error", Obj.Error);
 Io.mapOptional("info", Obj.Info);
@@ -301,6 +300,13 @@ struct MappingContextTraits {
 MappingNormalization> BinaryString(
 Io, Obj.AssembledSnippet);
 Io.mapOptional("assembled_snippet", BinaryString->Binary);
+// Optionally map num_repetitions and min_instructions to the same
+// value to preserve backwards compatibility.
+// TODO(boomanaiden154): Move min_instructions to mapRequired and
+// remove num_repetitions once num_repetitions is ready to be removed
+// completely.
+Io.mapOptional("num_repetitions", Obj.MinInstructions);
+Io.mapOptional("min_instructions", Obj.MinInstructions);
   }
 };
 
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb..ffba634151586 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -100,7 +100,7 @@ struct Benchmark {
   const MCInst () const { return Key.Instructions[0]; }
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
-  unsigned NumRepetitions = 0;
+  unsigned MinInstructions = 0;
   enum RepetitionModeE { Duplicate, Loop, AggregateMin };
   // Note that measurements are per instruction.
   std::vector Measurements;
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index dee7af5fd520a..6c5f9e557812d 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -498,7 +498,7 @@ Expected> BenchmarkRunner::assembleSnippet(
 
 Expected
 BenchmarkRunner::getRunnableConfiguration(
-const BenchmarkCode , unsigned NumRepetitions, unsigned LoopBodySize,
+const BenchmarkCode , unsigned MinInstructions, unsigned LoopBodySize,
 const SnippetRepetitor ) const {
   RunnableConfiguration RC;
 
@@ -508,7 +508,7 @@ BenchmarkRunner::getRunnableConfiguration(
   std::string(State.getTargetMachine().getTargetCPU());
   BenchmarkResult.LLVMTriple =
   State.getTargetMachine().getTargetTriple().normalize();
-  BenchmarkResult.NumRepetitions = NumRepetitions;
+  BenchmarkResult.MinInstructions = MinInstructions;
   BenchmarkResult.Info = BC.Info;
 
   const std::vector  = BC.Key.Instructions;
@@ -534,12 +534,12 @@ BenchmarkRunner::getRunnableConfiguration(
   return std::move(Err);
   }
 
-  // Assemble NumRepetitions instructions repetitions of the snippet for
-  // measurements.
+  // Assemble enough repetitions of the snippet so we have at least
+  // MinInstructios instructions.
   if (BenchmarkPhaseSelector >
   BenchmarkPhaseSelectorE::PrepareAndAssembleSnippet) {
 auto Snippet =
-assembleSnippet(BC, Repetitor, BenchmarkResult.NumRepetitions,
+assembleSnippet(BC, Repetitor, BenchmarkResult.MinInstructions,
 LoopBodySize, GenerateMemoryInstructions);
 if (Error E = Snippet.takeError())
   return std::move(E);
@@ -610,14 +610,14 @@ std::pair 
BenchmarkRunner::runConfiguration(
   if (Error E = NewMeasurements.takeError()) {
 return {std::move(E), 

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Replace --num-repetitions with --min-instructions (PR #77153)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

@legrosbuffle This is ready for review when you have a chance. Thanks!

https://github.com/llvm/llvm-project/pull/77153
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Replace --num-repetitions with --min-instructions (PR #77153)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77153

>From 1a86a03fdc9d6eee08830ff2f113c39e6096d564 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Fri, 5 Jan 2024 14:40:26 -0800
Subject: [PATCH 1/4] [llvm-exegesis] Replace --num-repetitions with
 --min-instructions

This patch replaces --num-repetitions with --min-instructions to make it
more clear that the value refers to the minimum number of instructions
in the final assembled snippet rather than the number of repetitions  of
the snippet. This patch also refactors some llvm-exegesis internal
variable names to reflect the name change.

Fixes #76890.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp |  8 +++-
 llvm/tools/llvm-exegesis/lib/BenchmarkResult.h   |  2 +-
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp | 16 
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h   |  2 +-
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp   | 13 +++--
 .../llvm-exegesis/Mips/BenchmarkResultTest.cpp   |  6 +++---
 .../llvm-exegesis/X86/BenchmarkResultTest.cpp|  6 +++---
 7 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
index 02c4da11e032d..d13409fe4738a 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
@@ -293,7 +293,6 @@ struct MappingContextTraits {
 Io.mapRequired("key", Obj.Key, Context);
 Io.mapRequired("cpu_name", Obj.CpuName);
 Io.mapRequired("llvm_triple", Obj.LLVMTriple);
-Io.mapRequired("num_repetitions", Obj.NumRepetitions);
 Io.mapRequired("measurements", Obj.Measurements);
 Io.mapRequired("error", Obj.Error);
 Io.mapOptional("info", Obj.Info);
@@ -301,6 +300,13 @@ struct MappingContextTraits {
 MappingNormalization> BinaryString(
 Io, Obj.AssembledSnippet);
 Io.mapOptional("assembled_snippet", BinaryString->Binary);
+// Optionally map num_repetitions and min_instructions to the same
+// value to preserve backwards compatibility.
+// TODO(boomanaiden154): Move min_instructions to mapRequired and
+// remove num_repetitions once num_repetitions is ready to be removed
+// completely.
+Io.mapOptional("num_repetitions", Obj.MinInstructions);
+Io.mapOptional("min_instructions", Obj.MinInstructions);
   }
 };
 
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb..ffba634151586 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -100,7 +100,7 @@ struct Benchmark {
   const MCInst () const { return Key.Instructions[0]; }
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
-  unsigned NumRepetitions = 0;
+  unsigned MinInstructions = 0;
   enum RepetitionModeE { Duplicate, Loop, AggregateMin };
   // Note that measurements are per instruction.
   std::vector Measurements;
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index dee7af5fd520a..6c5f9e557812d 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -498,7 +498,7 @@ Expected> BenchmarkRunner::assembleSnippet(
 
 Expected
 BenchmarkRunner::getRunnableConfiguration(
-const BenchmarkCode , unsigned NumRepetitions, unsigned LoopBodySize,
+const BenchmarkCode , unsigned MinInstructions, unsigned LoopBodySize,
 const SnippetRepetitor ) const {
   RunnableConfiguration RC;
 
@@ -508,7 +508,7 @@ BenchmarkRunner::getRunnableConfiguration(
   std::string(State.getTargetMachine().getTargetCPU());
   BenchmarkResult.LLVMTriple =
   State.getTargetMachine().getTargetTriple().normalize();
-  BenchmarkResult.NumRepetitions = NumRepetitions;
+  BenchmarkResult.MinInstructions = MinInstructions;
   BenchmarkResult.Info = BC.Info;
 
   const std::vector  = BC.Key.Instructions;
@@ -534,12 +534,12 @@ BenchmarkRunner::getRunnableConfiguration(
   return std::move(Err);
   }
 
-  // Assemble NumRepetitions instructions repetitions of the snippet for
-  // measurements.
+  // Assemble enough repetitions of the snippet so we have at least
+  // MinInstructios instructions.
   if (BenchmarkPhaseSelector >
   BenchmarkPhaseSelectorE::PrepareAndAssembleSnippet) {
 auto Snippet =
-assembleSnippet(BC, Repetitor, BenchmarkResult.NumRepetitions,
+assembleSnippet(BC, Repetitor, BenchmarkResult.MinInstructions,
 LoopBodySize, GenerateMemoryInstructions);
 if (Error E = Snippet.takeError())
   return std::move(E);
@@ -610,14 +610,14 @@ std::pair 
BenchmarkRunner::runConfiguration(
   if (Error E = NewMeasurements.takeError()) {
 return {std::move(E), 

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Replace --num-repetitions with --min-instructions (PR #77153)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77153

>From 1a86a03fdc9d6eee08830ff2f113c39e6096d564 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Fri, 5 Jan 2024 14:40:26 -0800
Subject: [PATCH 1/3] [llvm-exegesis] Replace --num-repetitions with
 --min-instructions

This patch replaces --num-repetitions with --min-instructions to make it
more clear that the value refers to the minimum number of instructions
in the final assembled snippet rather than the number of repetitions  of
the snippet. This patch also refactors some llvm-exegesis internal
variable names to reflect the name change.

Fixes #76890.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp |  8 +++-
 llvm/tools/llvm-exegesis/lib/BenchmarkResult.h   |  2 +-
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp | 16 
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.h   |  2 +-
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp   | 13 +++--
 .../llvm-exegesis/Mips/BenchmarkResultTest.cpp   |  6 +++---
 .../llvm-exegesis/X86/BenchmarkResultTest.cpp|  6 +++---
 7 files changed, 30 insertions(+), 23 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
index 02c4da11e032d..d13409fe4738a 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.cpp
@@ -293,7 +293,6 @@ struct MappingContextTraits {
 Io.mapRequired("key", Obj.Key, Context);
 Io.mapRequired("cpu_name", Obj.CpuName);
 Io.mapRequired("llvm_triple", Obj.LLVMTriple);
-Io.mapRequired("num_repetitions", Obj.NumRepetitions);
 Io.mapRequired("measurements", Obj.Measurements);
 Io.mapRequired("error", Obj.Error);
 Io.mapOptional("info", Obj.Info);
@@ -301,6 +300,13 @@ struct MappingContextTraits {
 MappingNormalization> BinaryString(
 Io, Obj.AssembledSnippet);
 Io.mapOptional("assembled_snippet", BinaryString->Binary);
+// Optionally map num_repetitions and min_instructions to the same
+// value to preserve backwards compatibility.
+// TODO(boomanaiden154): Move min_instructions to mapRequired and
+// remove num_repetitions once num_repetitions is ready to be removed
+// completely.
+Io.mapOptional("num_repetitions", Obj.MinInstructions);
+Io.mapOptional("min_instructions", Obj.MinInstructions);
   }
 };
 
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb..ffba634151586 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -100,7 +100,7 @@ struct Benchmark {
   const MCInst () const { return Key.Instructions[0]; }
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
-  unsigned NumRepetitions = 0;
+  unsigned MinInstructions = 0;
   enum RepetitionModeE { Duplicate, Loop, AggregateMin };
   // Note that measurements are per instruction.
   std::vector Measurements;
diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index dee7af5fd520a..6c5f9e557812d 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -498,7 +498,7 @@ Expected> BenchmarkRunner::assembleSnippet(
 
 Expected
 BenchmarkRunner::getRunnableConfiguration(
-const BenchmarkCode , unsigned NumRepetitions, unsigned LoopBodySize,
+const BenchmarkCode , unsigned MinInstructions, unsigned LoopBodySize,
 const SnippetRepetitor ) const {
   RunnableConfiguration RC;
 
@@ -508,7 +508,7 @@ BenchmarkRunner::getRunnableConfiguration(
   std::string(State.getTargetMachine().getTargetCPU());
   BenchmarkResult.LLVMTriple =
   State.getTargetMachine().getTargetTriple().normalize();
-  BenchmarkResult.NumRepetitions = NumRepetitions;
+  BenchmarkResult.MinInstructions = MinInstructions;
   BenchmarkResult.Info = BC.Info;
 
   const std::vector  = BC.Key.Instructions;
@@ -534,12 +534,12 @@ BenchmarkRunner::getRunnableConfiguration(
   return std::move(Err);
   }
 
-  // Assemble NumRepetitions instructions repetitions of the snippet for
-  // measurements.
+  // Assemble enough repetitions of the snippet so we have at least
+  // MinInstructios instructions.
   if (BenchmarkPhaseSelector >
   BenchmarkPhaseSelectorE::PrepareAndAssembleSnippet) {
 auto Snippet =
-assembleSnippet(BC, Repetitor, BenchmarkResult.NumRepetitions,
+assembleSnippet(BC, Repetitor, BenchmarkResult.MinInstructions,
 LoopBodySize, GenerateMemoryInstructions);
 if (Error E = Snippet.takeError())
   return std::move(E);
@@ -610,14 +610,14 @@ std::pair 
BenchmarkRunner::runConfiguration(
   if (Error E = NewMeasurements.takeError()) {
 return {std::move(E), 

[llvm] [compiler-rt] [clang-tools-extra] [flang] [clang] [libc] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/77020
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [compiler-rt] [clang-tools-extra] [flang] [clang] [libc] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77020

>From c5bac325802c65f65a8d99e8d82a7e8079c21fb4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Thu, 4 Jan 2024 01:47:49 -0800
Subject: [PATCH 1/6] [llvm-exegesis] Add middle half repetition mode

This patch adds two new repetition modes to llvm-exegesis, particularly
loop and duplicate repetition modes of what I am terming the middle half
repetition mode. The middle half repetition mode essentially runs each
measurement twice, one with twice the number of iterations of the other.
These two measurements are then agregated by taking their difference.
This subtracts away any setup/overhead that is unrelated to the code in
the snippet, providing more accurate results.

Using this mode on a couple toy examples, I am able to get exact
(integer) throughput values on all of them in contrast to the default
duplicate/loop repetition modes which show a little bit of noise on the
snippet value.
---
 .../tools/llvm-exegesis/lib/BenchmarkResult.h | 12 +++-
 .../llvm-exegesis/lib/SnippetRepetitor.cpp|  2 +
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp| 67 ++-
 3 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb..3be58fa7f6853 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -78,7 +78,7 @@ struct BenchmarkKey {
 struct BenchmarkMeasure {
   // A helper to create an unscaled BenchmarkMeasure.
   static BenchmarkMeasure Create(std::string Key, double Value) {
-return {Key, Value, Value};
+return {Key, Value, Value, Value};
   }
   std::string Key;
   // This is the per-instruction value, i.e. measured quantity scaled per
@@ -87,6 +87,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // This is the raw value collected from the full execution.
+  double RawValue;
 };
 
 // The result of an instruction benchmark.
@@ -101,7 +103,13 @@ struct Benchmark {
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
   unsigned NumRepetitions = 0;
-  enum RepetitionModeE { Duplicate, Loop, AggregateMin };
+  enum RepetitionModeE {
+Duplicate,
+Loop,
+AggregateMin,
+MiddleHalfDuplicate,
+MiddleHalfLoop
+  };
   // Note that measurements are per instruction.
   std::vector Measurements;
   std::string Error;
diff --git a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp 
b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
index cc5a045a8be5d..7e19e8a9bb521 100644
--- a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
+++ b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
@@ -135,8 +135,10 @@ SnippetRepetitor::Create(Benchmark::RepetitionModeE Mode,
  const LLVMState ) {
   switch (Mode) {
   case Benchmark::Duplicate:
+  case Benchmark::MiddleHalfDuplicate:
 return std::make_unique(State);
   case Benchmark::Loop:
+  case Benchmark::MiddleHalfLoop:
 return std::make_unique(State);
   case Benchmark::AggregateMin:
 break;
diff --git a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp 
b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
index ffbf94ce0fcb2..b963ca0252a72 100644
--- a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
+++ b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
@@ -106,10 +106,13 @@ static cl::opt 
RepetitionMode(
 cl::values(
 clEnumValN(exegesis::Benchmark::Duplicate, "duplicate",
"Duplicate the snippet"),
-clEnumValN(exegesis::Benchmark::Loop, "loop",
-   "Loop over the snippet"),
+clEnumValN(exegesis::Benchmark::Loop, "loop", "Loop over the snippet"),
 clEnumValN(exegesis::Benchmark::AggregateMin, "min",
-   "All of the above and take the minimum of measurements")),
+   "All of the above and take the minimum of measurements"),
+clEnumValN(exegesis::Benchmark::MiddleHalfDuplicate,
+   "middle-half-duplicate", "Middle half duplicate mode"),
+clEnumValN(exegesis::Benchmark::MiddleHalfLoop, "middle-half-loop",
+   "Middle half loop mode")),
 cl::init(exegesis::Benchmark::Duplicate));
 
 static cl::opt BenchmarkMeasurementsPrintProgress(
@@ -399,29 +402,37 @@ static void runBenchmarkConfigurations(
   std::optional> Meter;
   if (BenchmarkMeasurementsPrintProgress)
 Meter.emplace(Configurations.size());
+
+  SmallVector MinInstructions = {NumRepetitions};
+  if (RepetitionMode == Benchmark::MiddleHalfDuplicate ||
+  RepetitionMode == Benchmark::MiddleHalfLoop)
+MinInstructions.push_back(NumRepetitions * 2);
+
   for (const BenchmarkCode  : Configurations) {
 ProgressMeter<>::ProgressMeterStep 

[clang] [llvm] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

> In general, CSSPGO is meant as a more practical alternative for IRPGO thanks 
> to i) the use of sampled profile, ii) context-sensitivity. However, IRPGO is 
> still expected to provide better performance in all cases where it's 
> applicable thanks to accurate profile information.
Due to that, I don't think it makes sense to apply CSSPGO on top of IRPGO, and 
CMake automation would not permit mixing the two (both implemented as 
LLVM_BUILD_INSTRUMENTED exclusive options).

Ah. Might've been getting this confused with another technique (maybe 
IRPGO+instrumented context sensitive PGO?). I remember Google saying in 
https://discourse.llvm.org/t/practical-compiler-optimizations-for-wsc-workshop-us-llvm-dev-meeting-2023/73998
 that a valid pipeline was doing two rounds of profiling with different levels 
of instrumentation.

Thanks for the link to the existing performance results.

> With CMake stuff implemented in this diff, it's expected that the amount of 
> profile information collected would be miniscule (in-tree perf-training only 
> has a single hello world source file) and inadequate for getting much perf 
> boost with CSSPGO (it even prints the warning that it needs 6985000.0x more 
> profile).

Right. Not sure if you've seen https://github.com/llvm/llvm-project/pull/78879, 
but that should allow making that pretty easy to do. There's also 
https://github.com/llvm/llvm-project/pull/77347 proposing to change the 
default, but there hasn't really been a consensus on that one.

Thank you for the additional information. Sorry for the derailment/noise.

https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [Clang][CMake] Add CSSPGO support to LLVM_BUILD_INSTRUMENTED (PR #79942)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

Do you have benchmarks on how well this performs relative to standard IRPGO (or 
on top of it)?

https://github.com/llvm/llvm-project/pull/79942
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libc] [llvm] [clang-tools-extra] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

@legrosbuffle This should be ready to again when you have a chance to take a 
look. Thanks!

https://github.com/llvm/llvm-project/pull/77020
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libc] [llvm] [clang-tools-extra] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77020

>From c5bac325802c65f65a8d99e8d82a7e8079c21fb4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Thu, 4 Jan 2024 01:47:49 -0800
Subject: [PATCH 1/6] [llvm-exegesis] Add middle half repetition mode

This patch adds two new repetition modes to llvm-exegesis, particularly
loop and duplicate repetition modes of what I am terming the middle half
repetition mode. The middle half repetition mode essentially runs each
measurement twice, one with twice the number of iterations of the other.
These two measurements are then agregated by taking their difference.
This subtracts away any setup/overhead that is unrelated to the code in
the snippet, providing more accurate results.

Using this mode on a couple toy examples, I am able to get exact
(integer) throughput values on all of them in contrast to the default
duplicate/loop repetition modes which show a little bit of noise on the
snippet value.
---
 .../tools/llvm-exegesis/lib/BenchmarkResult.h | 12 +++-
 .../llvm-exegesis/lib/SnippetRepetitor.cpp|  2 +
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp| 67 ++-
 3 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb3e..3be58fa7f6853c3 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -78,7 +78,7 @@ struct BenchmarkKey {
 struct BenchmarkMeasure {
   // A helper to create an unscaled BenchmarkMeasure.
   static BenchmarkMeasure Create(std::string Key, double Value) {
-return {Key, Value, Value};
+return {Key, Value, Value, Value};
   }
   std::string Key;
   // This is the per-instruction value, i.e. measured quantity scaled per
@@ -87,6 +87,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // This is the raw value collected from the full execution.
+  double RawValue;
 };
 
 // The result of an instruction benchmark.
@@ -101,7 +103,13 @@ struct Benchmark {
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
   unsigned NumRepetitions = 0;
-  enum RepetitionModeE { Duplicate, Loop, AggregateMin };
+  enum RepetitionModeE {
+Duplicate,
+Loop,
+AggregateMin,
+MiddleHalfDuplicate,
+MiddleHalfLoop
+  };
   // Note that measurements are per instruction.
   std::vector Measurements;
   std::string Error;
diff --git a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp 
b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
index cc5a045a8be5dd4..7e19e8a9bb52118 100644
--- a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
+++ b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
@@ -135,8 +135,10 @@ SnippetRepetitor::Create(Benchmark::RepetitionModeE Mode,
  const LLVMState ) {
   switch (Mode) {
   case Benchmark::Duplicate:
+  case Benchmark::MiddleHalfDuplicate:
 return std::make_unique(State);
   case Benchmark::Loop:
+  case Benchmark::MiddleHalfLoop:
 return std::make_unique(State);
   case Benchmark::AggregateMin:
 break;
diff --git a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp 
b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
index ffbf94ce0fcb267..b963ca0252a72ed 100644
--- a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
+++ b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
@@ -106,10 +106,13 @@ static cl::opt 
RepetitionMode(
 cl::values(
 clEnumValN(exegesis::Benchmark::Duplicate, "duplicate",
"Duplicate the snippet"),
-clEnumValN(exegesis::Benchmark::Loop, "loop",
-   "Loop over the snippet"),
+clEnumValN(exegesis::Benchmark::Loop, "loop", "Loop over the snippet"),
 clEnumValN(exegesis::Benchmark::AggregateMin, "min",
-   "All of the above and take the minimum of measurements")),
+   "All of the above and take the minimum of measurements"),
+clEnumValN(exegesis::Benchmark::MiddleHalfDuplicate,
+   "middle-half-duplicate", "Middle half duplicate mode"),
+clEnumValN(exegesis::Benchmark::MiddleHalfLoop, "middle-half-loop",
+   "Middle half loop mode")),
 cl::init(exegesis::Benchmark::Duplicate));
 
 static cl::opt BenchmarkMeasurementsPrintProgress(
@@ -399,29 +402,37 @@ static void runBenchmarkConfigurations(
   std::optional> Meter;
   if (BenchmarkMeasurementsPrintProgress)
 Meter.emplace(Configurations.size());
+
+  SmallVector MinInstructions = {NumRepetitions};
+  if (RepetitionMode == Benchmark::MiddleHalfDuplicate ||
+  RepetitionMode == Benchmark::MiddleHalfLoop)
+MinInstructions.push_back(NumRepetitions * 2);
+
   for (const BenchmarkCode  : Configurations) {
 

[clang] [libc] [llvm] [clang-tools-extra] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77020

>From c5bac325802c65f65a8d99e8d82a7e8079c21fb4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Thu, 4 Jan 2024 01:47:49 -0800
Subject: [PATCH 1/5] [llvm-exegesis] Add middle half repetition mode

This patch adds two new repetition modes to llvm-exegesis, particularly
loop and duplicate repetition modes of what I am terming the middle half
repetition mode. The middle half repetition mode essentially runs each
measurement twice, one with twice the number of iterations of the other.
These two measurements are then agregated by taking their difference.
This subtracts away any setup/overhead that is unrelated to the code in
the snippet, providing more accurate results.

Using this mode on a couple toy examples, I am able to get exact
(integer) throughput values on all of them in contrast to the default
duplicate/loop repetition modes which show a little bit of noise on the
snippet value.
---
 .../tools/llvm-exegesis/lib/BenchmarkResult.h | 12 +++-
 .../llvm-exegesis/lib/SnippetRepetitor.cpp|  2 +
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp| 67 ++-
 3 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb3e..3be58fa7f6853c3 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -78,7 +78,7 @@ struct BenchmarkKey {
 struct BenchmarkMeasure {
   // A helper to create an unscaled BenchmarkMeasure.
   static BenchmarkMeasure Create(std::string Key, double Value) {
-return {Key, Value, Value};
+return {Key, Value, Value, Value};
   }
   std::string Key;
   // This is the per-instruction value, i.e. measured quantity scaled per
@@ -87,6 +87,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // This is the raw value collected from the full execution.
+  double RawValue;
 };
 
 // The result of an instruction benchmark.
@@ -101,7 +103,13 @@ struct Benchmark {
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
   unsigned NumRepetitions = 0;
-  enum RepetitionModeE { Duplicate, Loop, AggregateMin };
+  enum RepetitionModeE {
+Duplicate,
+Loop,
+AggregateMin,
+MiddleHalfDuplicate,
+MiddleHalfLoop
+  };
   // Note that measurements are per instruction.
   std::vector Measurements;
   std::string Error;
diff --git a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp 
b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
index cc5a045a8be5dd4..7e19e8a9bb52118 100644
--- a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
+++ b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
@@ -135,8 +135,10 @@ SnippetRepetitor::Create(Benchmark::RepetitionModeE Mode,
  const LLVMState ) {
   switch (Mode) {
   case Benchmark::Duplicate:
+  case Benchmark::MiddleHalfDuplicate:
 return std::make_unique(State);
   case Benchmark::Loop:
+  case Benchmark::MiddleHalfLoop:
 return std::make_unique(State);
   case Benchmark::AggregateMin:
 break;
diff --git a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp 
b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
index ffbf94ce0fcb267..b963ca0252a72ed 100644
--- a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
+++ b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
@@ -106,10 +106,13 @@ static cl::opt 
RepetitionMode(
 cl::values(
 clEnumValN(exegesis::Benchmark::Duplicate, "duplicate",
"Duplicate the snippet"),
-clEnumValN(exegesis::Benchmark::Loop, "loop",
-   "Loop over the snippet"),
+clEnumValN(exegesis::Benchmark::Loop, "loop", "Loop over the snippet"),
 clEnumValN(exegesis::Benchmark::AggregateMin, "min",
-   "All of the above and take the minimum of measurements")),
+   "All of the above and take the minimum of measurements"),
+clEnumValN(exegesis::Benchmark::MiddleHalfDuplicate,
+   "middle-half-duplicate", "Middle half duplicate mode"),
+clEnumValN(exegesis::Benchmark::MiddleHalfLoop, "middle-half-loop",
+   "Middle half loop mode")),
 cl::init(exegesis::Benchmark::Duplicate));
 
 static cl::opt BenchmarkMeasurementsPrintProgress(
@@ -399,29 +402,37 @@ static void runBenchmarkConfigurations(
   std::optional> Meter;
   if (BenchmarkMeasurementsPrintProgress)
 Meter.emplace(Configurations.size());
+
+  SmallVector MinInstructions = {NumRepetitions};
+  if (RepetitionMode == Benchmark::MiddleHalfDuplicate ||
+  RepetitionMode == Benchmark::MiddleHalfLoop)
+MinInstructions.push_back(NumRepetitions * 2);
+
   for (const BenchmarkCode  : Configurations) {
 

[clang] [libc] [llvm] [clang-tools-extra] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77020

>From c5bac325802c65f65a8d99e8d82a7e8079c21fb4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Thu, 4 Jan 2024 01:47:49 -0800
Subject: [PATCH 1/4] [llvm-exegesis] Add middle half repetition mode

This patch adds two new repetition modes to llvm-exegesis, particularly
loop and duplicate repetition modes of what I am terming the middle half
repetition mode. The middle half repetition mode essentially runs each
measurement twice, one with twice the number of iterations of the other.
These two measurements are then agregated by taking their difference.
This subtracts away any setup/overhead that is unrelated to the code in
the snippet, providing more accurate results.

Using this mode on a couple toy examples, I am able to get exact
(integer) throughput values on all of them in contrast to the default
duplicate/loop repetition modes which show a little bit of noise on the
snippet value.
---
 .../tools/llvm-exegesis/lib/BenchmarkResult.h | 12 +++-
 .../llvm-exegesis/lib/SnippetRepetitor.cpp|  2 +
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp| 67 ++-
 3 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb3e..3be58fa7f6853c3 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -78,7 +78,7 @@ struct BenchmarkKey {
 struct BenchmarkMeasure {
   // A helper to create an unscaled BenchmarkMeasure.
   static BenchmarkMeasure Create(std::string Key, double Value) {
-return {Key, Value, Value};
+return {Key, Value, Value, Value};
   }
   std::string Key;
   // This is the per-instruction value, i.e. measured quantity scaled per
@@ -87,6 +87,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // This is the raw value collected from the full execution.
+  double RawValue;
 };
 
 // The result of an instruction benchmark.
@@ -101,7 +103,13 @@ struct Benchmark {
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
   unsigned NumRepetitions = 0;
-  enum RepetitionModeE { Duplicate, Loop, AggregateMin };
+  enum RepetitionModeE {
+Duplicate,
+Loop,
+AggregateMin,
+MiddleHalfDuplicate,
+MiddleHalfLoop
+  };
   // Note that measurements are per instruction.
   std::vector Measurements;
   std::string Error;
diff --git a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp 
b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
index cc5a045a8be5dd4..7e19e8a9bb52118 100644
--- a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
+++ b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
@@ -135,8 +135,10 @@ SnippetRepetitor::Create(Benchmark::RepetitionModeE Mode,
  const LLVMState ) {
   switch (Mode) {
   case Benchmark::Duplicate:
+  case Benchmark::MiddleHalfDuplicate:
 return std::make_unique(State);
   case Benchmark::Loop:
+  case Benchmark::MiddleHalfLoop:
 return std::make_unique(State);
   case Benchmark::AggregateMin:
 break;
diff --git a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp 
b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
index ffbf94ce0fcb267..b963ca0252a72ed 100644
--- a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
+++ b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
@@ -106,10 +106,13 @@ static cl::opt 
RepetitionMode(
 cl::values(
 clEnumValN(exegesis::Benchmark::Duplicate, "duplicate",
"Duplicate the snippet"),
-clEnumValN(exegesis::Benchmark::Loop, "loop",
-   "Loop over the snippet"),
+clEnumValN(exegesis::Benchmark::Loop, "loop", "Loop over the snippet"),
 clEnumValN(exegesis::Benchmark::AggregateMin, "min",
-   "All of the above and take the minimum of measurements")),
+   "All of the above and take the minimum of measurements"),
+clEnumValN(exegesis::Benchmark::MiddleHalfDuplicate,
+   "middle-half-duplicate", "Middle half duplicate mode"),
+clEnumValN(exegesis::Benchmark::MiddleHalfLoop, "middle-half-loop",
+   "Middle half loop mode")),
 cl::init(exegesis::Benchmark::Duplicate));
 
 static cl::opt BenchmarkMeasurementsPrintProgress(
@@ -399,29 +402,37 @@ static void runBenchmarkConfigurations(
   std::optional> Meter;
   if (BenchmarkMeasurementsPrintProgress)
 Meter.emplace(Configurations.size());
+
+  SmallVector MinInstructions = {NumRepetitions};
+  if (RepetitionMode == Benchmark::MiddleHalfDuplicate ||
+  RepetitionMode == Benchmark::MiddleHalfLoop)
+MinInstructions.push_back(NumRepetitions * 2);
+
   for (const BenchmarkCode  : Configurations) {
 

[llvm] [clang-tools-extra] [clang] [libc] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

@@ -455,6 +466,26 @@ static void runBenchmarkConfigurations(
   Measurement.PerSnippetValue, NewMeasurement.PerSnippetValue);
 }
   }
+} else if (RepetitionMode ==

boomanaiden154 wrote:

Moved them to `ResultAggregator.cpp` and added unit tests.

https://github.com/llvm/llvm-project/pull/77020
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang-tools-extra] [clang] [libc] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77020

>From c5bac325802c65f65a8d99e8d82a7e8079c21fb4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Thu, 4 Jan 2024 01:47:49 -0800
Subject: [PATCH 1/3] [llvm-exegesis] Add middle half repetition mode

This patch adds two new repetition modes to llvm-exegesis, particularly
loop and duplicate repetition modes of what I am terming the middle half
repetition mode. The middle half repetition mode essentially runs each
measurement twice, one with twice the number of iterations of the other.
These two measurements are then agregated by taking their difference.
This subtracts away any setup/overhead that is unrelated to the code in
the snippet, providing more accurate results.

Using this mode on a couple toy examples, I am able to get exact
(integer) throughput values on all of them in contrast to the default
duplicate/loop repetition modes which show a little bit of noise on the
snippet value.
---
 .../tools/llvm-exegesis/lib/BenchmarkResult.h | 12 +++-
 .../llvm-exegesis/lib/SnippetRepetitor.cpp|  2 +
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp| 67 ++-
 3 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb3..3be58fa7f6853c 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -78,7 +78,7 @@ struct BenchmarkKey {
 struct BenchmarkMeasure {
   // A helper to create an unscaled BenchmarkMeasure.
   static BenchmarkMeasure Create(std::string Key, double Value) {
-return {Key, Value, Value};
+return {Key, Value, Value, Value};
   }
   std::string Key;
   // This is the per-instruction value, i.e. measured quantity scaled per
@@ -87,6 +87,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // This is the raw value collected from the full execution.
+  double RawValue;
 };
 
 // The result of an instruction benchmark.
@@ -101,7 +103,13 @@ struct Benchmark {
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
   unsigned NumRepetitions = 0;
-  enum RepetitionModeE { Duplicate, Loop, AggregateMin };
+  enum RepetitionModeE {
+Duplicate,
+Loop,
+AggregateMin,
+MiddleHalfDuplicate,
+MiddleHalfLoop
+  };
   // Note that measurements are per instruction.
   std::vector Measurements;
   std::string Error;
diff --git a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp 
b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
index cc5a045a8be5dd..7e19e8a9bb5211 100644
--- a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
+++ b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
@@ -135,8 +135,10 @@ SnippetRepetitor::Create(Benchmark::RepetitionModeE Mode,
  const LLVMState ) {
   switch (Mode) {
   case Benchmark::Duplicate:
+  case Benchmark::MiddleHalfDuplicate:
 return std::make_unique(State);
   case Benchmark::Loop:
+  case Benchmark::MiddleHalfLoop:
 return std::make_unique(State);
   case Benchmark::AggregateMin:
 break;
diff --git a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp 
b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
index ffbf94ce0fcb26..b963ca0252a72e 100644
--- a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
+++ b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
@@ -106,10 +106,13 @@ static cl::opt 
RepetitionMode(
 cl::values(
 clEnumValN(exegesis::Benchmark::Duplicate, "duplicate",
"Duplicate the snippet"),
-clEnumValN(exegesis::Benchmark::Loop, "loop",
-   "Loop over the snippet"),
+clEnumValN(exegesis::Benchmark::Loop, "loop", "Loop over the snippet"),
 clEnumValN(exegesis::Benchmark::AggregateMin, "min",
-   "All of the above and take the minimum of measurements")),
+   "All of the above and take the minimum of measurements"),
+clEnumValN(exegesis::Benchmark::MiddleHalfDuplicate,
+   "middle-half-duplicate", "Middle half duplicate mode"),
+clEnumValN(exegesis::Benchmark::MiddleHalfLoop, "middle-half-loop",
+   "Middle half loop mode")),
 cl::init(exegesis::Benchmark::Duplicate));
 
 static cl::opt BenchmarkMeasurementsPrintProgress(
@@ -399,29 +402,37 @@ static void runBenchmarkConfigurations(
   std::optional> Meter;
   if (BenchmarkMeasurementsPrintProgress)
 Meter.emplace(Configurations.size());
+
+  SmallVector MinInstructions = {NumRepetitions};
+  if (RepetitionMode == Benchmark::MiddleHalfDuplicate ||
+  RepetitionMode == Benchmark::MiddleHalfLoop)
+MinInstructions.push_back(NumRepetitions * 2);
+
   for (const BenchmarkCode  : Configurations) {
 

[clang-tools-extra] [libc] [llvm] [clang] [llvm-exegesis] Add middle half repetition mode (PR #77020)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/77020

>From c5bac325802c65f65a8d99e8d82a7e8079c21fb4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Thu, 4 Jan 2024 01:47:49 -0800
Subject: [PATCH 1/2] [llvm-exegesis] Add middle half repetition mode

This patch adds two new repetition modes to llvm-exegesis, particularly
loop and duplicate repetition modes of what I am terming the middle half
repetition mode. The middle half repetition mode essentially runs each
measurement twice, one with twice the number of iterations of the other.
These two measurements are then agregated by taking their difference.
This subtracts away any setup/overhead that is unrelated to the code in
the snippet, providing more accurate results.

Using this mode on a couple toy examples, I am able to get exact
(integer) throughput values on all of them in contrast to the default
duplicate/loop repetition modes which show a little bit of noise on the
snippet value.
---
 .../tools/llvm-exegesis/lib/BenchmarkResult.h | 12 +++-
 .../llvm-exegesis/lib/SnippetRepetitor.cpp|  2 +
 llvm/tools/llvm-exegesis/llvm-exegesis.cpp| 67 ++-
 3 files changed, 61 insertions(+), 20 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h 
b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
index 0d08febae20cb3..3be58fa7f6853c 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkResult.h
@@ -78,7 +78,7 @@ struct BenchmarkKey {
 struct BenchmarkMeasure {
   // A helper to create an unscaled BenchmarkMeasure.
   static BenchmarkMeasure Create(std::string Key, double Value) {
-return {Key, Value, Value};
+return {Key, Value, Value, Value};
   }
   std::string Key;
   // This is the per-instruction value, i.e. measured quantity scaled per
@@ -87,6 +87,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // This is the raw value collected from the full execution.
+  double RawValue;
 };
 
 // The result of an instruction benchmark.
@@ -101,7 +103,13 @@ struct Benchmark {
   // The number of instructions inside the repeated snippet. For example, if a
   // snippet of 3 instructions is repeated 4 times, this is 12.
   unsigned NumRepetitions = 0;
-  enum RepetitionModeE { Duplicate, Loop, AggregateMin };
+  enum RepetitionModeE {
+Duplicate,
+Loop,
+AggregateMin,
+MiddleHalfDuplicate,
+MiddleHalfLoop
+  };
   // Note that measurements are per instruction.
   std::vector Measurements;
   std::string Error;
diff --git a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp 
b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
index cc5a045a8be5dd..7e19e8a9bb5211 100644
--- a/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
+++ b/llvm/tools/llvm-exegesis/lib/SnippetRepetitor.cpp
@@ -135,8 +135,10 @@ SnippetRepetitor::Create(Benchmark::RepetitionModeE Mode,
  const LLVMState ) {
   switch (Mode) {
   case Benchmark::Duplicate:
+  case Benchmark::MiddleHalfDuplicate:
 return std::make_unique(State);
   case Benchmark::Loop:
+  case Benchmark::MiddleHalfLoop:
 return std::make_unique(State);
   case Benchmark::AggregateMin:
 break;
diff --git a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp 
b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
index ffbf94ce0fcb26..b963ca0252a72e 100644
--- a/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
+++ b/llvm/tools/llvm-exegesis/llvm-exegesis.cpp
@@ -106,10 +106,13 @@ static cl::opt 
RepetitionMode(
 cl::values(
 clEnumValN(exegesis::Benchmark::Duplicate, "duplicate",
"Duplicate the snippet"),
-clEnumValN(exegesis::Benchmark::Loop, "loop",
-   "Loop over the snippet"),
+clEnumValN(exegesis::Benchmark::Loop, "loop", "Loop over the snippet"),
 clEnumValN(exegesis::Benchmark::AggregateMin, "min",
-   "All of the above and take the minimum of measurements")),
+   "All of the above and take the minimum of measurements"),
+clEnumValN(exegesis::Benchmark::MiddleHalfDuplicate,
+   "middle-half-duplicate", "Middle half duplicate mode"),
+clEnumValN(exegesis::Benchmark::MiddleHalfLoop, "middle-half-loop",
+   "Middle half loop mode")),
 cl::init(exegesis::Benchmark::Duplicate));
 
 static cl::opt BenchmarkMeasurementsPrintProgress(
@@ -399,29 +402,37 @@ static void runBenchmarkConfigurations(
   std::optional> Meter;
   if (BenchmarkMeasurementsPrintProgress)
 Meter.emplace(Configurations.size());
+
+  SmallVector MinInstructions = {NumRepetitions};
+  if (RepetitionMode == Benchmark::MiddleHalfDuplicate ||
+  RepetitionMode == Benchmark::MiddleHalfLoop)
+MinInstructions.push_back(NumRepetitions * 2);
+
   for (const BenchmarkCode  : Configurations) {
 

[lld] [clang] [llvm] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

@@ -73,68 +83,89 @@ FileHeader:
 Sections:
   - Name:.text.foo
 Type:SHT_PROGBITS
-Address: [[FOO_ADDR]]
+Address: 0x4000
 Flags:   [SHF_ALLOC, SHF_EXECINSTR]
-Content: '503b050520907d02ebf5c3'
+Content: '503b050530907d08ebf50f8dee1fc3'
   - Name:.text.bar
 Type:SHT_PROGBITS
-Address: [[BAR_ADDR]]
+Address: 0x5000
 Flags:   [SHF_ALLOC, SHF_EXECINSTR]
 Content: '5089d0740231f6e8f4ffc3'
+  - Name:.text.split
+Type:SHT_PROGBITS
+Address: 0x6000
+Flags:   [SHF_ALLOC, SHF_EXECINSTR]
+Content: 'c3'
   - Name:.data
 Type:SHT_PROGBITS
 Flags:   [SHF_ALLOC, SHF_WRITE]
-Address: 0x6000
+Address: 0x7000
   - Name:   .llvm_bb_addr_map.foo
 Type:   SHT_LLVM_BB_ADDR_MAP
 Link:   .text.foo
 Entries:
   - Version: 2
-Address: [[FOO_ADDR]]
-BBEntries:
-  - ID:3
-AddressOffset: 0x0
-Size:  0x1
-Metadata:  0x1
-  - ID:1
-AddressOffset: 0x0
-Size:  0x6
-Metadata:  0x0
-  - ID:2
-AddressOffset: 0x1
-Size:  0x4
-Metadata:  0x0
-  - ID:5
-AddressOffset: 0x0
-Size:  0x1
-Metadata:  0x2
+Feature: 0x8
+BBRanges:
+  - BaseAddress: 0x4000
+BBEntries:
+ - ID:3
+   AddressOffset: 0x0

boomanaiden154 wrote:

Why do all of these BBs except for one have an address offset of zero? The 
symbolization seems fine, but given the address offset is relative to the base 
address of the range, it seems like these should all map to the same object? 
Unless the code in `yaml2obj` automatically takes into account the size and the 
AddressOffset is additional?

https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [lld] [clang] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

@@ -172,6 +172,105 @@ class OtoolOptTable : public CommonOptTable {
"Mach-O object file displaying tool") {}
 };
 
+struct BBAddrMapLabel {
+  std::string BlockLabel;
+  std::string PGOAnalysis;
+};
+
+// This class represents the BBAddrMap and PGOMap associated with a single
+// function.
+class BBAddrMapFunctionEntry {
+public:
+  BBAddrMapFunctionEntry(BBAddrMap AddrMap, PGOAnalysisMap PGOMap)
+  : AddrMap(std::move(AddrMap)), PGOMap(std::move(PGOMap)) {}
+
+  const BBAddrMap () const { return AddrMap; }
+
+  // Returns the PGO string associated with the entry of index 
`PGOBBEntryIndex`
+  // in `PGOMap`.
+  std::string constructPGOLabelString(size_t PGOBBEntryIndex) const {
+if (!PGOMap.FeatEnable.hasPGOAnalysis())
+  return "";
+std::string PGOString;
+raw_string_ostream PGOSS(PGOString);
+
+PGOSS << " (";
+if (PGOMap.FeatEnable.FuncEntryCount && PGOBBEntryIndex == 0) {
+  PGOSS << "Entry count: " << Twine(PGOMap.FuncEntryCount);
+  if (PGOMap.FeatEnable.hasPGOAnalysisBBData()) {
+PGOSS << ", ";
+  }
+}
+
+if (PGOMap.FeatEnable.hasPGOAnalysisBBData()) {
+
+  assert(PGOBBEntryIndex < PGOMap.BBEntries.size() &&
+ "Expected PGOAnalysisMap and BBAddrMap to have the same entires");
+  const PGOAnalysisMap::PGOBBEntry  =
+  PGOMap.BBEntries[PGOBBEntryIndex];
+
+  if (PGOMap.FeatEnable.BBFreq) {
+PGOSS << "Frequency: " << Twine(PGOBBEntry.BlockFreq.getFrequency());
+if (PGOMap.FeatEnable.BrProb && PGOBBEntry.Successors.size() > 0) {
+  PGOSS << ", ";
+}
+  }
+  if (PGOMap.FeatEnable.BrProb && PGOBBEntry.Successors.size() > 0) {
+PGOSS << "Successors: ";
+interleaveComma(
+PGOBBEntry.Successors, PGOSS,
+[](const PGOAnalysisMap::PGOBBEntry::SuccessorEntry ) {
+  PGOSS << "BB" << SE.ID << ":";
+  PGOSS.write_hex(SE.Prob.getNumerator());
+});
+  }
+}
+PGOSS << ")";
+
+return PGOString;
+  }
+
+private:
+  const BBAddrMap AddrMap;
+  const PGOAnalysisMap PGOMap;
+};
+
+// This class represents the BBAddrMap and PGOMap of potentially multiple
+// functions in a section.
+class BBAddrMapInfo {
+public:
+  void clear() {
+FunctionAddrToMap.clear();
+RangeBaseAddrToFunctionAddr.clear();
+  }
+  bool empty() const { return FunctionAddrToMap.empty(); }
+
+  void AddFunctionEntry(BBAddrMap AddrMap, PGOAnalysisMap PGOMap) {
+uint64_t FunctionAddr = AddrMap.getFunctionAddress();
+for (size_t I = 1; I < AddrMap.BBRanges.size(); ++I)
+  RangeBaseAddrToFunctionAddr.emplace(AddrMap.BBRanges[I].BaseAddress,
+  FunctionAddr);
+[[maybe_unused]] auto R = FunctionAddrToMap.try_emplace(
+FunctionAddr, std::move(AddrMap), std::move(PGOMap));
+assert(R.second && "duplicate function address");
+  }
+
+  const BBAddrMapFunctionEntry *getEntryForAddress(uint64_t BaseAddress) const 
{

boomanaiden154 wrote:

I'm assuming this is returning a pointer rather than a `std::optional` due to 
the inability to return an optional reference without something like a 
`std::reference_wrapper`?

https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [lld] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

@@ -172,6 +172,105 @@ class OtoolOptTable : public CommonOptTable {
"Mach-O object file displaying tool") {}
 };
 
+struct BBAddrMapLabel {
+  std::string BlockLabel;
+  std::string PGOAnalysis;
+};
+
+// This class represents the BBAddrMap and PGOMap associated with a single
+// function.
+class BBAddrMapFunctionEntry {
+public:
+  BBAddrMapFunctionEntry(BBAddrMap AddrMap, PGOAnalysisMap PGOMap)
+  : AddrMap(std::move(AddrMap)), PGOMap(std::move(PGOMap)) {}
+
+  const BBAddrMap () const { return AddrMap; }
+
+  // Returns the PGO string associated with the entry of index 
`PGOBBEntryIndex`
+  // in `PGOMap`.
+  std::string constructPGOLabelString(size_t PGOBBEntryIndex) const {
+if (!PGOMap.FeatEnable.hasPGOAnalysis())
+  return "";
+std::string PGOString;
+raw_string_ostream PGOSS(PGOString);
+
+PGOSS << " (";
+if (PGOMap.FeatEnable.FuncEntryCount && PGOBBEntryIndex == 0) {
+  PGOSS << "Entry count: " << Twine(PGOMap.FuncEntryCount);
+  if (PGOMap.FeatEnable.hasPGOAnalysisBBData()) {
+PGOSS << ", ";
+  }
+}
+
+if (PGOMap.FeatEnable.hasPGOAnalysisBBData()) {
+
+  assert(PGOBBEntryIndex < PGOMap.BBEntries.size() &&
+ "Expected PGOAnalysisMap and BBAddrMap to have the same entires");

boomanaiden154 wrote:

`/s/entires/entries`

Looks like a typo carried over from my `llvm-objdump` patch adding in support 
for symbolizing PGO Analysis data.

https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [lld] [clang] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

@@ -1693,24 +1750,21 @@ disassembleObject(ObjectFile , const ObjectFile 
,
 
   LLVM_DEBUG(LVP.dump());
 
-  std::unordered_map AddrToBBAddrMap;
-  std::unordered_map AddrToPGOAnalysisMap;
+  BBAddrMapInfo FullAddrMap;
   auto ReadBBAddrMap = [&](std::optional SectionIndex =
std::nullopt) {
-AddrToBBAddrMap.clear();
+FullAddrMap.clear();
 if (const auto *Elf = dyn_cast()) {
   std::vector PGOAnalyses;
   auto BBAddrMapsOrErr = Elf->readBBAddrMap(SectionIndex, );
   if (!BBAddrMapsOrErr) {
 reportWarning(toString(BBAddrMapsOrErr.takeError()), 
Obj.getFileName());
 return;
   }
-  for (const auto &[FunctionBBAddrMap, FunctionPGOAnalysis] :
+  for (auto &&[FunctionBBAddrMap, FunctionPGOAnalysis] :

boomanaiden154 wrote:

Looks like this is still here since this hasn't been rebased/main hasn't been 
merged in recently?

https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [lld] [llvm] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 approved this pull request.

Just nits/minor questions from me, code otherwise LGTM.

Sorry for the delay in review. I didn't realize you explicitly wanted me to 
review portions of the code.

https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[lld] [clang] [llvm] [SHT_LLVM_BB_ADDR_MAP] Allow basic-block-sections and labels be used together by decoupling the handling of the two features. (PR #74128)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 edited 
https://github.com/llvm/llvm-project/pull/74128
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [clang] [llvm] [AsmPrinter] Remove mbb-profile-dump flag (PR #76595)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/76595
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [clang] [AsmPrinter] Remove mbb-profile-dump flag (PR #76595)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76595

>From 55cf94d98c38cd2b5cd8fbf76e5fd83b5a47f4af Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Fri, 29 Dec 2023 16:14:15 -0800
Subject: [PATCH] [AsmPrinter] Remove mbb-profile-dump flag

Now that the work embedding PGO information in SHT_LLVM_BB_ADDR_MAP ELF
sections has landed, there is no longer a need to keep around the
mbb-profile-dump flag.
---
 llvm/include/llvm/CodeGen/AsmPrinter.h   |  4 --
 llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp   | 51 -
 llvm/test/CodeGen/Generic/bb-profile-dump.ll | 59 
 3 files changed, 114 deletions(-)
 delete mode 100644 llvm/test/CodeGen/Generic/bb-profile-dump.ll

diff --git a/llvm/include/llvm/CodeGen/AsmPrinter.h 
b/llvm/include/llvm/CodeGen/AsmPrinter.h
index 5ec246ee7015c4..fbd198a75a2436 100644
--- a/llvm/include/llvm/CodeGen/AsmPrinter.h
+++ b/llvm/include/llvm/CodeGen/AsmPrinter.h
@@ -236,10 +236,6 @@ class AsmPrinter : public MachineFunctionPass {
   /// split stack prologue.
   bool HasNoSplitStack = false;
 
-  /// Raw FDOstream for outputting machine basic block frequncies if the
-  /// --mbb-profile-dump flag is set for downstream cost modelling applications
-  std::unique_ptr MBBProfileDumpFileOutput;
-
 protected:
   explicit AsmPrinter(TargetMachine , std::unique_ptr Streamer);
 
diff --git a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp 
b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
index 4dd27702786e42..93857b05488b77 100644
--- a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
@@ -133,13 +133,6 @@ using namespace llvm;
 
 #define DEBUG_TYPE "asm-printer"
 
-static cl::opt BasicBlockProfileDump(
-"mbb-profile-dump", cl::Hidden,
-cl::desc("Basic block profile dump for external cost modelling. If "
- "matching up BBs with afterwards, the compilation must be "
- "performed with -basic-block-sections=labels. Enabling this "
- "flag during in-process ThinLTO is not supported."));
-
 const char DWARFGroupName[] = "dwarf";
 const char DWARFGroupDescription[] = "DWARF Emission";
 const char DbgTimerName[] = "emit";
@@ -624,16 +617,6 @@ bool AsmPrinter::doInitialization(Module ) {
 HI.Handler->beginModule();
   }
 
-  if (!BasicBlockProfileDump.empty()) {
-std::error_code PossibleFileError;
-MBBProfileDumpFileOutput = std::make_unique(
-BasicBlockProfileDump, PossibleFileError);
-if (PossibleFileError) {
-  M.getContext().emitError("Failed to open file for MBB Profile Dump: " +
-   PossibleFileError.message() + "\n");
-}
-  }
-
   return false;
 }
 
@@ -1952,40 +1935,6 @@ void AsmPrinter::emitFunctionBody() {
 OutStreamer->getCommentOS() << "-- End function\n";
 
   OutStreamer->addBlankLine();
-
-  // Output MBB ids, function names, and frequencies if the flag to dump
-  // MBB profile information has been set
-  if (MBBProfileDumpFileOutput && !MF->empty() &&
-  MF->getFunction().getEntryCount()) {
-if (!MF->hasBBLabels()) {
-  MF->getContext().reportError(
-  SMLoc(),
-  "Unable to find BB labels for MBB profile dump. -mbb-profile-dump "
-  "must be called with -basic-block-sections=labels");
-} else {
-  MachineBlockFrequencyInfo  =
-  getAnalysis().getBFI();
-  // The entry count and the entry basic block frequency aren't the same. 
We
-  // want to capture "absolute" frequencies, i.e. the frequency with which 
a
-  // MBB is executed when the program is executed. From there, we can 
derive
-  // Function-relative frequencies (divide by the value for the first MBB).
-  // We also have the information about frequency with which functions
-  // were called. This helps, for example, in a type of integration tests
-  // where we want to cross-validate the compiler's profile with a real
-  // profile.
-  // Using double precision because uint64 values used to encode mbb
-  // "frequencies" may be quite large.
-  const double EntryCount =
-  static_cast(MF->getFunction().getEntryCount()->getCount());
-  for (const auto  : *MF) {
-const double MBBRelFreq = MBFI.getBlockFreqRelativeToEntryBlock();
-const double AbsMBBFreq = MBBRelFreq * EntryCount;
-*MBBProfileDumpFileOutput.get()
-<< MF->getName() << "," << MBB.getBBID()->BaseID << ","
-<< AbsMBBFreq << "\n";
-  }
-}
-  }
 }
 
 /// Compute the number of Global Variables that uses a Constant.
diff --git a/llvm/test/CodeGen/Generic/bb-profile-dump.ll 
b/llvm/test/CodeGen/Generic/bb-profile-dump.ll
deleted file mode 100644
index 7391a6ee6f9128..00
--- a/llvm/test/CodeGen/Generic/bb-profile-dump.ll
+++ /dev/null
@@ -1,59 +0,0 @@
-; REQUIRES: x86-registered-target
-;
-; Check that the basic block profile dump outputs data and in the correct
-; format.
-;
-; RUN: 

[clang] [clang-tools-extra] [mlir] [libc] [libcxx] [llvm] [lldb] [MLGO] Remove absl dependency from scripts (PR #78880)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/78880
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [mlir] [libc] [libcxx] [llvm] [lldb] [MLGO] Remove absl dependency from scripts (PR #78880)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/78880

>From 80c9507d7f49ddbc5f2554f597950f797355c255 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sun, 21 Jan 2024 03:53:03 +
Subject: [PATCH 1/7] Add make_corpus script test

---
 .../tests/corpus/make_corpus_script.test  | 22 +++
 llvm/utils/mlgo-utils/tests/lit.cfg   |  7 +-
 2 files changed, 28 insertions(+), 1 deletion(-)
 create mode 100644 llvm/utils/mlgo-utils/tests/corpus/make_corpus_script.test

diff --git a/llvm/utils/mlgo-utils/tests/corpus/make_corpus_script.test 
b/llvm/utils/mlgo-utils/tests/corpus/make_corpus_script.test
new file mode 100644
index 000..f4f97544bce47d3
--- /dev/null
+++ b/llvm/utils/mlgo-utils/tests/corpus/make_corpus_script.test
@@ -0,0 +1,22 @@
+## Testing that the make_corpus script works as expected when invoked.
+
+# RUN: rm -rf %t.dir && mkdir %t.dir
+# RUN: touch %t.dir/test1.bc
+# RUN: touch %t.dir/test2.bc
+# RUN: rm -rf %t.out.dir && mkdir %t.out.dir
+
+# RUN: %python %scripts_dir/corpus/make_corpus.py --input_dir=%t.dir 
--output_dir=%t.out.dir --default_args="-test"
+
+# RUN: cat %t.out.dir/corpus_description.json | FileCheck %s
+
+## Check that we get the expected command in the global command override
+# CHECK: "-test"
+# CHECK: "has_thinlto": false
+## Check that the modules are in the corpus description
+# CHECK: "test1"
+# CHECK: "test2"
+
+# RUN: ls %t.out.dir | FileCheck %s --check-prefix CHECK-DIR
+
+# CHECK-DIR: test1.bc
+# CHECK-DIR: test2.bc
diff --git a/llvm/utils/mlgo-utils/tests/lit.cfg 
b/llvm/utils/mlgo-utils/tests/lit.cfg
index 055f0945942fc1c..9afced53f195c5f 100644
--- a/llvm/utils/mlgo-utils/tests/lit.cfg
+++ b/llvm/utils/mlgo-utils/tests/lit.cfg
@@ -1,3 +1,5 @@
+import os
+
 import lit.formats
 
 from lit.llvm import llvm_config
@@ -5,7 +7,7 @@ from lit.llvm import llvm_config
 config.name = "mlgo-utils"
 config.test_format = lit.formats.ShTest(execute_external=False)
 
-config.suffixes = [".py"]
+config.suffixes = [".py", ".test"]
 
 config.test_source_root = os.path.dirname(__file__)
 config.test_exec_root = config.obj_root
@@ -13,3 +15,6 @@ config.test_exec_root = config.obj_root
 config.environment["PYTHONPATH"] = os.path.join(config.src_root, "utils", 
"mlgo-utils")
 
 llvm_config.use_default_substitutions()
+
+scripts_dir = os.path.join(config.src_root, "utils/mlgo-utils/mlgo")
+config.substitutions.append(("%scripts_dir", scripts_dir))

>From d99f5d4cd2c7c6d9e70125e893dc2ae40c897d36 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sun, 21 Jan 2024 04:58:42 +
Subject: [PATCH 2/7] Add another test

---
 llvm/utils/mlgo-utils/CMakeLists.txt  |  2 +-
 .../combine_training_corpus_script.test   | 29 +++
 llvm/utils/mlgo-utils/tests/lit.cfg   |  1 +
 3 files changed, 31 insertions(+), 1 deletion(-)
 create mode 100644 
llvm/utils/mlgo-utils/tests/corpus/combine_training_corpus_script.test

diff --git a/llvm/utils/mlgo-utils/CMakeLists.txt 
b/llvm/utils/mlgo-utils/CMakeLists.txt
index 7b303c7639401ae..c263c92c632797e 100644
--- a/llvm/utils/mlgo-utils/CMakeLists.txt
+++ b/llvm/utils/mlgo-utils/CMakeLists.txt
@@ -5,7 +5,7 @@ configure_lit_site_cfg(
 
 add_lit_testsuite(check-mlgo-utils "Running mlgo-utils tests"
   ${CMAKE_CURRENT_BINARY_DIR}
-  DEPENDS "FileCheck" "not" "count"
+  DEPENDS "FileCheck" "not" "count" "split-file"
 )
 
 set_target_properties(check-mlgo-utils PROPERTIES FOLDER "Tests")
diff --git 
a/llvm/utils/mlgo-utils/tests/corpus/combine_training_corpus_script.test 
b/llvm/utils/mlgo-utils/tests/corpus/combine_training_corpus_script.test
new file mode 100644
index 000..1aa182146a49ee4
--- /dev/null
+++ b/llvm/utils/mlgo-utils/tests/corpus/combine_training_corpus_script.test
@@ -0,0 +1,29 @@
+## Testing that the combine_trainig_corpus script works as expected when
+## invoked.
+
+# RUN: rm -rf %t.dir && mkdir %t.dir
+# RUN: split-file %s %t.dir
+# RUN: %python %scripts_dir/corpus/combine_training_corpus.py --root_dir=%t.dir
+# RUN: cat %t.dir/corpus_description.json | FileCheck %s
+
+## Check that we end up with the same properties as the original corpora
+# CHECK: "has_thinlto": false
+
+## Check that the modules end up in the combined corpus. Order does not matter.
+# CHECK-DAG: "subcorpus1/test1.o"
+# CHECK-DAG: "subcorpus2/test2.o"
+
+#--- subcorpus1/corpus_description.json
+{
+  "has_thinlto": false,
+  "modules": [
+"test1.o"
+  ]
+}
+#--- subcorpus2/corpus_description.json
+{
+  "has_thinlto": false,
+  "modules": [
+"test2.o"
+  ]
+}
diff --git a/llvm/utils/mlgo-utils/tests/lit.cfg 
b/llvm/utils/mlgo-utils/tests/lit.cfg
index 9afced53f195c5f..58c35e69c652c58 100644
--- a/llvm/utils/mlgo-utils/tests/lit.cfg
+++ b/llvm/utils/mlgo-utils/tests/lit.cfg
@@ -15,6 +15,7 @@ config.test_exec_root = config.obj_root
 config.environment["PYTHONPATH"] = os.path.join(config.src_root, "utils", 
"mlgo-utils")
 
 

[clang] [CMake][PGO] Add libunwind to list of stage1 runtimes (PR #78869)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

This fixes #78487.

https://github.com/llvm/llvm-project/pull/78869
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [libc] [lldb] [compiler-rt] [libcxxabi] [flang] [lld] [libcxx] [clang] [clang-tools-extra] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/76653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang-tools-extra] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

I believe this is ready to go now. All the tests have been converted to 
lit-style tests and everything has been wired up into a `check-mlgo-utils` 
CMake target.

I talked with Mircea about the absl dependency. It's not in the tests anymore 
and the plan is to refactor in a subsequent commit the rest of the utilities 
that use absl to the generic python versions. For now the tests have been gated 
to actually having `absl` installed.

https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,6 @@
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.

boomanaiden154 wrote:

Looks like all the other Python tests within the monorepo are pretty much lit 
tests. I'll work on converting these tests to lit tests later today. Should be 
feasible since we're essentially just using Python tooling to validate where 
files are, which we can easily do in lit.

https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,6 @@
+# Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.

boomanaiden154 wrote:

Or what's probably better for structuring is we can do `mlgo/mlgo/corpus` and 
then the package would be accessed as `mlgo.corpus` while still keeping 
everything together if we want to add more in the future. I'll switch to that 
for now.

https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,12 @@
+# MLGO Python Library
+
+This folder contains the MLGO python library. This library consists of telling

boomanaiden154 wrote:

Updated it to calling this the folder for MLGO Python Utilities. Good catch on 
the first line. Not sure exactly what happened there.

https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libclang/python] Bump minimum compatibility to Python 3.6 (PR #77228)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/77228
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

> Would it be also possible to remove the dependency on 
> [Abseil](https://github.com/abseil/abseil-py)? None of the existing scripts 
> in LLVM use it and I don't think we should be introducing this dependency. It 
> looks like Abseil is only used for flag parsing, logging and testing; those 
> should be straightforward to replace with standard libraries like `argparse`, 
> `logging` or `unittest`.

Yes. My plan was to remove the dependency on abseil as well. My plan was to get 
this landed with all the infrastructure setup and the code basically just 
directly copied and then remove the abseil dependency in a follow-up patch so 
that the different pieces get reviewed appropriately.

https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[flang] [clang] [llvm] [clang-tools-extra] [compiler-rt] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 approved this pull request.


https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[flang] [clang] [llvm] [clang-tools-extra] [compiler-rt] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

@@ -26,21 +35,26 @@ jobs:
   prepare:
 name: Prepare to build binaries
 runs-on: ubuntu-22.04
-if: github.repository == 'llvm/llvm-project'
 outputs:
-  release-version: ${{ steps.validate-tag.outputs.release-version }}
-  flags: ${{ steps.validate-tag.outputs.flags }}
-  build-dir: ${{ steps.validate-tag.outputs.build-dir }}
-  rc-flags: ${{ steps.validate-tag.outputs.rc-flags }}
-  ref: ${{ steps.validate-tag.outputs.ref }}
-  upload: ${{ steps.validate-tag.outputs.upload }}
+  release-version: ${{ steps.vars.outputs.release-version }}
+  flags: ${{ steps.vars.outputs.flags }}
+  build-dir: ${{ steps.vars.outputs.build-dir }}
+  rc-flags: ${{ steps.vars.outputs.rc-flags }}
+  ref: ${{ steps.vars.outputs.ref }}
+  upload: ${{ steps.vars.outputs.upload }}
 
 steps:
 - name: Checkout LLVM
   uses: actions/checkout@v4
 
-- name: Validate and parse tag
-  id: validate-tag
+- name: Check Permissions
+  env:
+GITHUB_TOKEN: ${{ github.token }}
+  run: |
+./llvm/utils/release/./github-upload-release.py --token 
"$GITHUB_TOKEN" --user ${{ github.actor }} check-permissions

boomanaiden154 wrote:

Ah, makes sense. Wasn't thinking that anyone with commit access is allowed to 
run the job rather than just the release managers.

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [libcxx] [lld] [libc] [compiler-rt] [clang-tools-extra] [llvm] [flang] [lldb] [libcxxabi] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 edited 
https://github.com/llvm/llvm-project/pull/76653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [llvm] [clang-tools-extra] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

@@ -10,112 +10,70 @@ on:
   - 'llvmorg-*'
 
 jobs:
-  release-tasks:
-permissions:
-  contents: write # To upload assets to release.
+  validate-tag:
+name: Validate Tag
 runs-on: ubuntu-latest
 if: github.repository == 'llvm/llvm-project'
+outputs:
+  release-version: ${{ steps.validate-tag.outputs.release-version }}
 steps:
   - name: Validate Tag
 id: validate-tag
 run: |
-  test "${{ github.actor }}" = "tstellar" || test "${{ github.actor 
}}" = "tru"
   echo "${{ github.ref_name }}" | grep -e 
'^llvmorg-[0-9]\+\.[0-9]\+\.[0-9]\+\(-rc[0-9]\+\)\?$'
   release_version=$(echo "${{ github.ref_name }}" | sed 
's/llvmorg-//g')
   echo "release-version=$release_version" >> "$GITHUB_OUTPUT"
 
-  - name: Checkout LLVM
-uses: actions/checkout@v4
-
+  release-create:
+name: Create a New Release
+runs-on: ubuntu-latest
+needs: validate-tag
+steps:
   - name: Install Dependencies
 run: |
   sudo apt-get update
-  sudo apt-get install -y \
-  doxygen \
-  graphviz \
-  python3-github \
-  ninja-build \
-  texlive-font-utils
-  pip3 install --user -r ./llvm/docs/requirements.txt
-
-  - name: Create Release
-run: |
-  ./llvm/utils/release/./github-upload-release.py --token ${{ 
github.token }} --release ${{ steps.validate-tag.outputs.release-version }} 
create
-
-  - name: Build Documentation
-run: |
-  ./llvm/utils/release/build-docs.sh -release ${{ 
steps.validate-tag.outputs.release-version }}
-  ./llvm/utils/release/github-upload-release.py --token ${{ 
github.token }} --release ${{ steps.validate-tag.outputs.release-version }} 
upload --files ./*doxygen*.tar.xz
-
-  - name: Create Release Notes Artifact
-uses: actions/upload-artifact@v3
-with:
-  name: release-notes
-  path: docs-build/html-export/
-
-  - name: Clone www-releases
-if: ${{ !contains(steps.validate-tag.outputs.release-version, 'rc') }}
-uses: actions/checkout@v4
-with:
-  repository: ${{ github.repository_owner }}/www-releases
-  ref: main
-  fetch-depth: 0
-  path: www-releases
-
-  - name: Upload Release Notes
-if: ${{ !contains(steps.validate-tag.outputs.release-version, 'rc') }}
-run: |
-  mkdir -p ../www-releases/${{ 
steps.validate-tag.outputs.release-version }}
-  mv ./docs-build/html-export/* ../www-releases/${{ 
steps.validate-tag.outputs.release-version }}
-  cd ../www-releases
-  git add ${{ steps.validate-tag.outputs.release-version }}
-  git config user.email "llvm...@llvm.org"
-  git config user.name "llvmbot"
-  git commit -a -m "Add ${{ steps.validate-tag.outputs.release-version 
}} documentation"
-  git push https://${{ secrets.WWW_RELEASES_TOKEN }}@github.com/${{ 
github.repository_owner }}/www-releases main:main
+  sudo apt-get install python3-github

boomanaiden154 wrote:

Hashpinning the action dependencies can probably be done in a follow-up patch 
and that would help keep the diff clean.

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [clang] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,74 @@
+name: Release Lit
+
+permissions:
+  contents: read
+
+on:
+  workflow_dispatch:
+inputs:
+  release-version:
+description: 'Release Version'
+required: true
+type: string
+
+  workflow_call:
+inputs:
+  release-version:
+description: 'Release Version'
+required: true
+type: string
+
+jobs:
+  release-lit:
+name: Release Lit
+runs-on: ubuntu-latest
+steps:
+  - name: Checkout LLVM
+uses: actions/checkout@v4
+with:
+  ref: "llvmorg-${{ inputs.release-version }}"
+
+  - name: Install dependencies
+run: |
+  sudo apt-get update
+  sudo apt-get install -y python3-setuptools python3-psutil 
python3-github
+
+  - name: Check Permissions
+env:
+  GITHUB_TOKEN: ${{ github.token }}
+run: |
+  ./llvm/utils/release/./github-upload-release.py --token 
"$GITHUB_TOKEN" --user ${{ github.actor }} check-permissions
+
+  - name: Setup Cpp
+uses: aminya/setup-cpp@v1
+with:
+  compiler: llvm-16.0.6

boomanaiden154 wrote:

Ah, yep.
https://github.com/llvm/llvm-project/blob/84f7fb6217fd417f3b5cb65fe7636e0aab84f6c7/llvm/utils/lit/CMakeLists.txt#L25C1-L25C1

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang] [clang-tools-extra] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

@@ -26,21 +35,26 @@ jobs:
   prepare:
 name: Prepare to build binaries
 runs-on: ubuntu-22.04
-if: github.repository == 'llvm/llvm-project'
 outputs:
-  release-version: ${{ steps.validate-tag.outputs.release-version }}
-  flags: ${{ steps.validate-tag.outputs.flags }}
-  build-dir: ${{ steps.validate-tag.outputs.build-dir }}
-  rc-flags: ${{ steps.validate-tag.outputs.rc-flags }}
-  ref: ${{ steps.validate-tag.outputs.ref }}
-  upload: ${{ steps.validate-tag.outputs.upload }}
+  release-version: ${{ steps.vars.outputs.release-version }}
+  flags: ${{ steps.vars.outputs.flags }}
+  build-dir: ${{ steps.vars.outputs.build-dir }}
+  rc-flags: ${{ steps.vars.outputs.rc-flags }}
+  ref: ${{ steps.vars.outputs.ref }}
+  upload: ${{ steps.vars.outputs.upload }}
 
 steps:
 - name: Checkout LLVM
   uses: actions/checkout@v4
 
-- name: Validate and parse tag
-  id: validate-tag
+- name: Check Permissions
+  env:
+GITHUB_TOKEN: ${{ github.token }}
+  run: |
+./llvm/utils/release/./github-upload-release.py --token 
"$GITHUB_TOKEN" --user ${{ github.actor }} check-permissions

boomanaiden154 wrote:

What's the purpose of this permissions check?

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang-tools-extra] [clang] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

@@ -10,112 +10,70 @@ on:
   - 'llvmorg-*'
 
 jobs:
-  release-tasks:
-permissions:
-  contents: write # To upload assets to release.
+  validate-tag:
+name: Validate Tag
 runs-on: ubuntu-latest
 if: github.repository == 'llvm/llvm-project'
+outputs:
+  release-version: ${{ steps.validate-tag.outputs.release-version }}
 steps:
   - name: Validate Tag
 id: validate-tag
 run: |
-  test "${{ github.actor }}" = "tstellar" || test "${{ github.actor 
}}" = "tru"
   echo "${{ github.ref_name }}" | grep -e 
'^llvmorg-[0-9]\+\.[0-9]\+\.[0-9]\+\(-rc[0-9]\+\)\?$'
   release_version=$(echo "${{ github.ref_name }}" | sed 
's/llvmorg-//g')
   echo "release-version=$release_version" >> "$GITHUB_OUTPUT"
 
-  - name: Checkout LLVM
-uses: actions/checkout@v4
-
+  release-create:
+name: Create a New Release
+runs-on: ubuntu-latest
+needs: validate-tag
+steps:
   - name: Install Dependencies
 run: |
   sudo apt-get update
-  sudo apt-get install -y \
-  doxygen \
-  graphviz \
-  python3-github \
-  ninja-build \
-  texlive-font-utils
-  pip3 install --user -r ./llvm/docs/requirements.txt
-
-  - name: Create Release
-run: |
-  ./llvm/utils/release/./github-upload-release.py --token ${{ 
github.token }} --release ${{ steps.validate-tag.outputs.release-version }} 
create
-
-  - name: Build Documentation
-run: |
-  ./llvm/utils/release/build-docs.sh -release ${{ 
steps.validate-tag.outputs.release-version }}
-  ./llvm/utils/release/github-upload-release.py --token ${{ 
github.token }} --release ${{ steps.validate-tag.outputs.release-version }} 
upload --files ./*doxygen*.tar.xz
-
-  - name: Create Release Notes Artifact
-uses: actions/upload-artifact@v3
-with:
-  name: release-notes
-  path: docs-build/html-export/
-
-  - name: Clone www-releases
-if: ${{ !contains(steps.validate-tag.outputs.release-version, 'rc') }}
-uses: actions/checkout@v4
-with:
-  repository: ${{ github.repository_owner }}/www-releases
-  ref: main
-  fetch-depth: 0
-  path: www-releases
-
-  - name: Upload Release Notes
-if: ${{ !contains(steps.validate-tag.outputs.release-version, 'rc') }}
-run: |
-  mkdir -p ../www-releases/${{ 
steps.validate-tag.outputs.release-version }}
-  mv ./docs-build/html-export/* ../www-releases/${{ 
steps.validate-tag.outputs.release-version }}
-  cd ../www-releases
-  git add ${{ steps.validate-tag.outputs.release-version }}
-  git config user.email "llvm...@llvm.org"
-  git config user.name "llvmbot"
-  git commit -a -m "Add ${{ steps.validate-tag.outputs.release-version 
}} documentation"
-  git push https://${{ secrets.WWW_RELEASES_TOKEN }}@github.com/${{ 
github.repository_owner }}/www-releases main:main
+  sudo apt-get install python3-github

boomanaiden154 wrote:

Might not be a major issue, but it would be good to version match this with 
other workflows that use the `github` python package.

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

@@ -0,0 +1,74 @@
+name: Release Lit
+
+permissions:
+  contents: read
+
+on:
+  workflow_dispatch:
+inputs:
+  release-version:
+description: 'Release Version'
+required: true
+type: string
+
+  workflow_call:
+inputs:
+  release-version:
+description: 'Release Version'
+required: true
+type: string
+
+jobs:
+  release-lit:
+name: Release Lit
+runs-on: ubuntu-latest
+steps:
+  - name: Checkout LLVM
+uses: actions/checkout@v4
+with:
+  ref: "llvmorg-${{ inputs.release-version }}"
+
+  - name: Install dependencies
+run: |
+  sudo apt-get update
+  sudo apt-get install -y python3-setuptools python3-psutil 
python3-github
+
+  - name: Check Permissions
+env:
+  GITHUB_TOKEN: ${{ github.token }}
+run: |
+  ./llvm/utils/release/./github-upload-release.py --token 
"$GITHUB_TOKEN" --user ${{ github.actor }} check-permissions
+
+  - name: Setup Cpp
+uses: aminya/setup-cpp@v1
+with:
+  compiler: llvm-16.0.6

boomanaiden154 wrote:

Why do we need a full C/C++ toolchain to build lit. Does ninja `check-lit` end 
up building things other than lit?

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [clang] [llvm] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 commented:

No issues on my end. Definitely seems to be an improvement over what was there 
before.

It would be nice to unify the documentation test action and the documentation 
release action (at least somewhat), but it doesn't seem like they're 
particularly amenable to being combined given the different purposes they serve.

https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[llvm] [clang-tools-extra] [clang] workflows: Refactor release-tasks.yml (PR #69523)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 edited 
https://github.com/llvm/llvm-project/pull/69523
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [clang] [llvm] Hashpin sensitive dependencies and configure dependabot to update them automatically (PR #75859)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/75859
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

After this lands, my plan is to work on getting CI up and running, both to run 
testing and also to publish the package.

https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 ready_for_review 
https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 edited 
https://github.com/llvm/llvm-project/pull/72319
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/72319

>From c3f723c8a975cc5e075d56350645b0be486f3cda Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Tue, 14 Nov 2023 14:20:24 -0800
Subject: [PATCH 1/2] [MLGO] Upstream the corpus extraction tooling

---
 llvm/py/Pyproject.toml|   1 +
 llvm/py/src/mlgo/combine_training_corpus.py   |  55 +++
 .../src/mlgo/combine_training_corpus_lib.py   |  50 +++
 .../src/mlgo/combine_training_corpus_test.py  | 104 +
 llvm/py/src/mlgo/extract_ir.py| 142 +++
 llvm/py/src/mlgo/extract_ir_lib.py| 373 ++
 llvm/py/src/mlgo/extract_ir_test.py   | 231 +++
 llvm/py/src/mlgo/make_corpus.py   |  58 +++
 llvm/py/src/mlgo/make_corpus_lib.py   |  90 +
 llvm/py/src/mlgo/make_corpus_test.py  |  66 
 10 files changed, 1170 insertions(+)
 create mode 100644 llvm/py/Pyproject.toml
 create mode 100644 llvm/py/src/mlgo/combine_training_corpus.py
 create mode 100644 llvm/py/src/mlgo/combine_training_corpus_lib.py
 create mode 100644 llvm/py/src/mlgo/combine_training_corpus_test.py
 create mode 100644 llvm/py/src/mlgo/extract_ir.py
 create mode 100644 llvm/py/src/mlgo/extract_ir_lib.py
 create mode 100644 llvm/py/src/mlgo/extract_ir_test.py
 create mode 100644 llvm/py/src/mlgo/make_corpus.py
 create mode 100644 llvm/py/src/mlgo/make_corpus_lib.py
 create mode 100644 llvm/py/src/mlgo/make_corpus_test.py

diff --git a/llvm/py/Pyproject.toml b/llvm/py/Pyproject.toml
new file mode 100644
index 00..dcf2c804da5e19
--- /dev/null
+++ b/llvm/py/Pyproject.toml
@@ -0,0 +1 @@
+# Placeholder
diff --git a/llvm/py/src/mlgo/combine_training_corpus.py 
b/llvm/py/src/mlgo/combine_training_corpus.py
new file mode 100644
index 00..94ee1cbac9cea4
--- /dev/null
+++ b/llvm/py/src/mlgo/combine_training_corpus.py
@@ -0,0 +1,55 @@
+# coding=utf-8
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+r"""Combine multiple training corpus into a single training corpus.
+
+Currently only support the case that multiple corpus share the same
+configurables except the "modules" field.
+
+Usage: we'd like to combine training corpus corpus1 and corpus2 into
+combinedcorpus; we first structure the files as follows:
+
+combinedcorpus
+combinedcorpus/corpus1
+combinedcorpus/corpus2
+
+Running this script with
+
+python3 \
+compiler_opt/tools/combine_training_corpus.py \
+  --root_dir=$PATH_TO_combinedcorpus
+
+generates combinedcorpus/corpus_description.json file. In this way corpus1
+and corpus2 are combined into combinedcorpus.
+"""
+
+from absl import app
+from absl import flags
+
+from compiler_opt.tools import combine_training_corpus_lib
+
+flags.DEFINE_string('root_dir', '', 'root dir of module paths to combine.')
+
+FLAGS = flags.FLAGS
+
+
+def main(argv):
+  if len(argv) > 1:
+raise app.UsageError('Too many command-line arguments.')
+
+  combine_training_corpus_lib.combine_corpus(FLAGS.root_dir)
+
+
+if __name__ == '__main__':
+  app.run(main)
diff --git a/llvm/py/src/mlgo/combine_training_corpus_lib.py 
b/llvm/py/src/mlgo/combine_training_corpus_lib.py
new file mode 100644
index 00..0359961266a240
--- /dev/null
+++ b/llvm/py/src/mlgo/combine_training_corpus_lib.py
@@ -0,0 +1,50 @@
+# coding=utf-8
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Library for combining training corpora."""
+
+import os
+import json
+
+from absl import logging
+
+import tensorflow as tf
+
+_FILE_NAME = 'corpus_description.json'
+
+
+def combine_corpus(root_dir: str) -> None:
+  module_names = []
+  output_corpus_description = {}
+
+  corpus_description_glob = os.path.join(root_dir, '*/' + _FILE_NAME)
+  for corpus_description_path in tf.io.gfile.glob(corpus_description_glob):
+logging.info('processing %s', corpus_description_path)
+
+with 

[clang-tools-extra] [llvm] [MLGO] Upstream the corpus extraction tooling (PR #72319)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/72319

>From c3f723c8a975cc5e075d56350645b0be486f3cda Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Tue, 14 Nov 2023 14:20:24 -0800
Subject: [PATCH] [MLGO] Upstream the corpus extraction tooling

---
 llvm/py/Pyproject.toml|   1 +
 llvm/py/src/mlgo/combine_training_corpus.py   |  55 +++
 .../src/mlgo/combine_training_corpus_lib.py   |  50 +++
 .../src/mlgo/combine_training_corpus_test.py  | 104 +
 llvm/py/src/mlgo/extract_ir.py| 142 +++
 llvm/py/src/mlgo/extract_ir_lib.py| 373 ++
 llvm/py/src/mlgo/extract_ir_test.py   | 231 +++
 llvm/py/src/mlgo/make_corpus.py   |  58 +++
 llvm/py/src/mlgo/make_corpus_lib.py   |  90 +
 llvm/py/src/mlgo/make_corpus_test.py  |  66 
 10 files changed, 1170 insertions(+)
 create mode 100644 llvm/py/Pyproject.toml
 create mode 100644 llvm/py/src/mlgo/combine_training_corpus.py
 create mode 100644 llvm/py/src/mlgo/combine_training_corpus_lib.py
 create mode 100644 llvm/py/src/mlgo/combine_training_corpus_test.py
 create mode 100644 llvm/py/src/mlgo/extract_ir.py
 create mode 100644 llvm/py/src/mlgo/extract_ir_lib.py
 create mode 100644 llvm/py/src/mlgo/extract_ir_test.py
 create mode 100644 llvm/py/src/mlgo/make_corpus.py
 create mode 100644 llvm/py/src/mlgo/make_corpus_lib.py
 create mode 100644 llvm/py/src/mlgo/make_corpus_test.py

diff --git a/llvm/py/Pyproject.toml b/llvm/py/Pyproject.toml
new file mode 100644
index 00..dcf2c804da5e19
--- /dev/null
+++ b/llvm/py/Pyproject.toml
@@ -0,0 +1 @@
+# Placeholder
diff --git a/llvm/py/src/mlgo/combine_training_corpus.py 
b/llvm/py/src/mlgo/combine_training_corpus.py
new file mode 100644
index 00..94ee1cbac9cea4
--- /dev/null
+++ b/llvm/py/src/mlgo/combine_training_corpus.py
@@ -0,0 +1,55 @@
+# coding=utf-8
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+r"""Combine multiple training corpus into a single training corpus.
+
+Currently only support the case that multiple corpus share the same
+configurables except the "modules" field.
+
+Usage: we'd like to combine training corpus corpus1 and corpus2 into
+combinedcorpus; we first structure the files as follows:
+
+combinedcorpus
+combinedcorpus/corpus1
+combinedcorpus/corpus2
+
+Running this script with
+
+python3 \
+compiler_opt/tools/combine_training_corpus.py \
+  --root_dir=$PATH_TO_combinedcorpus
+
+generates combinedcorpus/corpus_description.json file. In this way corpus1
+and corpus2 are combined into combinedcorpus.
+"""
+
+from absl import app
+from absl import flags
+
+from compiler_opt.tools import combine_training_corpus_lib
+
+flags.DEFINE_string('root_dir', '', 'root dir of module paths to combine.')
+
+FLAGS = flags.FLAGS
+
+
+def main(argv):
+  if len(argv) > 1:
+raise app.UsageError('Too many command-line arguments.')
+
+  combine_training_corpus_lib.combine_corpus(FLAGS.root_dir)
+
+
+if __name__ == '__main__':
+  app.run(main)
diff --git a/llvm/py/src/mlgo/combine_training_corpus_lib.py 
b/llvm/py/src/mlgo/combine_training_corpus_lib.py
new file mode 100644
index 00..0359961266a240
--- /dev/null
+++ b/llvm/py/src/mlgo/combine_training_corpus_lib.py
@@ -0,0 +1,50 @@
+# coding=utf-8
+# Copyright 2020 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#  http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""Library for combining training corpora."""
+
+import os
+import json
+
+from absl import logging
+
+import tensorflow as tf
+
+_FILE_NAME = 'corpus_description.json'
+
+
+def combine_corpus(root_dir: str) -> None:
+  module_names = []
+  output_corpus_description = {}
+
+  corpus_description_glob = os.path.join(root_dir, '*/' + _FILE_NAME)
+  for corpus_description_path in tf.io.gfile.glob(corpus_description_glob):
+logging.info('processing %s', corpus_description_path)
+
+with 

[clang-tools-extra] [lld] [clang] [llvm] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

@@ -112,9 +116,11 @@ class Counter {
   PerfEvent Event;
   int FileDescriptor = -1;
   bool IsDummyEvent;
+  std::vector ValidationEvents;

boomanaiden154 wrote:

My understanding is that `Counter` was an abstraction over a specific event 
that we want to measure (like uops through a specific port or cycles) rather 
than over an individual performance counter created with `perf_event_open`.

It is a little weird (maybe renaming it to something like `CounterGroup` would 
make this better?), but I think this still makes the most sense given the tight 
coupling between the validation counters and the actual event counter both 
conceptually and in regards to how everything is set up in the perf subsystem 
(as an individual group).

https://github.com/llvm/llvm-project/pull/76653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [clang] [lld] [llvm] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

@@ -87,6 +93,8 @@ struct BenchmarkMeasure {
   // This is the per-snippet value, i.e. measured quantity for one repetition 
of
   // the whole snippet.
   double PerSnippetValue;
+  // These are the validation counter values.
+  std::unordered_map ValidationCounters;

boomanaiden154 wrote:

Good point. Updated.

https://github.com/llvm/llvm-project/pull/76653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] [lld] [clang] [llvm] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76653

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 01/14] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 02/14] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure 

[clang-tools-extra] [clang] [lld] [llvm] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76653

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 01/13] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 02/13] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure 

[clang-tools-extra] [llvm] [lld] [clang] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76653

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 01/12] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 02/12] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure 

[llvm] [clang] [clang-tools-extra] [lld] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76653

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 01/12] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 02/12] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure 

[llvm] [clang] [clang-tools-extra] [llvm-exegesis] Add support for validation counters (PR #76653)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 edited 
https://github.com/llvm/llvm-project/pull/76653
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Add tablegen support for validation counters (PR #76652)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/76652
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-tools-extra] [llvm] [llvm-exegesis] Add tablegen support for validation counters (PR #76652)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76652

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 1/7] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 2/7] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure the 

[llvm] [clang] [clang-tools-extra] [llvm-exegesis] Add tablegen support for validation counters (PR #76652)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76652

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 1/6] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 2/6] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure the 

[clang] [llvm] [clang-tools-extra] [llvm-exegesis] Add tablegen support for validation counters (PR #76652)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/76652

>From 0196a8780f2d3a378f0fd3b86a6e6b6590ca02de Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 15:37:56 -0800
Subject: [PATCH 1/5] [llvm-exegesis] Remove unused Counter::read method

This method was simply a wrapper around readOrError. All users within
the llvm-exegesis code base should have been processing an actual error
rather than using the wrapper. This patch removes the wrapper and
rewrites the users (just 1) to use the readOrError method.
---
 llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp |  6 +-
 llvm/tools/llvm-exegesis/lib/PerfHelper.cpp  | 13 -
 llvm/tools/llvm-exegesis/lib/PerfHelper.h|  3 ---
 3 files changed, 5 insertions(+), 17 deletions(-)

diff --git a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp 
b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
index 1ee59a86ebbdcf..c57fce970b2139 100644
--- a/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
+++ b/llvm/tools/llvm-exegesis/lib/BenchmarkRunner.cpp
@@ -356,7 +356,11 @@ class SubProcessFunctionExecutorImpl
   if (ChildExitCode == 0) {
 // The child exited succesfully, read counter values and return
 // success
-CounterValues[0] = Counter->read();
+auto CounterValueOrErr = Counter->readOrError();
+if (!CounterValueOrErr)
+  return CounterValueOrErr.takeError();
+CounterValues.swap(*CounterValueOrErr);
+
 return Error::success();
   }
   // The child exited, but not successfully
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
index 3ff1745e9e0629..314de1ec32366f 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.cpp
@@ -148,17 +148,6 @@ void Counter::stop() {
 ioctl(FileDescriptor, PERF_EVENT_IOC_DISABLE, 0);
 }
 
-int64_t Counter::read() const {
-  auto ValueOrError = readOrError();
-  if (ValueOrError) {
-if (!ValueOrError.get().empty())
-  return ValueOrError.get()[0];
-errs() << "Counter has no reading\n";
-  } else
-errs() << ValueOrError.takeError() << "\n";
-  return -1;
-}
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   int64_t Count = 0;
@@ -187,8 +176,6 @@ void Counter::start() {}
 
 void Counter::stop() {}
 
-int64_t Counter::read() const { return 42; }
-
 llvm::Expected>
 Counter::readOrError(StringRef /*unused*/) const {
   if (IsDummyEvent) {
diff --git a/llvm/tools/llvm-exegesis/lib/PerfHelper.h 
b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
index a50974f0a67be9..894aac1f197ed1 100644
--- a/llvm/tools/llvm-exegesis/lib/PerfHelper.h
+++ b/llvm/tools/llvm-exegesis/lib/PerfHelper.h
@@ -95,9 +95,6 @@ class Counter {
   /// Stops the measurement of the event.
   void stop();
 
-  /// Returns the current value of the counter or -1 if it cannot be read.
-  int64_t read() const;
-
   /// Returns the current value of the counter or error if it cannot be read.
   /// FunctionBytes: The benchmark function being executed.
   /// This is used to filter out the measurements to ensure they are only

>From 75261b9626878b92fa4bde2b4801815251238748 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 30 Dec 2023 17:03:59 -0800
Subject: [PATCH 2/5] [llvm-exegesis] Add tablegen support for validation
 counters

---
 llvm/include/llvm/Target/TargetPfmCounters.td | 19 +
 llvm/lib/Target/X86/X86PfmCounters.td |  6 
 llvm/tools/llvm-exegesis/lib/Target.cpp   | 13 +
 llvm/tools/llvm-exegesis/lib/Target.h | 11 
 llvm/utils/TableGen/ExegesisEmitter.cpp   | 28 +--
 5 files changed, 69 insertions(+), 8 deletions(-)

diff --git a/llvm/include/llvm/Target/TargetPfmCounters.td 
b/llvm/include/llvm/Target/TargetPfmCounters.td
index b00f3e19c35f90..72f6b39f4878f4 100644
--- a/llvm/include/llvm/Target/TargetPfmCounters.td
+++ b/llvm/include/llvm/Target/TargetPfmCounters.td
@@ -28,6 +28,22 @@ class PfmIssueCounter
   string ResourceName = resource_name;
 }
 
+class ValidationEvent  {
+  int EventNumber = event_number;
+}
+
+def L1DCacheLoadMiss: ValidationEvent<0>;
+def InstructionRetired  : ValidationEvent<1>;
+def DataTLBLoadMiss : ValidationEvent<2>;
+def DataTLBStoreMiss: ValidationEvent<3>;
+
+// Validation counters can be tied to a specific event
+class PfmValidationCounter
+: PfmCounter {
+  // The name of the event that the validation counter detects.
+  ValidationEvent EventType = event_type;
+}
+
 def NoPfmCounter : PfmCounter <""> {}
 
 // Set of PfmCounters for measuring sched model characteristics.
@@ -38,6 +54,9 @@ class ProcPfmCounters {
   PfmCounter UopsCounter = NoPfmCounter;
   // Processors can define how to measure issued uops by defining 
IssueCounters.
   list IssueCounters = [];
+  // Processor can list mappings between validation events and real counters
+  // to measure the 

[clang-tools-extra] [clang] [llvm] code-format: Improve the code-format-helper to be able to run as a git hook (PR #73957)

[Aiden Grossman via cfe-commits]

@@ -1,21 +1,56 @@
 #!/usr/bin/env python3
 #
-# - code-format-helper, runs code formatters from the ci --*- python 
-*--==#
+# - code-format-helper, runs code formatters from the ci or in a hook --*- 
python -*--==#
 #
 # Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
 # See https://llvm.org/LICENSE.txt for license information.
 # SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
 #
-# 
==-==#
+# 
==--==#
 
 import argparse
 import os
 import subprocess
 import sys
-from functools import cached_property
+from typing import List, Optional
 
-import github
-from github import IssueComment, PullRequest
+"""
+This script is run by GitHub actions to ensure that the code in PR's conform to
+the coding style of LLVM. It can also be installed as a pre-commit git hook to
+check the coding style before submitting it. The canonical source of this 
script
+is in the LLVM source tree under llvm/utils/git.
+
+For C/C++ code it uses clang-format and for Python code it uses darker (which
+in turn invokes black).
+
+You can learn more about the LLVM coding style on llvm.org:
+https://llvm.org/docs/CodingStandards.html
+
+You can install this script as a git hook by symlinking it to the .git/hooks
+directory:
+
+ln -s $(pwd)/llvm/utils/git/code-format-helper.py .git/hooks/pre-commit
+
+You can control the exact path to clang-format or darker with the following
+environment variables: $CLANG_FORMAT_PATH and $DARKER_FORMAT_PATH.
+"""
+
+
+class FormatArgs:

boomanaiden154 wrote:

Ah. Didn't realize it required a special header in python versions before 3.7. 
Thanks for taking a look!

https://github.com/llvm/llvm-project/pull/73957
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[compiler-rt] [clang] [llvm] [flang] [libc] [clang-tools-extra] [clang-format] Add "three dot" diff option to git-clang-format (PR #74230)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/74230
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-format] Add "three dot" diff option to git-clang-format (PR #74230)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/74230

>From ae4097b53b90e31802be0be5c8a81fb74c81efc9 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 2 Dec 2023 23:46:58 -0800
Subject: [PATCH 1/3] [clang-format] Add "three dot" diff option to
 git-clang-format

This patch adds in the ability to do a "three dot" git-clang-format
between two commits. This looks at the diff between the second commit
and the common merge base rather than comparing at the point of the
specified commits. This is needed to improve the reliability of the LLVM
code formatting CI action which currently breaks in some cases where
files have been modified in the upstream tree and when the person
created their branch, leaving phantom formatting diffs that weren't
touched by the PR author.

Part of a fix for #73873
---
 clang/tools/clang-format/git-clang-format | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/clang/tools/clang-format/git-clang-format 
b/clang/tools/clang-format/git-clang-format
index 6e827e17b4ee2..4fe4671482e3f 100755
--- a/clang/tools/clang-format/git-clang-format
+++ b/clang/tools/clang-format/git-clang-format
@@ -132,6 +132,10 @@ def main():
  help='passed to clang-format'),
   p.add_argument('-v', '--verbose', action='count', default=0,
  help='print extra information')
+  p.add_argument('--diff_from_common_commit', action='store_true',
+ help=('diff from the last common commit for commits in '
+  'separate branches rather than the exact point of the '
+  'commits'))
   # We gather all the remaining positional arguments into 'args' since we need
   # to use some heuristics to determine whether or not  was present.
   # However, to print pretty messages, we make use of metavar and help.
@@ -154,7 +158,10 @@ def main():
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
   opts.binary=os.path.abspath(opts.binary)
-  changed_lines = compute_diff_and_extract_lines(commits, files, opts.staged)
+  changed_lines = compute_diff_and_extract_lines(commits,
+ files,
+ opts.staged,
+ opts.diff_from_common_commit)
   if opts.verbose >= 1:
 ignored_files = set(changed_lines)
   filter_by_extension(changed_lines, opts.extensions.lower().split(','))
@@ -302,9 +309,9 @@ def get_object_type(value):
   return convert_string(stdout.strip())
 
 
-def compute_diff_and_extract_lines(commits, files, staged):
+def compute_diff_and_extract_lines(commits, files, staged, diff_common_commit):
   """Calls compute_diff() followed by extract_lines()."""
-  diff_process = compute_diff(commits, files, staged)
+  diff_process = compute_diff(commits, files, staged, diff_common_commit)
   changed_lines = extract_lines(diff_process.stdout)
   diff_process.stdout.close()
   diff_process.wait()
@@ -314,7 +321,7 @@ def compute_diff_and_extract_lines(commits, files, staged):
   return changed_lines
 
 
-def compute_diff(commits, files, staged):
+def compute_diff(commits, files, staged, diff_common_commit):
   """Return a subprocess object producing the diff from `commits`.
 
   The return value's `stdin` file object will produce a patch with the
@@ -328,6 +335,10 @@ def compute_diff(commits, files, staged):
 git_tool = 'diff-tree'
   elif staged:
 extra_args += ['--cached']
+
+  if len(commits) > 1 and diff_common_commit:
+commits = [f'{commits[0]}...{commits[1]}']
+
   cmd = ['git', git_tool, '-p', '-U0'] + extra_args + commits + ['--']
   cmd.extend(files)
   p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

>From 4584ebc58337f04290a5c753f585426c8c6f739f Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sun, 3 Dec 2023 13:08:16 -0800
Subject: [PATCH 2/3] Add early exit when flag is specified with invalid
 parameters

---
 clang/tools/clang-format/git-clang-format | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/clang/tools/clang-format/git-clang-format 
b/clang/tools/clang-format/git-clang-format
index 4fe4671482e3f..51ae8739bee0e 100755
--- a/clang/tools/clang-format/git-clang-format
+++ b/clang/tools/clang-format/git-clang-format
@@ -157,6 +157,8 @@ def main():
   else:
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
+  if len(commits) < 2 and opts.diff_from_common_commit:
+die('--diff_from_common_commit is only allowed when two commits are given')
   opts.binary=os.path.abspath(opts.binary)
   changed_lines = compute_diff_and_extract_lines(commits,
  files,

>From a8b385070845ba20f63f1e0c6835ff9bc609548f Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Tue, 5 Dec 2023 17:42:12 -0800
Subject: [PATCH 3/3] Address reviewer feedback

---
 

[clang] [clang-format] Add "three dot" diff option to git-clang-format (PR #74230)

[Aiden Grossman via cfe-commits]

@@ -153,7 +157,10 @@ def main():
   else:
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
-  changed_lines = compute_diff_and_extract_lines(commits, files, opts.staged)
+  changed_lines = compute_diff_and_extract_lines(commits,

boomanaiden154 wrote:

Fixed. Thanks for the suggestion!

https://github.com/llvm/llvm-project/pull/74230
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-format] Add "three dot" diff option to git-clang-format (PR #74230)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/74230

>From ae4097b53b90e31802be0be5c8a81fb74c81efc9 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 2 Dec 2023 23:46:58 -0800
Subject: [PATCH 1/2] [clang-format] Add "three dot" diff option to
 git-clang-format

This patch adds in the ability to do a "three dot" git-clang-format
between two commits. This looks at the diff between the second commit
and the common merge base rather than comparing at the point of the
specified commits. This is needed to improve the reliability of the LLVM
code formatting CI action which currently breaks in some cases where
files have been modified in the upstream tree and when the person
created their branch, leaving phantom formatting diffs that weren't
touched by the PR author.

Part of a fix for #73873
---
 clang/tools/clang-format/git-clang-format | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/clang/tools/clang-format/git-clang-format 
b/clang/tools/clang-format/git-clang-format
index 6e827e17b4ee2..4fe4671482e3f 100755
--- a/clang/tools/clang-format/git-clang-format
+++ b/clang/tools/clang-format/git-clang-format
@@ -132,6 +132,10 @@ def main():
  help='passed to clang-format'),
   p.add_argument('-v', '--verbose', action='count', default=0,
  help='print extra information')
+  p.add_argument('--diff_from_common_commit', action='store_true',
+ help=('diff from the last common commit for commits in '
+  'separate branches rather than the exact point of the '
+  'commits'))
   # We gather all the remaining positional arguments into 'args' since we need
   # to use some heuristics to determine whether or not  was present.
   # However, to print pretty messages, we make use of metavar and help.
@@ -154,7 +158,10 @@ def main():
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
   opts.binary=os.path.abspath(opts.binary)
-  changed_lines = compute_diff_and_extract_lines(commits, files, opts.staged)
+  changed_lines = compute_diff_and_extract_lines(commits,
+ files,
+ opts.staged,
+ opts.diff_from_common_commit)
   if opts.verbose >= 1:
 ignored_files = set(changed_lines)
   filter_by_extension(changed_lines, opts.extensions.lower().split(','))
@@ -302,9 +309,9 @@ def get_object_type(value):
   return convert_string(stdout.strip())
 
 
-def compute_diff_and_extract_lines(commits, files, staged):
+def compute_diff_and_extract_lines(commits, files, staged, diff_common_commit):
   """Calls compute_diff() followed by extract_lines()."""
-  diff_process = compute_diff(commits, files, staged)
+  diff_process = compute_diff(commits, files, staged, diff_common_commit)
   changed_lines = extract_lines(diff_process.stdout)
   diff_process.stdout.close()
   diff_process.wait()
@@ -314,7 +321,7 @@ def compute_diff_and_extract_lines(commits, files, staged):
   return changed_lines
 
 
-def compute_diff(commits, files, staged):
+def compute_diff(commits, files, staged, diff_common_commit):
   """Return a subprocess object producing the diff from `commits`.
 
   The return value's `stdin` file object will produce a patch with the
@@ -328,6 +335,10 @@ def compute_diff(commits, files, staged):
 git_tool = 'diff-tree'
   elif staged:
 extra_args += ['--cached']
+
+  if len(commits) > 1 and diff_common_commit:
+commits = [f'{commits[0]}...{commits[1]}']
+
   cmd = ['git', git_tool, '-p', '-U0'] + extra_args + commits + ['--']
   cmd.extend(files)
   p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

>From 4584ebc58337f04290a5c753f585426c8c6f739f Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sun, 3 Dec 2023 13:08:16 -0800
Subject: [PATCH 2/2] Add early exit when flag is specified with invalid
 parameters

---
 clang/tools/clang-format/git-clang-format | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/clang/tools/clang-format/git-clang-format 
b/clang/tools/clang-format/git-clang-format
index 4fe4671482e3f..51ae8739bee0e 100755
--- a/clang/tools/clang-format/git-clang-format
+++ b/clang/tools/clang-format/git-clang-format
@@ -157,6 +157,8 @@ def main():
   else:
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
+  if len(commits) < 2 and opts.diff_from_common_commit:
+die('--diff_from_common_commit is only allowed when two commits are given')
   opts.binary=os.path.abspath(opts.binary)
   changed_lines = compute_diff_and_extract_lines(commits,
  files,

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-format] Add "three dot" diff option to git-clang-format (PR #74230)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 updated 
https://github.com/llvm/llvm-project/pull/74230

>From ae4097b53b90e31802be0be5c8a81fb74c81efc9 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 2 Dec 2023 23:46:58 -0800
Subject: [PATCH] [clang-format] Add "three dot" diff option to
 git-clang-format

This patch adds in the ability to do a "three dot" git-clang-format
between two commits. This looks at the diff between the second commit
and the common merge base rather than comparing at the point of the
specified commits. This is needed to improve the reliability of the LLVM
code formatting CI action which currently breaks in some cases where
files have been modified in the upstream tree and when the person
created their branch, leaving phantom formatting diffs that weren't
touched by the PR author.

Part of a fix for #73873
---
 clang/tools/clang-format/git-clang-format | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/clang/tools/clang-format/git-clang-format 
b/clang/tools/clang-format/git-clang-format
index 6e827e17b4ee2..4fe4671482e3f 100755
--- a/clang/tools/clang-format/git-clang-format
+++ b/clang/tools/clang-format/git-clang-format
@@ -132,6 +132,10 @@ def main():
  help='passed to clang-format'),
   p.add_argument('-v', '--verbose', action='count', default=0,
  help='print extra information')
+  p.add_argument('--diff_from_common_commit', action='store_true',
+ help=('diff from the last common commit for commits in '
+  'separate branches rather than the exact point of the '
+  'commits'))
   # We gather all the remaining positional arguments into 'args' since we need
   # to use some heuristics to determine whether or not  was present.
   # However, to print pretty messages, we make use of metavar and help.
@@ -154,7 +158,10 @@ def main():
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
   opts.binary=os.path.abspath(opts.binary)
-  changed_lines = compute_diff_and_extract_lines(commits, files, opts.staged)
+  changed_lines = compute_diff_and_extract_lines(commits,
+ files,
+ opts.staged,
+ opts.diff_from_common_commit)
   if opts.verbose >= 1:
 ignored_files = set(changed_lines)
   filter_by_extension(changed_lines, opts.extensions.lower().split(','))
@@ -302,9 +309,9 @@ def get_object_type(value):
   return convert_string(stdout.strip())
 
 
-def compute_diff_and_extract_lines(commits, files, staged):
+def compute_diff_and_extract_lines(commits, files, staged, diff_common_commit):
   """Calls compute_diff() followed by extract_lines()."""
-  diff_process = compute_diff(commits, files, staged)
+  diff_process = compute_diff(commits, files, staged, diff_common_commit)
   changed_lines = extract_lines(diff_process.stdout)
   diff_process.stdout.close()
   diff_process.wait()
@@ -314,7 +321,7 @@ def compute_diff_and_extract_lines(commits, files, staged):
   return changed_lines
 
 
-def compute_diff(commits, files, staged):
+def compute_diff(commits, files, staged, diff_common_commit):
   """Return a subprocess object producing the diff from `commits`.
 
   The return value's `stdin` file object will produce a patch with the
@@ -328,6 +335,10 @@ def compute_diff(commits, files, staged):
 git_tool = 'diff-tree'
   elif staged:
 extra_args += ['--cached']
+
+  if len(commits) > 1 and diff_common_commit:
+commits = [f'{commits[0]}...{commits[1]}']
+
   cmd = ['git', git_tool, '-p', '-U0'] + extra_args + commits + ['--']
   cmd.extend(files)
   p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [clang-format] Add "three dot" diff option to git-clang-format (PR #74230)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 created 
https://github.com/llvm/llvm-project/pull/74230

This patch adds in the ability to do a "three dot" git-clang-format between two 
commits. This looks at the diff between the second commit and the common merge 
base rather than comparing at the point of the specified commits. This is 
needed to improve the reliability of the LLVM code formatting CI action which 
currently breaks in some cases where files have been modified in the upstream 
tree and when the person created their branch, leaving phantom formatting diffs 
that weren't touched by the PR author.

Part of a fix for #73873

>From 60032c548da7fa5bdf7033cf3bc8f8589dc1d13a Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 2 Dec 2023 23:46:58 -0800
Subject: [PATCH] [clang-format] Add "three dot" diff option to
 git-clang-format

This patch adds in the ability to do a "three dot" git-clang-format
between two commits. This looks at the diff between the second commit
and the common merge base rather than comparing at the point of the
specified commits. This is needed to improve the reliability of the LLVM
code formatting CI action which currently breaks in some cases where
files have been modified in the upstream tree and when the person
created their branch, leaving phantom formatting diffs that weren't
touched by the PR author.

Part of a fix for #73873
---
 clang/tools/clang-format/git-clang-format | 19 +++
 1 file changed, 15 insertions(+), 4 deletions(-)

diff --git a/clang/tools/clang-format/git-clang-format 
b/clang/tools/clang-format/git-clang-format
index 0f33b5339ec14..1f5d9a9bbf676 100755
--- a/clang/tools/clang-format/git-clang-format
+++ b/clang/tools/clang-format/git-clang-format
@@ -132,6 +132,10 @@ def main():
  help='passed to clang-format'),
   p.add_argument('-v', '--verbose', action='count', default=0,
  help='print extra information')
+  p.add_argument('--diff_from_common_commit', action='store_true',
+ help=('diff from the last common commit for commits in '
+  'separate branches rather than the exact point of the '
+  'commits'))
   # We gather all the remaining positional arguments into 'args' since we need
   # to use some heuristics to determine whether or not  was present.
   # However, to print pretty messages, we make use of metavar and help.
@@ -153,7 +157,10 @@ def main():
   else:
 if len(commits) > 2:
   die('at most two commits allowed; %d given' % len(commits))
-  changed_lines = compute_diff_and_extract_lines(commits, files, opts.staged)
+  changed_lines = compute_diff_and_extract_lines(commits,
+ files,
+ opts.staged,
+ opts.diff_from_common_commit)
   if opts.verbose >= 1:
 ignored_files = set(changed_lines)
   filter_by_extension(changed_lines, opts.extensions.lower().split(','))
@@ -301,9 +308,9 @@ def get_object_type(value):
   return convert_string(stdout.strip())
 
 
-def compute_diff_and_extract_lines(commits, files, staged):
+def compute_diff_and_extract_lines(commits, files, staged, diff_common_commit):
   """Calls compute_diff() followed by extract_lines()."""
-  diff_process = compute_diff(commits, files, staged)
+  diff_process = compute_diff(commits, files, staged, diff_common_commit)
   changed_lines = extract_lines(diff_process.stdout)
   diff_process.stdout.close()
   diff_process.wait()
@@ -313,7 +320,7 @@ def compute_diff_and_extract_lines(commits, files, staged):
   return changed_lines
 
 
-def compute_diff(commits, files, staged):
+def compute_diff(commits, files, staged, diff_common_commit):
   """Return a subprocess object producing the diff from `commits`.
 
   The return value's `stdin` file object will produce a patch with the
@@ -327,6 +334,10 @@ def compute_diff(commits, files, staged):
 git_tool = 'diff-tree'
   elif staged:
 extra_args += ['--cached']
+
+  if len(commits) > 1 and diff_common_commit:
+commits = [f'{commits[0]}...{commits[1]}']
+
   cmd = ['git', git_tool, '-p', '-U0'] + extra_args + commits + ['--']
   cmd.extend(files)
   p = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE)

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] Clang tools docs ci test (PR #69826)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/69826
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang-tools-extra] Clang tools docs ci test (PR #69826)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 created 
https://github.com/llvm/llvm-project/pull/69826

None

>From f8bd3e5f45e05f68522f41bb0b6ea733095eb7a4 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 21 Oct 2023 02:33:56 -0700
Subject: [PATCH 1/2] [Github] Add clang-tools-extra docs to CI

This patch adds the clang-tools-extra docs to the Github CI job that
builds docs, enabling the ability to easily ensure the docs build
properly without warnings in PRs and at the tip of tree.
---
 .github/workflows/docs.yml | 9 +
 1 file changed, 9 insertions(+)

diff --git a/.github/workflows/docs.yml b/.github/workflows/docs.yml
index 2900f73c77c5a66..6d91112cca5518f 100644
--- a/.github/workflows/docs.yml
+++ b/.github/workflows/docs.yml
@@ -15,10 +15,12 @@ on:
 paths:
   - 'llvm/docs/**'
   - 'clang/docs/**'
+  - 'clang-tools-extra/docs/**'
   pull_request:
 paths:
   - 'llvm/docs/**'
   - 'clang/docs/**'
+  - 'clang-tools-extra/docs/**'
 
 jobs:
   check-docs-build:
@@ -47,6 +49,8 @@ jobs:
   - 'llvm/docs/**'
 clang:
   - 'clang/docs/**'
+clang-tools-extra:
+  - 'clang-tools-extra/docs/**'
   - name: Setup Python env
 uses: actions/setup-python@v4
 with:
@@ -69,4 +73,9 @@ jobs:
 run: |
   cmake -B clang-build -GNinja -DCMAKE_BUILD_TYPE=Release 
-DLLVM_ENABLE_PROJECTS="clang" -DLLVM_ENABLE_SPHINX=ON ./llvm
   TZ=UTC ninja -C clang-build docs-clang-html docs-clang-man
+  - name: Build clang-tools-extra docs
+if: 
steps.docs-changed-subprojects.outputs.clang-tools-extra_any_changed == 'true'
+run: |
+  cmake -B clang-tools-extra-build -GNinja -DCMAKE_BUILD_TYPE=Release 
-DLLVM_ENABLE_PROJECTS="clang;clang-tools-extra" -DLLVM_ENABLE_SPHINX=ON ./llvm
+  TZ=UTC ninja -C clang-tools-extra-build docs-clang-tools-html 
docs-clang-tools-man
 

>From 20afc281b2fdd642fc75fc3a275cb882e2ffd3b6 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Sat, 21 Oct 2023 02:35:39 -0700
Subject: [PATCH 2/2] testing

---
 clang-tools-extra/docs/clang-rename.rst | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/clang-tools-extra/docs/clang-rename.rst 
b/clang-tools-extra/docs/clang-rename.rst
index e13d8c3ad25f98e..513056d06f5744f 100644
--- a/clang-tools-extra/docs/clang-rename.rst
+++ b/clang-tools-extra/docs/clang-rename.rst
@@ -2,6 +2,8 @@
 Clang-Rename
 
 
+This is just a test
+
 .. contents::
 
 See also:

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Sanitizer][Docs] Improve docs on building Asan (PR #68636)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

Updated in 68281947f7df4c7325b862857fbf3ed06d711f74.

https://github.com/llvm/llvm-project/pull/68636
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 6828194 - [Sanitizer][Docs] Reformat CMake invocation in docs

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-10-09T22:45:34Z
New Revision: 68281947f7df4c7325b862857fbf3ed06d711f74

URL: 
https://github.com/llvm/llvm-project/commit/68281947f7df4c7325b862857fbf3ed06d711f74
DIFF: 
https://github.com/llvm/llvm-project/commit/68281947f7df4c7325b862857fbf3ed06d711f74.diff

LOG: [Sanitizer][Docs] Reformat CMake invocation in docs

This was based on post-commit reviewer feedback in #68636.

Added: 


Modified: 
clang/docs/AddressSanitizer.rst

Removed: 




diff  --git a/clang/docs/AddressSanitizer.rst b/clang/docs/AddressSanitizer.rst
index 94ff3404cba6024..e1997153f203702 100644
--- a/clang/docs/AddressSanitizer.rst
+++ b/clang/docs/AddressSanitizer.rst
@@ -32,10 +32,7 @@ for the use/testing of AddressSanitizer:
 
 .. code-block:: console
 
-   cmake -DCMAKE_BUILD_TYPE=Release \
- -DLLVM_ENABLE_PROJECTS="clang" \
- -DLLVM_ENABLE_RUNTIMES="compiler-rt" \
- ../llvm
+   $ cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS="clang" 
-DLLVM_ENABLE_RUNTIMES="compiler-rt" /llvm
 
 Usage
 =



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Sanitizer][Docs] Improve docs on building Asan (PR #68636)

[Aiden Grossman via cfe-commits]

boomanaiden154 wrote:

Sorry. I'll wait longer to merge next time. I'll push a fixup to address the 
comment. Thanks for the comment/review!

https://github.com/llvm/llvm-project/pull/68636
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Sanitizer][Docs] Improve docs on building Asan (PR #68636)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 closed 
https://github.com/llvm/llvm-project/pull/68636
___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] [Sanitizer][Docs] Improve docs on building Asan (PR #68636)

[Aiden Grossman via cfe-commits]

https://github.com/boomanaiden154 created 
https://github.com/llvm/llvm-project/pull/68636

Currently the documentation for building Asan doesn't specify that compiler-rt 
needs to be built as well. In addition, there's no minimal example for the LLVM 
CMake configuration. This patch addresses both of these issues.

The lack of specification about building compiler-rt has shown up on Discourse 
(e.g., https://discourse.llvm.org/t/enabling-address-sanitizer/73940/2).

>From 49d010d28ba10ba86c58e8cd71e9c5af35a7c9e7 Mon Sep 17 00:00:00 2001
From: Aiden Grossman 
Date: Mon, 9 Oct 2023 11:11:18 -0700
Subject: [PATCH] [Sanitizer][Docs] Improve docs on building Asan

Currently the documentation for building Asan doesn't specify that
compiler-rt needs to be built as well. In addition, there's no minimal
example for the LLVM CMake configuration. This patch addresses both of
these issues.
---
 clang/docs/AddressSanitizer.rst | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/clang/docs/AddressSanitizer.rst b/clang/docs/AddressSanitizer.rst
index 37f34cb0cc9936d..94ff3404cba6024 100644
--- a/clang/docs/AddressSanitizer.rst
+++ b/clang/docs/AddressSanitizer.rst
@@ -26,7 +26,16 @@ Typical slowdown introduced by AddressSanitizer is **2x**.
 How to build
 
 
-Build LLVM/Clang with `CMake `_.
+Build LLVM/Clang with `CMake ` and enable
+the ``compiler-rt`` runtime. An example CMake configuration that will allow
+for the use/testing of AddressSanitizer:
+
+.. code-block:: console
+
+   cmake -DCMAKE_BUILD_TYPE=Release \
+ -DLLVM_ENABLE_PROJECTS="clang" \
+ -DLLVM_ENABLE_RUNTIMES="compiler-rt" \
+ ../llvm
 
 Usage
 =

___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 8b6f09e - Revert "[clang][X86] Add __cpuidex function to cpuid.h"

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-08-19T17:18:10-07:00
New Revision: 8b6f09e257b521947b50761737290e5bf31c80f3

URL: 
https://github.com/llvm/llvm-project/commit/8b6f09e257b521947b50761737290e5bf31c80f3
DIFF: 
https://github.com/llvm/llvm-project/commit/8b6f09e257b521947b50761737290e5bf31c80f3.diff

LOG: Revert "[clang][X86] Add __cpuidex function to cpuid.h"

This reverts commit 58696d2f5bbae32dddcaec6891293e769465e77c.

Accidentally had this in my branch for my structural hash patch.

Added: 


Modified: 
clang/lib/Headers/cpuid.h
clang/test/Headers/cpuid.c

Removed: 
clang/test/Headers/__cpuidex_conflict.c



diff  --git a/clang/lib/Headers/cpuid.h b/clang/lib/Headers/cpuid.h
index 9b31db9f7d4f32..1ad6853a97c9d2 100644
--- a/clang/lib/Headers/cpuid.h
+++ b/clang/lib/Headers/cpuid.h
@@ -328,14 +328,4 @@ static __inline int __get_cpuid_count (unsigned int __leaf,
 return 1;
 }
 
-// In some configurations, __cpuidex is defined as a builtin (primarily
-// -fms-extensions) which will conflict with the __cpuidex definition below.
-#if !(__has_builtin(__cpuidex))
-static __inline void __cpuidex (int __cpu_info[4], int __leaf, int __subleaf)
-{
-  __cpuid_count(__leaf, __subleaf, __cpu_info[0], __cpu_info[1], __cpu_info[2],
-__cpu_info[3]);
-}
-#endif
-
 #endif /* __CPUID_H */

diff  --git a/clang/test/Headers/__cpuidex_conflict.c 
b/clang/test/Headers/__cpuidex_conflict.c
deleted file mode 100644
index 8687a6aa2f897a..00
--- a/clang/test/Headers/__cpuidex_conflict.c
+++ /dev/null
@@ -1,22 +0,0 @@
-// Make sure that __cpuidex in cpuid.h doesn't conflict with the MS
-// extensions built in by ensuring compilation succeeds:
-// RUN: %clang_cc1 %s -ffreestanding -fms-extensions -fms-compatibility \
-// RUN:  -fms-compatibility-version=19.00 -triple x86_64-pc-windows-msvc 
-emit-llvm -o -
-// %clang_cc1 %s -ffreestanding -triple x86_64-w64-windows-gnu -fms-extensions 
-emit-llvm -o -
-// RUN: %clang_cc1 %s -ffreestanding -fopenmp -fopenmp-is-target-device 
-aux-triple x86_64-unknown-linux-gnu
-
-typedef __SIZE_TYPE__ size_t;
-
-// We declare __cpuidex here as where the buitlin should be exposed (MSVC), the
-// declaration is in , but  is not available from all the
-// targets that are being tested here.
-void __cpuidex (int[4], int, int);
-
-#include 
-
-int cpuid_info[4];
-
-void test_cpuidex(unsigned level, unsigned count) {
-  __cpuidex(cpuid_info, level, count);
-}
-

diff  --git a/clang/test/Headers/cpuid.c b/clang/test/Headers/cpuid.c
index 6ed12eca7a61d4..7e485495c10665 100644
--- a/clang/test/Headers/cpuid.c
+++ b/clang/test/Headers/cpuid.c
@@ -6,19 +6,14 @@
 
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A 
cpuid\0A xchgq %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
-// CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
 
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
-// CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
 
 unsigned eax0, ebx0, ecx0, edx0;
 unsigned eax1, ebx1, ecx1, edx1;
 
-int cpuid_info[4];
-
 void test_cpuid(unsigned level, unsigned count) {
   __cpuid(level, eax1, ebx1, ecx1, edx1);
   __cpuid_count(level, count, eax0, ebx0, ecx0, edx0);
-  __cpuidex(cpuid_info, level, count);
 }



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 58696d2 - [clang][X86] Add __cpuidex function to cpuid.h

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-08-19T17:12:23-07:00
New Revision: 58696d2f5bbae32dddcaec6891293e769465e77c

URL: 
https://github.com/llvm/llvm-project/commit/58696d2f5bbae32dddcaec6891293e769465e77c
DIFF: 
https://github.com/llvm/llvm-project/commit/58696d2f5bbae32dddcaec6891293e769465e77c.diff

LOG: [clang][X86] Add __cpuidex function to cpuid.h

MSVC has a __cpuidex function implemented to call the underlying cpuid
instruction which accepts a leaf, subleaf, and data array that the output
data is written into. This patch adds this functionality into clang
under the cpuid.h header. This also makes clang match GCC's behavior.
GCC has had __cpuidex in its cpuid.h since 2020.

Added: 
clang/test/Headers/__cpuidex_conflict.c

Modified: 
clang/lib/Headers/cpuid.h
clang/test/Headers/cpuid.c

Removed: 




diff  --git a/clang/lib/Headers/cpuid.h b/clang/lib/Headers/cpuid.h
index 1ad6853a97c9d2..9b31db9f7d4f32 100644
--- a/clang/lib/Headers/cpuid.h
+++ b/clang/lib/Headers/cpuid.h
@@ -328,4 +328,14 @@ static __inline int __get_cpuid_count (unsigned int __leaf,
 return 1;
 }
 
+// In some configurations, __cpuidex is defined as a builtin (primarily
+// -fms-extensions) which will conflict with the __cpuidex definition below.
+#if !(__has_builtin(__cpuidex))
+static __inline void __cpuidex (int __cpu_info[4], int __leaf, int __subleaf)
+{
+  __cpuid_count(__leaf, __subleaf, __cpu_info[0], __cpu_info[1], __cpu_info[2],
+__cpu_info[3]);
+}
+#endif
+
 #endif /* __CPUID_H */

diff  --git a/clang/test/Headers/__cpuidex_conflict.c 
b/clang/test/Headers/__cpuidex_conflict.c
new file mode 100644
index 00..8687a6aa2f897a
--- /dev/null
+++ b/clang/test/Headers/__cpuidex_conflict.c
@@ -0,0 +1,22 @@
+// Make sure that __cpuidex in cpuid.h doesn't conflict with the MS
+// extensions built in by ensuring compilation succeeds:
+// RUN: %clang_cc1 %s -ffreestanding -fms-extensions -fms-compatibility \
+// RUN:  -fms-compatibility-version=19.00 -triple x86_64-pc-windows-msvc 
-emit-llvm -o -
+// %clang_cc1 %s -ffreestanding -triple x86_64-w64-windows-gnu -fms-extensions 
-emit-llvm -o -
+// RUN: %clang_cc1 %s -ffreestanding -fopenmp -fopenmp-is-target-device 
-aux-triple x86_64-unknown-linux-gnu
+
+typedef __SIZE_TYPE__ size_t;
+
+// We declare __cpuidex here as where the buitlin should be exposed (MSVC), the
+// declaration is in , but  is not available from all the
+// targets that are being tested here.
+void __cpuidex (int[4], int, int);
+
+#include 
+
+int cpuid_info[4];
+
+void test_cpuidex(unsigned level, unsigned count) {
+  __cpuidex(cpuid_info, level, count);
+}
+

diff  --git a/clang/test/Headers/cpuid.c b/clang/test/Headers/cpuid.c
index 7e485495c10665..6ed12eca7a61d4 100644
--- a/clang/test/Headers/cpuid.c
+++ b/clang/test/Headers/cpuid.c
@@ -6,14 +6,19 @@
 
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A 
cpuid\0A xchgq %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
+// CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
 
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
+// CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
 
 unsigned eax0, ebx0, ecx0, edx0;
 unsigned eax1, ebx1, ecx1, edx1;
 
+int cpuid_info[4];
+
 void test_cpuid(unsigned level, unsigned count) {
   __cpuid(level, eax1, ebx1, ecx1, edx1);
   __cpuid_count(level, count, eax0, ebx0, ecx0, edx0);
+  __cpuidex(cpuid_info, level, count);
 }



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] f3baf63 - Revert "[clang][X86] Add __cpuidex function to cpuid.h"

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-08-04T13:28:53-07:00
New Revision: f3baf63d9a1ba91974f4df6abb8f2abd9a0df5b5

URL: 
https://github.com/llvm/llvm-project/commit/f3baf63d9a1ba91974f4df6abb8f2abd9a0df5b5
DIFF: 
https://github.com/llvm/llvm-project/commit/f3baf63d9a1ba91974f4df6abb8f2abd9a0df5b5.diff

LOG: Revert "[clang][X86] Add __cpuidex function to cpuid.h"

This reverts commit 2df77ac20a1ed996706b164b0c4ed5ad140f635f.

This has been causing some issues with some windows builds as
_MSC_EXTENSIONS isn't defined when only -fms-extensions is set, but the
builtin that conflicts with __cpuidex is. This was also causing problems
as it exposed some latent issues with how auxiliary triples are handled
in clang.

Differential Revision: https://reviews.llvm.org/D157115

Added: 


Modified: 
clang/lib/Headers/cpuid.h
clang/test/Headers/cpuid.c

Removed: 
clang/test/Headers/__cpuidex_conflict.c



diff  --git a/clang/lib/Headers/cpuid.h b/clang/lib/Headers/cpuid.h
index 454f74e92f85a0..1ad6853a97c9d2 100644
--- a/clang/lib/Headers/cpuid.h
+++ b/clang/lib/Headers/cpuid.h
@@ -328,14 +328,4 @@ static __inline int __get_cpuid_count (unsigned int __leaf,
 return 1;
 }
 
-// If MS extensions are enabled, __cpuidex is defined as a builtin which will
-// conflict with the __cpuidex definition below.
-#ifndef _MSC_EXTENSIONS
-static __inline void __cpuidex (int __cpu_info[4], int __leaf, int __subleaf)
-{
-  __cpuid_count(__leaf, __subleaf, __cpu_info[0], __cpu_info[1], __cpu_info[2],
-__cpu_info[3]);
-}
-#endif
-
 #endif /* __CPUID_H */

diff  --git a/clang/test/Headers/__cpuidex_conflict.c 
b/clang/test/Headers/__cpuidex_conflict.c
deleted file mode 100644
index 0eff1ff425f83e..00
--- a/clang/test/Headers/__cpuidex_conflict.c
+++ /dev/null
@@ -1,15 +0,0 @@
-// Make sure that __cpuidex in cpuid.h doesn't conflict with the MS
-// compatibility built in by ensuring compilation succeeds:
-// RUN: %clang_cc1 %s -ffreestanding -fms-extensions -fms-compatibility \
-// RUN:  -fms-compatibility-version=19.00 -triple x86_64-pc-windows-msvc 
-emit-llvm -o -
-
-typedef __SIZE_TYPE__ size_t;
-
-#include 
-#include 
-
-int cpuid_info[4];
-
-void test_cpuidex(unsigned level, unsigned count) {
-  __cpuidex(cpuid_info, level, count);
-}

diff  --git a/clang/test/Headers/cpuid.c b/clang/test/Headers/cpuid.c
index 6ed12eca7a61d4..7e485495c10665 100644
--- a/clang/test/Headers/cpuid.c
+++ b/clang/test/Headers/cpuid.c
@@ -6,19 +6,14 @@
 
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A 
cpuid\0A xchgq %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
-// CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
 
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
-// CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
 
 unsigned eax0, ebx0, ecx0, edx0;
 unsigned eax1, ebx1, ecx1, edx1;
 
-int cpuid_info[4];
-
 void test_cpuid(unsigned level, unsigned count) {
   __cpuid(level, eax1, ebx1, ecx1, edx1);
   __cpuid_count(level, count, eax0, ebx0, ecx0, edx0);
-  __cpuidex(cpuid_info, level, count);
 }



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 2ff0aa2 - [CMake][Fuchsia] Add LLVM_ENABLE_HTTPLIB to Stage 2 build

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-06-03T22:30:19Z
New Revision: 2ff0aa207fd55604604bb9eec33dada1a80842db

URL: 
https://github.com/llvm/llvm-project/commit/2ff0aa207fd55604604bb9eec33dada1a80842db
DIFF: 
https://github.com/llvm/llvm-project/commit/2ff0aa207fd55604604bb9eec33dada1a80842db.diff

LOG: [CMake][Fuchsia] Add LLVM_ENABLE_HTTPLIB to Stage 2 build

This patch sets the LLVM_ENABLE_HTTPLIB flag to ON in the stage 2 build
similar to how many of the other dependency flags are already specified.
This is necessary to configure the stage 2 build by itself, otherwise
the CMake configuration crashes.

This is currently causing the MLGO demo to fail since we're only using
stage 2 to avoid having to build stage 1 to save some compile time.

Reviewed By: phosek

Differential Revision: https://reviews.llvm.org/D152057

Added: 


Modified: 
clang/cmake/caches/Fuchsia-stage2.cmake

Removed: 




diff  --git a/clang/cmake/caches/Fuchsia-stage2.cmake 
b/clang/cmake/caches/Fuchsia-stage2.cmake
index 78c123f191305..b4bc8cb4eb64e 100644
--- a/clang/cmake/caches/Fuchsia-stage2.cmake
+++ b/clang/cmake/caches/Fuchsia-stage2.cmake
@@ -11,6 +11,7 @@ set(LLVM_ENABLE_RUNTIMES 
"compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "
 
 set(LLVM_ENABLE_BACKTRACES OFF CACHE BOOL "")
 set(LLVM_ENABLE_DIA_SDK OFF CACHE BOOL "")
+set(LLVM_ENABLE_HTTPLIB ON CACHE BOOL "")
 set(LLVM_ENABLE_LIBCXX ON CACHE BOOL "")
 set(LLVM_ENABLE_LIBEDIT OFF CACHE BOOL "")
 set(LLVM_ENABLE_LLD ON CACHE BOOL "")



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 2df77ac - [clang][X86] Add __cpuidex function to cpuid.h

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-05-24T06:27:46Z
New Revision: 2df77ac20a1ed996706b164b0c4ed5ad140f635f

URL: 
https://github.com/llvm/llvm-project/commit/2df77ac20a1ed996706b164b0c4ed5ad140f635f
DIFF: 
https://github.com/llvm/llvm-project/commit/2df77ac20a1ed996706b164b0c4ed5ad140f635f.diff

LOG: [clang][X86] Add __cpuidex function to cpuid.h

MSVC has a `__cpuidex` function implemented to call the underlying cpuid
instruction which accepts a leaf, subleaf, and data array that the output
data is written into. This patch adds this functionality into clang
under the cpuid.h header. This also makes clang match GCC's behavior.
GCC has had `__cpuidex` in its cpuid.h since 2020.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D150646

Added: 
clang/test/Headers/__cpuidex_conflict.c

Modified: 
clang/lib/Headers/cpuid.h
clang/test/Headers/cpuid.c

Removed: 




diff  --git a/clang/lib/Headers/cpuid.h b/clang/lib/Headers/cpuid.h
index 1ad6853a97c9d..454f74e92f85a 100644
--- a/clang/lib/Headers/cpuid.h
+++ b/clang/lib/Headers/cpuid.h
@@ -328,4 +328,14 @@ static __inline int __get_cpuid_count (unsigned int __leaf,
 return 1;
 }
 
+// If MS extensions are enabled, __cpuidex is defined as a builtin which will
+// conflict with the __cpuidex definition below.
+#ifndef _MSC_EXTENSIONS
+static __inline void __cpuidex (int __cpu_info[4], int __leaf, int __subleaf)
+{
+  __cpuid_count(__leaf, __subleaf, __cpu_info[0], __cpu_info[1], __cpu_info[2],
+__cpu_info[3]);
+}
+#endif
+
 #endif /* __CPUID_H */

diff  --git a/clang/test/Headers/__cpuidex_conflict.c 
b/clang/test/Headers/__cpuidex_conflict.c
new file mode 100644
index 0..0eff1ff425f83
--- /dev/null
+++ b/clang/test/Headers/__cpuidex_conflict.c
@@ -0,0 +1,15 @@
+// Make sure that __cpuidex in cpuid.h doesn't conflict with the MS
+// compatibility built in by ensuring compilation succeeds:
+// RUN: %clang_cc1 %s -ffreestanding -fms-extensions -fms-compatibility \
+// RUN:  -fms-compatibility-version=19.00 -triple x86_64-pc-windows-msvc 
-emit-llvm -o -
+
+typedef __SIZE_TYPE__ size_t;
+
+#include 
+#include 
+
+int cpuid_info[4];
+
+void test_cpuidex(unsigned level, unsigned count) {
+  __cpuidex(cpuid_info, level, count);
+}

diff  --git a/clang/test/Headers/cpuid.c b/clang/test/Headers/cpuid.c
index 7e485495c1066..6ed12eca7a61d 100644
--- a/clang/test/Headers/cpuid.c
+++ b/clang/test/Headers/cpuid.c
@@ -6,14 +6,19 @@
 
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A 
cpuid\0A xchgq %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
+// CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
 
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
+// CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
 
 unsigned eax0, ebx0, ecx0, edx0;
 unsigned eax1, ebx1, ecx1, edx1;
 
+int cpuid_info[4];
+
 void test_cpuid(unsigned level, unsigned count) {
   __cpuid(level, eax1, ebx1, ecx1, edx1);
   __cpuid_count(level, count, eax0, ebx0, ecx0, edx0);
+  __cpuidex(cpuid_info, level, count);
 }



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 4acee5b - Revert "[clang][X86] Add __cpuidex function to cpuid.h"

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-05-18T05:37:48Z
New Revision: 4acee5b5cfa462af66225acda7d3d9efe1036aa9

URL: 
https://github.com/llvm/llvm-project/commit/4acee5b5cfa462af66225acda7d3d9efe1036aa9
DIFF: 
https://github.com/llvm/llvm-project/commit/4acee5b5cfa462af66225acda7d3d9efe1036aa9.diff

LOG: Revert "[clang][X86] Add __cpuidex function to cpuid.h"

This reverts commit 286cefcf35d0f55c57184c4219b95e82c96f1420.

Patch caused build failures for downstream projects on Windows due to
the fact that __cpuidex was added as a built in on Windows in D121653.
Reverting for now so that others aren't blocked and I can figure out a
proper solution.

Added: 


Modified: 
clang/lib/Headers/cpuid.h
clang/test/Headers/cpuid.c

Removed: 




diff  --git a/clang/lib/Headers/cpuid.h b/clang/lib/Headers/cpuid.h
index 22260c48c0c42..1ad6853a97c9d 100644
--- a/clang/lib/Headers/cpuid.h
+++ b/clang/lib/Headers/cpuid.h
@@ -328,10 +328,4 @@ static __inline int __get_cpuid_count (unsigned int __leaf,
 return 1;
 }
 
-static __inline void __cpuidex (int __cpu_info[4], int __leaf, int __subleaf)
-{
-  __cpuid_count(__leaf, __subleaf, __cpu_info[0], __cpu_info[1], __cpu_info[2],
-__cpu_info[3]);
-}
-
 #endif /* __CPUID_H */

diff  --git a/clang/test/Headers/cpuid.c b/clang/test/Headers/cpuid.c
index 6ed12eca7a61d..7e485495c1066 100644
--- a/clang/test/Headers/cpuid.c
+++ b/clang/test/Headers/cpuid.c
@@ -6,19 +6,14 @@
 
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A 
cpuid\0A xchgq %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
-// CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
 
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
-// CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
 
 unsigned eax0, ebx0, ecx0, edx0;
 unsigned eax1, ebx1, ecx1, edx1;
 
-int cpuid_info[4];
-
 void test_cpuid(unsigned level, unsigned count) {
   __cpuid(level, eax1, ebx1, ecx1, edx1);
   __cpuid_count(level, count, eax0, ebx0, ecx0, edx0);
-  __cpuidex(cpuid_info, level, count);
 }



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] 286cefc - [clang][X86] Add __cpuidex function to cpuid.h

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-05-17T19:38:28Z
New Revision: 286cefcf35d0f55c57184c4219b95e82c96f1420

URL: 
https://github.com/llvm/llvm-project/commit/286cefcf35d0f55c57184c4219b95e82c96f1420
DIFF: 
https://github.com/llvm/llvm-project/commit/286cefcf35d0f55c57184c4219b95e82c96f1420.diff

LOG: [clang][X86] Add __cpuidex function to cpuid.h

MSVC has a `__cpuidex` function implemented to call the underlying cpuid
instruction which accepts a leaf, subleaf, and data array that the output
data is written into. This patch adds this functionality into clang
under the cpuid.h header. This also makes clang match GCC's behavior.
GCC has had `__cpuidex` in its cpuid.h since 2020.

Reviewed By: craig.topper

Differential Revision: https://reviews.llvm.org/D150646

Added: 


Modified: 
clang/lib/Headers/cpuid.h
clang/test/Headers/cpuid.c

Removed: 




diff  --git a/clang/lib/Headers/cpuid.h b/clang/lib/Headers/cpuid.h
index 1ad6853a97c9d..22260c48c0c42 100644
--- a/clang/lib/Headers/cpuid.h
+++ b/clang/lib/Headers/cpuid.h
@@ -328,4 +328,10 @@ static __inline int __get_cpuid_count (unsigned int __leaf,
 return 1;
 }
 
+static __inline void __cpuidex (int __cpu_info[4], int __leaf, int __subleaf)
+{
+  __cpuid_count(__leaf, __subleaf, __cpu_info[0], __cpu_info[1], __cpu_info[2],
+__cpu_info[3]);
+}
+
 #endif /* __CPUID_H */

diff  --git a/clang/test/Headers/cpuid.c b/clang/test/Headers/cpuid.c
index 7e485495c1066..6ed12eca7a61d 100644
--- a/clang/test/Headers/cpuid.c
+++ b/clang/test/Headers/cpuid.c
@@ -6,14 +6,19 @@
 
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A 
cpuid\0A xchgq %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
+// CHECK-64: {{.*}} call { i32, i32, i32, i32 } asm "  xchgq  %rbx,${1:q}\0A  
cpuid\0A  xchgq  %rbx,${1:q}", 
"={ax},=r,={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, i32 
%{{[a-z0-9]+}})
 
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}})
 // CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
+// CHECK-32: {{.*}} call { i32, i32, i32, i32 } asm "cpuid", 
"={ax},={bx},={cx},={dx},0,2,~{dirflag},~{fpsr},~{flags}"(i32 %{{[a-z0-9]+}}, 
i32 %{{[a-z0-9]+}})
 
 unsigned eax0, ebx0, ecx0, edx0;
 unsigned eax1, ebx1, ecx1, edx1;
 
+int cpuid_info[4];
+
 void test_cpuid(unsigned level, unsigned count) {
   __cpuid(level, eax1, ebx1, ecx1, edx1);
   __cpuid_count(level, count, eax0, ebx0, ecx0, edx0);
+  __cpuidex(cpuid_info, level, count);
 }



___
cfe-commits mailing list
cfe-commits@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits

[clang] db63fb5 - [Clang][Docs] Fix man page build

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2023-05-13T08:51:10Z
New Revision: db63fb5d45e0f58a1c657b23a24e85f28e3dbf73

URL: 
https://github.com/llvm/llvm-project/commit/db63fb5d45e0f58a1c657b23a24e85f28e3dbf73
DIFF: 
https://github.com/llvm/llvm-project/commit/db63fb5d45e0f58a1c657b23a24e85f28e3dbf73.diff

LOG: [Clang][Docs] Fix man page build

This patch fixes the man page build. It currently doesn't work as
SOURCE_DIR isn't set correctly (just undefined) within the
add_sphinx_target function. This patch also moves around the creation of
targets for autogenerated rst files so that both the man page and html
build can depend upon them as before only the html build depended on
them.

Fixes #62540

Reviewed By: tstellar

Differential Revision: https://reviews.llvm.org/D149809

Added: 


Modified: 
clang/docs/CMakeLists.txt

Removed: 




diff  --git a/clang/docs/CMakeLists.txt b/clang/docs/CMakeLists.txt
index 532907385df4..4163dd2d90ad 100644
--- a/clang/docs/CMakeLists.txt
+++ b/clang/docs/CMakeLists.txt
@@ -90,50 +90,60 @@ if (LLVM_ENABLE_DOXYGEN)
 endif()
 endif()
 
-function (gen_rst_file_from_td output_file td_option source docs_target)
+function (gen_rst_file_from_td output_file td_option source docs_targets)
   if (NOT EXISTS "${CMAKE_CURRENT_SOURCE_DIR}/${source}")
 message(FATAL_ERROR "Cannot find source file: ${source} in 
${CMAKE_CURRENT_SOURCE_DIR}")
   endif()
   get_filename_component(TABLEGEN_INCLUDE_DIR 
"${CMAKE_CURRENT_SOURCE_DIR}/${source}" DIRECTORY)
   list(APPEND LLVM_TABLEGEN_FLAGS "-I${TABLEGEN_INCLUDE_DIR}")
   clang_tablegen(${output_file} ${td_option} SOURCE ${source} TARGET 
"gen-${output_file}")
-  add_dependencies(${docs_target} "gen-${output_file}")
+  foreach(target ${docs_targets})
+add_dependencies(${target} gen-${output_file})
+  endforeach()
 endfunction()
 
 if (LLVM_ENABLE_SPHINX)
   include(AddSphinxTarget)
-  if (SPHINX_FOUND)
+  if (SPHINX_FOUND AND (${SPHINX_OUTPUT_HTML} OR ${SPHINX_OUTPUT_MAN}))
+# Copy rst files to build directory before generating the html
+# documentation.  Some of the rst files are generated, so they
+# only exist in the build directory.  Sphinx needs all files in
+# the same directory in order to generate the html, so we need to
+# copy all the non-gnerated rst files from the source to the build
+# directory before we run sphinx.
+add_custom_target(copy-clang-rst-docs
+  COMMAND "${CMAKE_COMMAND}" -E copy_directory
+  "${CMAKE_CURRENT_SOURCE_DIR}" "${CMAKE_CURRENT_BINARY_DIR}"
+
+  COMMAND "${CMAKE_COMMAND}" -E copy_if_
diff erent
+  "${CMAKE_CURRENT_SOURCE_DIR}/../CodeOwners.rst"
+  "${CMAKE_CURRENT_BINARY_DIR}"
+)
+
+set(docs_targets "")
+
 if (${SPHINX_OUTPUT_HTML})
   add_sphinx_target(html clang SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}")
 
-  # Copy rst files to build directory before generating the html
-  # documentation.  Some of the rst files are generated, so they
-  # only exist in the build directory.  Sphinx needs all files in
-  # the same directory in order to generate the html, so we need to
-  # copy all the non-gnerated rst files from the source to the build
-  # directory before we run sphinx.
-  add_custom_target(copy-clang-rst-docs
-COMMAND "${CMAKE_COMMAND}" -E copy_directory
-"${CMAKE_CURRENT_SOURCE_DIR}" "${CMAKE_CURRENT_BINARY_DIR}"
-
-COMMAND "${CMAKE_COMMAND}" -E copy_if_
diff erent
-"${CMAKE_CURRENT_SOURCE_DIR}/../CodeOwners.rst"
-"${CMAKE_CURRENT_BINARY_DIR}"
-  )
-  add_dependencies(docs-clang-html copy-clang-rst-docs)
-
   add_custom_command(TARGET docs-clang-html POST_BUILD
 COMMAND "${CMAKE_COMMAND}" -E copy
 "${CMAKE_CURRENT_SOURCE_DIR}/LibASTMatchersReference.html"
 "${CMAKE_CURRENT_BINARY_DIR}/html/LibASTMatchersReference.html")
 
-  # Generated files
-  gen_rst_file_from_td(AttributeReference.rst -gen-attr-docs 
../include/clang/Basic/Attr.td docs-clang-html)
-  gen_rst_file_from_td(DiagnosticsReference.rst -gen-diag-docs 
../include/clang/Basic/Diagnostic.td docs-clang-html)
-  gen_rst_file_from_td(ClangCommandLineReference.rst -gen-opt-docs 
../include/clang/Driver/ClangOptionDocs.td docs-clang-html)
+  list(APPEND docs_targets "docs-clang-html")
 endif()
 if (${SPHINX_OUTPUT_MAN})
-  add_sphinx_target(man clang)
+  add_sphinx_target(man clang SOURCE_DIR "${CMAKE_CURRENT_BINARY_DIR}")
+  list(APPEND docs_targets "docs-clang-man")
 endif()
+
+# Generated files
+gen_rst_file_from_td(AttributeReference.rst -gen-attr-docs 
../include/clang/Basic/Attr.td "${docs_targets}")
+gen_rst_file_from_td(DiagnosticsReference.rst -gen-diag-docs 
../include/clang/Basic/Diagnostic.td "${docs_targets}")
+gen_rst_file_from_td(ClangCommandLineReference.rst -gen-opt-docs 
../include/clang/Driver/ClangOptionDocs.td 

[clang] c0bc461 - [Clang] Give error message for invalid profile path when compiling IR

[Aiden Grossman via cfe-commits]

Author: Aiden Grossman
Date: 2022-09-16T19:45:57Z
New Revision: c0bc461999fdac918dd26867947c24eb6235c8d0

URL: 
https://github.com/llvm/llvm-project/commit/c0bc461999fdac918dd26867947c24eb6235c8d0
DIFF: 
https://github.com/llvm/llvm-project/commit/c0bc461999fdac918dd26867947c24eb6235c8d0.diff

LOG: [Clang] Give error message for invalid profile path when compiling IR

Before this patch, when compiling an IR file (eg the .llvmbc section
from an object file compiled with -Xclang -fembed-bitcode=all) and
profile data was passed in using the -fprofile-instrument-use-path
flag, there would be no error printed (as the previous implementation
relied on the error getting caught again in the constructor of
CodeGenModule which isn't called when -x ir is set). This patch
moves the error checking directly to where the error is caught
originally rather than failing silently in setPGOUseInstrumentor and
waiting to catch it in CodeGenModule to print diagnostic information to
the user.

Regression test added.

Reviewed By: xur, mtrofin

Differential Revision: https://reviews.llvm.org/D132991

Added: 
clang/test/Profile/profile-does-not-exist-ir.c

Modified: 
clang/lib/CodeGen/CodeGenModule.cpp
clang/lib/Frontend/CompilerInvocation.cpp
clang/test/Profile/profile-does-not-exist.c

Removed: 




diff  --git a/clang/lib/CodeGen/CodeGenModule.cpp 
b/clang/lib/CodeGen/CodeGenModule.cpp
index bbd42956a4b0..06ad67a492a1 100644
--- a/clang/lib/CodeGen/CodeGenModule.cpp
+++ b/clang/lib/CodeGen/CodeGenModule.cpp
@@ -182,15 +182,11 @@ CodeGenModule::CodeGenModule(ASTContext ,
   if (CodeGenOpts.hasProfileClangUse()) {
 auto ReaderOrErr = llvm::IndexedInstrProfReader::create(
 CodeGenOpts.ProfileInstrumentUsePath, 
CodeGenOpts.ProfileRemappingFile);
-if (auto E = ReaderOrErr.takeError()) {
-  unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
-  "Could not read profile %0: %1");
-  llvm::handleAllErrors(std::move(E), [&](const llvm::ErrorInfoBase ) {
-getDiags().Report(DiagID) << CodeGenOpts.ProfileInstrumentUsePath
-  << EI.message();
-  });
-} else
-  PGOReader = std::move(ReaderOrErr.get());
+// We're checking for profile read errors in CompilerInvocation, so if
+// there was an error it should've already been caught. If it hasn't been
+// somehow, trip an assertion.
+assert(ReaderOrErr);
+PGOReader = std::move(ReaderOrErr.get());
   }
 
   // If coverage mapping generation is enabled, create the

diff  --git a/clang/lib/Frontend/CompilerInvocation.cpp 
b/clang/lib/Frontend/CompilerInvocation.cpp
index f42bf3058d1a..9f9241054b1e 100644
--- a/clang/lib/Frontend/CompilerInvocation.cpp
+++ b/clang/lib/Frontend/CompilerInvocation.cpp
@@ -1293,12 +1293,15 @@ static std::string 
serializeXRayInstrumentationBundle(const XRayInstrSet ) {
 
 // Set the profile kind using fprofile-instrument-use-path.
 static void setPGOUseInstrumentor(CodeGenOptions ,
-  const Twine ) {
+  const Twine ,
+  DiagnosticsEngine ) {
   auto ReaderOrErr = llvm::IndexedInstrProfReader::create(ProfileName);
-  // In error, return silently and let Clang PGOUse report the error message.
   if (auto E = ReaderOrErr.takeError()) {
-llvm::consumeError(std::move(E));
-Opts.setProfileUse(CodeGenOptions::ProfileClangInstr);
+unsigned DiagID = Diags.getCustomDiagID(DiagnosticsEngine::Error,
+"Error in reading profile %0: %1");
+llvm::handleAllErrors(std::move(E), [&](const llvm::ErrorInfoBase ) {
+  Diags.Report(DiagID) << ProfileName.str() << EI.message();
+});
 return;
   }
   std::unique_ptr PGOReader =
@@ -1712,7 +1715,7 @@ bool CompilerInvocation::ParseCodeGenArgs(CodeGenOptions 
, ArgList ,
   }
 
   if (!Opts.ProfileInstrumentUsePath.empty())
-setPGOUseInstrumentor(Opts, Opts.ProfileInstrumentUsePath);
+setPGOUseInstrumentor(Opts, Opts.ProfileInstrumentUsePath, Diags);
 
   if (const Arg *A = Args.getLastArg(OPT_ftime_report, OPT_ftime_report_EQ)) {
 Opts.TimePasses = true;

diff  --git a/clang/test/Profile/profile-does-not-exist-ir.c 
b/clang/test/Profile/profile-does-not-exist-ir.c
new file mode 100644
index ..842a3d44a387
--- /dev/null
+++ b/clang/test/Profile/profile-does-not-exist-ir.c
@@ -0,0 +1,4 @@
+// RUN: not %clang_cc1 -emit-llvm -x ir %s -o - 
-fprofile-instrument-use-path=%t.nonexistent.profdata 2>&1 | FileCheck %s
+
+// CHECK: error: Error in reading profile {{.*}}.nonexistent.profdata:
+// CHECK-NOT: Assertion failed

diff  --git a/clang/test/Profile/profile-does-not-exist.c 
b/clang/test/Profile/profile-does-not-exist.c
index 5725f76eb80c..88d55d8668ef 100644
--- a/clang/test/Profile/profile-does-not-exist.c