[clang] [perf-training] Fix dependencies when using -DCLANG_PGO_TRAINING_DATA_SOURCE_DIR (PR #108488)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/108488 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [perf-training] Fix dependencies when using -DCLANG_PGO_TRAINING_DATA_SOURCE_DIR (PR #108488)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/108488 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [perf-training] Fix dependencies when using -DCLANG_PGO_TRAINING_DATA_SOURCE_DIR (PR #108488)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/108488 The generate-profraw-external target needs to be built after clang, but this was not modeled correctly in the CMake dependencies. The new dependency ordering ensures that clang is built before generate-profraw-external. Old Dependencies: generate-profdata -> clang -> generate-profraw -> generate-profraw-external New Dependencies: generate-profdata -> clang -> generate-profraw -> generate-profraw-external >From 227e2314df69949e2972ca26fac2080bd6304afc Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Fri, 13 Sep 2024 04:06:36 + Subject: [PATCH] [perf-training] Fix dependencies when using -DCLANG_PGO_TRAINING_DATA_SOURCE_DIR The generate-profraw-external target needs to be built after clang, but this was not modeled correctly in the CMake depencies. The new dependency ordering ensures that clang is built before generate-profraw-external. Old Dependencies: generate-profdata -> clang -> generate-profraw -> generate-profraw-external New Dependencies: generate-profdata -> clang -> generate-profraw -> generate-profraw-external --- clang/utils/perf-training/CMakeLists.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/clang/utils/perf-training/CMakeLists.txt b/clang/utils/perf-training/CMakeLists.txt index 49673790ff6e84..7790e373add57f 100644 --- a/clang/utils/perf-training/CMakeLists.txt +++ b/clang/utils/perf-training/CMakeLists.txt @@ -46,8 +46,8 @@ if(LLVM_BUILD_INSTRUMENTED) add_custom_target(generate-profdata DEPENDS ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata) if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR) llvm_ExternalProject_Add(generate-profraw-external ${CLANG_PGO_TRAINING_DATA_SOURCE_DIR} - USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw) - add_dependencies(generate-profdata generate-profraw-external) + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL) + add_dependencies(generate-profraw generate-profraw-external) endif() endif() endif() ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [libclc] [llvm] [libclc] More cross compilation fixes (PR #97811)
https://github.com/tstellar approved this pull request. https://github.com/llvm/llvm-project/pull/97811 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
tstellar wrote: /cherry-pick 8927576b8f6442bb6129bda597efee46176f8aec https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar milestoned https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
tstellar wrote: I'm going to push this tomorrow so it has a chance to land before -rc3. https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/101344 >From a39c6d5e3decabab87ff9360851981123ff962d7 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 31 Jul 2024 07:36:48 -0700 Subject: [PATCH 1/5] workflows/release-binaries: Enable flang builds on Windows This was fixed in 73d862e478738675f5d919c6a196429acd7b5f50. --- .github/workflows/release-binaries.yml | 8 1 file changed, 8 deletions(-) diff --git a/.github/workflows/release-binaries.yml b/.github/workflows/release-binaries.yml index b1b046dbad5f8b..2c6f4da8facd20 100644 --- a/.github/workflows/release-binaries.yml +++ b/.github/workflows/release-binaries.yml @@ -124,16 +124,8 @@ jobs: target_cmake_flags="$target_cmake_flags -DBOOTSTRAP_DARWIN_osx_ARCHS=$arches -DBOOTSTRAP_DARWIN_osx_BUILTIN_ARCHS=$arches" fi -# x86 macOS and x86 Windows have trouble building flang, so disable it. -# Windows: https://github.com/llvm/llvm-project/issues/100202 -# macOS: 'rebase opcodes terminated early at offset 1 of 80016' when building __fortran_builtins.mod build_flang="true" -if [ "$target" = "Windows-X64" ]; then - target_cmake_flags="$target_cmake_flags -DLLVM_RELEASE_ENABLE_PROJECTS=\"clang;lld;lldb;clang-tools-extra;bolt;polly;mlir\"" - build_flang="false" -fi - if [ "${{ runner.os }}" = "Windows" ]; then # The build times out on Windows, so we need to disable LTO. target_cmake_flags="$target_cmake_flags -DLLVM_RELEASE_ENABLE_LTO=OFF" >From b09d31aa4f2b9dae1439ec553e4a969152db1287 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Tue, 6 Aug 2024 17:31:58 -0700 Subject: [PATCH 2/5] [CMake][Release] Always build compiler-rt in stage1 It looks like flang has a dependecy on compiler-rt. Fixes #100202. --- clang/cmake/caches/Release.cmake | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index e5161dd9a27b96..b82b7aaf686529 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -47,11 +47,10 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") -set(STAGE1_RUNTIMES "") +set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") - list(APPEND STAGE1_RUNTIMES "compiler-rt") set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2-package >From 81f42350f0a3802703fa98a0ab2adf90a3db0038 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 7 Aug 2024 08:34:42 -0700 Subject: [PATCH 3/5] Fixup compiler-rt in Release cache --- clang/cmake/caches/Release.cmake | 6 ++ 1 file changed, 6 insertions(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index b82b7aaf686529..8c9ac04c9a220d 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -47,10 +47,16 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") + +# Building Flang on Windows requires compiler-rt, so we need to build it in +# stage1. set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") + if (NOT "compiler-rt" IN_LIST STAGE1_RUNTIMES) +list(APPEND STAGE1_PROJECTS "compiler-rt") + endif() set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2-package >From 607213117788f2ca8a8c91e42ee42581621a9bc1 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Thu, 8 Aug 2024 09:59:54 -0700 Subject: [PATCH 4/5] Update Release.cmake --- clang/cmake/caches/Release.cmake | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index 8c9ac04c9a220d..827ba2ccec403a 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -49,7 +49,8 @@ set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") # Building Flang on Windows requires compiler-rt, so we need to build it in -# stage1. +# stage1. compiler-rt is also required for building the Flang tests on +# macOS. set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) >From 0f746c9895a99c847cc8e263572b562eaf57db08 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Fri, 9 Aug 2024 12:50:50 -0700 Subject: [PATCH 5/5] Remove unneded lines --- clang/cmake/caches/Release.cmake | 3 --- 1 file changed, 3 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index 827ba2ccec403a..6d5f75ca0074ee 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -55,9 +55,6 @@ set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") - if (NOT "compiler-rt" IN_LIST STAGE1_RUNTIMES) -list(A
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
@@ -47,11 +47,17 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") -set(STAGE1_RUNTIMES "") + +# Building Flang on Windows requires compiler-rt, so we need to build it in +# stage1. compiler-rt is also required for building the Flang tests on +# macOS. +set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") - list(APPEND STAGE1_RUNTIMES "compiler-rt") + if (NOT "compiler-rt" IN_LIST STAGE1_RUNTIMES) +list(APPEND STAGE1_PROJECTS "compiler-rt") + endif() tstellar wrote: It's probably not needed any more. I was planning to conditionally add compiler-rt to stage1 targets based on if flang was enabled or not, but now I think that is too complicated. https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/101344 >From a39c6d5e3decabab87ff9360851981123ff962d7 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 31 Jul 2024 07:36:48 -0700 Subject: [PATCH 1/4] workflows/release-binaries: Enable flang builds on Windows This was fixed in 73d862e478738675f5d919c6a196429acd7b5f50. --- .github/workflows/release-binaries.yml | 8 1 file changed, 8 deletions(-) diff --git a/.github/workflows/release-binaries.yml b/.github/workflows/release-binaries.yml index b1b046dbad5f8..2c6f4da8facd2 100644 --- a/.github/workflows/release-binaries.yml +++ b/.github/workflows/release-binaries.yml @@ -124,16 +124,8 @@ jobs: target_cmake_flags="$target_cmake_flags -DBOOTSTRAP_DARWIN_osx_ARCHS=$arches -DBOOTSTRAP_DARWIN_osx_BUILTIN_ARCHS=$arches" fi -# x86 macOS and x86 Windows have trouble building flang, so disable it. -# Windows: https://github.com/llvm/llvm-project/issues/100202 -# macOS: 'rebase opcodes terminated early at offset 1 of 80016' when building __fortran_builtins.mod build_flang="true" -if [ "$target" = "Windows-X64" ]; then - target_cmake_flags="$target_cmake_flags -DLLVM_RELEASE_ENABLE_PROJECTS=\"clang;lld;lldb;clang-tools-extra;bolt;polly;mlir\"" - build_flang="false" -fi - if [ "${{ runner.os }}" = "Windows" ]; then # The build times out on Windows, so we need to disable LTO. target_cmake_flags="$target_cmake_flags -DLLVM_RELEASE_ENABLE_LTO=OFF" >From b09d31aa4f2b9dae1439ec553e4a969152db1287 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Tue, 6 Aug 2024 17:31:58 -0700 Subject: [PATCH 2/4] [CMake][Release] Always build compiler-rt in stage1 It looks like flang has a dependecy on compiler-rt. Fixes #100202. --- clang/cmake/caches/Release.cmake | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index e5161dd9a27b9..b82b7aaf68652 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -47,11 +47,10 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") -set(STAGE1_RUNTIMES "") +set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") - list(APPEND STAGE1_RUNTIMES "compiler-rt") set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2-package >From 81f42350f0a3802703fa98a0ab2adf90a3db0038 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 7 Aug 2024 08:34:42 -0700 Subject: [PATCH 3/4] Fixup compiler-rt in Release cache --- clang/cmake/caches/Release.cmake | 6 ++ 1 file changed, 6 insertions(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index b82b7aaf68652..8c9ac04c9a220 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -47,10 +47,16 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") + +# Building Flang on Windows requires compiler-rt, so we need to build it in +# stage1. set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") + if (NOT "compiler-rt" IN_LIST STAGE1_RUNTIMES) +list(APPEND STAGE1_PROJECTS "compiler-rt") + endif() set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2-package >From 607213117788f2ca8a8c91e42ee42581621a9bc1 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Thu, 8 Aug 2024 09:59:54 -0700 Subject: [PATCH 4/4] Update Release.cmake --- clang/cmake/caches/Release.cmake | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index 8c9ac04c9a220..827ba2ccec403 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -49,7 +49,8 @@ set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") # Building Flang on Windows requires compiler-rt, so we need to build it in -# stage1. +# stage1. compiler-rt is also required for building the Flang tests on +# macOS. set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
tstellar wrote: It looks like this change also fixed some build failures on macOS too. https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/101344 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] workflows/release-binaries: Enable flang builds on Windows (PR #101344)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/101344 >From a39c6d5e3decabab87ff9360851981123ff962d7 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 31 Jul 2024 07:36:48 -0700 Subject: [PATCH 1/3] workflows/release-binaries: Enable flang builds on Windows This was fixed in 73d862e478738675f5d919c6a196429acd7b5f50. --- .github/workflows/release-binaries.yml | 8 1 file changed, 8 deletions(-) diff --git a/.github/workflows/release-binaries.yml b/.github/workflows/release-binaries.yml index b1b046dbad5f8..2c6f4da8facd2 100644 --- a/.github/workflows/release-binaries.yml +++ b/.github/workflows/release-binaries.yml @@ -124,16 +124,8 @@ jobs: target_cmake_flags="$target_cmake_flags -DBOOTSTRAP_DARWIN_osx_ARCHS=$arches -DBOOTSTRAP_DARWIN_osx_BUILTIN_ARCHS=$arches" fi -# x86 macOS and x86 Windows have trouble building flang, so disable it. -# Windows: https://github.com/llvm/llvm-project/issues/100202 -# macOS: 'rebase opcodes terminated early at offset 1 of 80016' when building __fortran_builtins.mod build_flang="true" -if [ "$target" = "Windows-X64" ]; then - target_cmake_flags="$target_cmake_flags -DLLVM_RELEASE_ENABLE_PROJECTS=\"clang;lld;lldb;clang-tools-extra;bolt;polly;mlir\"" - build_flang="false" -fi - if [ "${{ runner.os }}" = "Windows" ]; then # The build times out on Windows, so we need to disable LTO. target_cmake_flags="$target_cmake_flags -DLLVM_RELEASE_ENABLE_LTO=OFF" >From b09d31aa4f2b9dae1439ec553e4a969152db1287 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Tue, 6 Aug 2024 17:31:58 -0700 Subject: [PATCH 2/3] [CMake][Release] Always build compiler-rt in stage1 It looks like flang has a dependecy on compiler-rt. Fixes #100202. --- clang/cmake/caches/Release.cmake | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index e5161dd9a27b9..b82b7aaf68652 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -47,11 +47,10 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") -set(STAGE1_RUNTIMES "") +set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") - list(APPEND STAGE1_RUNTIMES "compiler-rt") set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2-package >From 81f42350f0a3802703fa98a0ab2adf90a3db0038 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 7 Aug 2024 08:34:42 -0700 Subject: [PATCH 3/3] Fixup compiler-rt in Release cache --- clang/cmake/caches/Release.cmake | 6 ++ 1 file changed, 6 insertions(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index b82b7aaf68652..8c9ac04c9a220 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -47,10 +47,16 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(STAGE1_PROJECTS "clang") + +# Building Flang on Windows requires compiler-rt, so we need to build it in +# stage1. set(STAGE1_RUNTIMES "compiler-rt") if (LLVM_RELEASE_ENABLE_PGO) list(APPEND STAGE1_PROJECTS "lld") + if (NOT "compiler-rt" IN_LIST STAGE1_RUNTIMES) +list(APPEND STAGE1_PROJECTS "compiler-rt") + endif() set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2-package ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/98431 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
tstellar wrote: Only 2 (linux x86 and macOS AArch64) of the 4 configurations are passing currently, but I still think it makes sense to merge this so we can start building binaries for 19.1.0-rc1. I can work on fixing the other configurations in a follow up patch. https://github.com/llvm/llvm-project/pull/98431 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] RFC: [cmake] Export CLANG_RESOURCE_DIR in ClangConfig (PR #97197)
tstellar wrote: I think that the `GetResourcesPath` function is incorrect. There's no reason it should return a wrong value, since the CLANG_RESOURCE_DIR value is always known at compile time. https://github.com/llvm/llvm-project/pull/97197 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/98431 >From 5417784c025128eaffc09981ee72c7cc2ba487b3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 22 May 2024 15:30:53 -0700 Subject: [PATCH 1/4] Build release binaries for multiple targets This adds release binary builds for the 4 platforms currently supported by the free GitHub Action runners: * Linux x86_64 * Windows x86_64 * Mac x86_64 * Mac AArch64 The test stages for these are known to fail, but the creating and upoading of the release binaries should pass. --- .github/workflows/release-binaries-all.yml| 94 .../release-binaries-save-stage/action.yml| 33 ++ .../release-binaries-setup-stage/action.yml | 55 ++ .github/workflows/release-binaries.yml| 472 -- .github/workflows/release-tasks.yml | 10 + clang/cmake/caches/Release.cmake | 6 +- 6 files changed, 509 insertions(+), 161 deletions(-) create mode 100644 .github/workflows/release-binaries-all.yml create mode 100644 .github/workflows/release-binaries-save-stage/action.yml create mode 100644 .github/workflows/release-binaries-setup-stage/action.yml diff --git a/.github/workflows/release-binaries-all.yml b/.github/workflows/release-binaries-all.yml new file mode 100644 index 0..73c9d96946e33 --- /dev/null +++ b/.github/workflows/release-binaries-all.yml @@ -0,0 +1,94 @@ +name: Release Binaries All + +permissions: + contents: read # Default everything to read-only + +on: + workflow_dispatch: +inputs: + release-version: +description: 'Release Version' +required: true +type: string + upload: +description: 'Upload binaries to the release page' +required: true +default: false +type: boolean + + workflow_call: +inputs: + release-version: +description: 'Release Version' +required: true +type: string + upload: +description: 'Upload binaries to the release page' +required: true +default: false +type: boolean + + pull_request: +types: + - opened + - synchronize + - reopened + # When a PR is closed, we still start this workflow, but then skip + # all the jobs, which makes it effectively a no-op. The reason to + # do this is that it allows us to take advantage of concurrency groups + # to cancel in progress CI jobs whenever the PR is closed. + - closed +paths: + - '.github/workflows/release-binaries-all.yml' + - '.github/workflows/release-binaries.yml' + - '.github/workflows/release-binaries-setup-stage/*' + - '.github/workflows/release-binaries-save-stage/*' + +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || 'dispatch' }} + cancel-in-progress: True + +jobs: + setup-variables: +if: >- + (github.event_name != 'pull_request' || github.event.action != 'closed') +runs-on: ubuntu-22.04 +outputs: + release-version: ${{ steps.vars.outputs.release-version }} + upload: ${{ steps.vars.outputs.upload }} +steps: + - shell: bash +id: vars +run: | + upload="${{ inputs.upload }}" + release_version="${{ inputs.release-version }}" + if [ "${{ github.event_name }}" = "pull_request" ]; then +upload="false" +release_version="" + fi + echo "release-version=$release_version" >> "$GITHUB_OUTPUT" + echo "upload=$upload" >> "$GITHUB_OUTPUT" + + release-binaries-all: +name: Build Release Binaries +needs: + - setup-variables +permissions: + contents: write # For release uploads + id-token: write # For artifact attestations + attestations: write # For artifact attestations +strategy: + fail-fast: false + matrix: +runs-on: + - ubuntu-22.04 + - windows-2022 + - macos-13 + - macos-14 + +uses: ./.github/workflows/release-binaries.yml +with: + release-version: "${{ needs.setup-variables.outputs.release-version }}" + upload: ${{ needs.setup-variables.outputs.upload == 'true'}} + runs-on: "${{ matrix.runs-on }}" + diff --git a/.github/workflows/release-binaries-save-stage/action.yml b/.github/workflows/release-binaries-save-stage/action.yml new file mode 100644 index 0..c2939d79ca0d0 --- /dev/null +++ b/.github/workflows/release-binaries-save-stage/action.yml @@ -0,0 +1,33 @@ +name: Save Stage +inputs: + build-prefix: +description: "Directory containing the build directory." +required: true +type: 'string' + +runs: + using: "composite" + steps: +# We need to create an archive of the build directory, because it has too +# many files to upload. +- name: Package Build and Source Directories + shell: bash + run: | +# Windows does not support symlinks, so we need to dere
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
@@ -0,0 +1,55 @@ +name: Setup Stage + +inputs: + previous-artifact: +description: >- + A unique descriptor for the artifact from the previous stage. This will + be used to construct the final artifact pattern, which is: + $RUNNER_OS-$RUNNER_ARCH-$PREVIOUS_ARTIFACT-* +required: false +type: 'string' + +outputs: + build-prefix: +description: "Directory containing the build directory." +value: ${{ steps.build-prefix.outputs.build-prefix }} + +runs: + using: "composite" + steps: +- name: Install Ninja + uses: llvm/actions/install-ninja@22e9f909d35b50bd1181709564bfe816eaeaae81 # main tstellar wrote: The only advantage of having them in a separate repo is that you can use them without checking out the project source code. However, for this specific action I don't think it would ever be used without already having the source code, so there is probably no reason to keep it here. https://github.com/llvm/llvm-project/pull/98431 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
tstellar wrote: Summary of the test failures: [AArch64 MacOS](https://github.com/llvm/llvm-project/actions/runs/9893093291/job/27346084277?pr=98431): ``` ld64.lld: error: undefined symbol: __divsc3 >>> referenced by lib/libFortranRuntime.a(complex-powi.cpp.o):(symbol float >>> complex tgpowi(float complex, int)+0x24c) >>> referenced by lib/libFortranRuntime.a(complex-powi.cpp.o):(symbol float >>> complex tgpowi(float complex, long long)+0x22c) ``` [X86 MacOS](https://github.com/llvm/llvm-project/actions/runs/9893093291/job/27351402889?pr=98431): ``` dyld[14349]: Symbol not found: ___cxa_pure_virtual Referenced from: /Applications/Xcode_15.0.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/ld Expected in: <4C4C44CC--3144-A1FA-C2457EF8FF0F> /Users/runner/work/llvm-project/llvm-project/build/tools/clang/stage2-bins/lib/libc++.1.0.dylib ``` [Windows](https://github.com/llvm/llvm-project/actions/runs/9893093291/job/27347639047?pr=98431): ``` Total Discovered Tests: 123955 Skipped : 41 (0.03%) Unsupported : 6002 (4.84%) Passed : 115934 (93.53%) Passed With Retry: 1 (0.00%) Expectedly Failed:394 (0.32%) Unresolved : 1190 (0.96%) Failed :393 (0.32%) ``` [Linux](https://github.com/llvm/llvm-project/actions/runs/9893093291/job/27357951415?pr=98431) ``` Failed Tests (8): BOLT :: X86/inline-debug-info.test BOLT :: X86/line-number.test BOLT :: X86/match-functions-with-calls-as-anchors.test BOLT :: perf2bolt/perf_test.test lld :: ELF/basic-sparcv9.s lldb-api :: commands/expression/import-std-module/deque-dbg-info-content/TestDbgInfoContentDequeFromStdModule.py lldb-api :: commands/expression/import-std-module/list-dbg-info-content/TestDbgInfoContentListFromStdModule.py lldb-api :: commands/expression/import-std-module/vector-dbg-info-content/TestDbgInfoContentVectorFromStdModule.py ``` https://github.com/llvm/llvm-project/pull/98431 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/98431 >From 5417784c025128eaffc09981ee72c7cc2ba487b3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 22 May 2024 15:30:53 -0700 Subject: [PATCH 1/2] Build release binaries for multiple targets This adds release binary builds for the 4 platforms currently supported by the free GitHub Action runners: * Linux x86_64 * Windows x86_64 * Mac x86_64 * Mac AArch64 The test stages for these are known to fail, but the creating and upoading of the release binaries should pass. --- .github/workflows/release-binaries-all.yml| 94 .../release-binaries-save-stage/action.yml| 33 ++ .../release-binaries-setup-stage/action.yml | 55 ++ .github/workflows/release-binaries.yml| 472 -- .github/workflows/release-tasks.yml | 10 + clang/cmake/caches/Release.cmake | 6 +- 6 files changed, 509 insertions(+), 161 deletions(-) create mode 100644 .github/workflows/release-binaries-all.yml create mode 100644 .github/workflows/release-binaries-save-stage/action.yml create mode 100644 .github/workflows/release-binaries-setup-stage/action.yml diff --git a/.github/workflows/release-binaries-all.yml b/.github/workflows/release-binaries-all.yml new file mode 100644 index 0..73c9d96946e33 --- /dev/null +++ b/.github/workflows/release-binaries-all.yml @@ -0,0 +1,94 @@ +name: Release Binaries All + +permissions: + contents: read # Default everything to read-only + +on: + workflow_dispatch: +inputs: + release-version: +description: 'Release Version' +required: true +type: string + upload: +description: 'Upload binaries to the release page' +required: true +default: false +type: boolean + + workflow_call: +inputs: + release-version: +description: 'Release Version' +required: true +type: string + upload: +description: 'Upload binaries to the release page' +required: true +default: false +type: boolean + + pull_request: +types: + - opened + - synchronize + - reopened + # When a PR is closed, we still start this workflow, but then skip + # all the jobs, which makes it effectively a no-op. The reason to + # do this is that it allows us to take advantage of concurrency groups + # to cancel in progress CI jobs whenever the PR is closed. + - closed +paths: + - '.github/workflows/release-binaries-all.yml' + - '.github/workflows/release-binaries.yml' + - '.github/workflows/release-binaries-setup-stage/*' + - '.github/workflows/release-binaries-save-stage/*' + +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || 'dispatch' }} + cancel-in-progress: True + +jobs: + setup-variables: +if: >- + (github.event_name != 'pull_request' || github.event.action != 'closed') +runs-on: ubuntu-22.04 +outputs: + release-version: ${{ steps.vars.outputs.release-version }} + upload: ${{ steps.vars.outputs.upload }} +steps: + - shell: bash +id: vars +run: | + upload="${{ inputs.upload }}" + release_version="${{ inputs.release-version }}" + if [ "${{ github.event_name }}" = "pull_request" ]; then +upload="false" +release_version="" + fi + echo "release-version=$release_version" >> "$GITHUB_OUTPUT" + echo "upload=$upload" >> "$GITHUB_OUTPUT" + + release-binaries-all: +name: Build Release Binaries +needs: + - setup-variables +permissions: + contents: write # For release uploads + id-token: write # For artifact attestations + attestations: write # For artifact attestations +strategy: + fail-fast: false + matrix: +runs-on: + - ubuntu-22.04 + - windows-2022 + - macos-13 + - macos-14 + +uses: ./.github/workflows/release-binaries.yml +with: + release-version: "${{ needs.setup-variables.outputs.release-version }}" + upload: ${{ needs.setup-variables.outputs.upload == 'true'}} + runs-on: "${{ matrix.runs-on }}" + diff --git a/.github/workflows/release-binaries-save-stage/action.yml b/.github/workflows/release-binaries-save-stage/action.yml new file mode 100644 index 0..c2939d79ca0d0 --- /dev/null +++ b/.github/workflows/release-binaries-save-stage/action.yml @@ -0,0 +1,33 @@ +name: Save Stage +inputs: + build-prefix: +description: "Directory containing the build directory." +required: true +type: 'string' + +runs: + using: "composite" + steps: +# We need to create an archive of the build directory, because it has too +# many files to upload. +- name: Package Build and Source Directories + shell: bash + run: | +# Windows does not support symlinks, so we need to dere
[clang] [llvm] Build release binaries for multiple targets (PR #98431)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/98431 This adds release binary builds for the 4 platforms currently supported by the free GitHub Action runners: * Linux x86_64 * Windows x86_64 * Mac x86_64 * Mac AArch64 The test stages for these are known to fail, but the creating and upoading of the release binaries should pass. >From 5417784c025128eaffc09981ee72c7cc2ba487b3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 22 May 2024 15:30:53 -0700 Subject: [PATCH] Build release binaries for multiple targets This adds release binary builds for the 4 platforms currently supported by the free GitHub Action runners: * Linux x86_64 * Windows x86_64 * Mac x86_64 * Mac AArch64 The test stages for these are known to fail, but the creating and upoading of the release binaries should pass. --- .github/workflows/release-binaries-all.yml| 94 .../release-binaries-save-stage/action.yml| 33 ++ .../release-binaries-setup-stage/action.yml | 55 ++ .github/workflows/release-binaries.yml| 472 -- .github/workflows/release-tasks.yml | 10 + clang/cmake/caches/Release.cmake | 6 +- 6 files changed, 509 insertions(+), 161 deletions(-) create mode 100644 .github/workflows/release-binaries-all.yml create mode 100644 .github/workflows/release-binaries-save-stage/action.yml create mode 100644 .github/workflows/release-binaries-setup-stage/action.yml diff --git a/.github/workflows/release-binaries-all.yml b/.github/workflows/release-binaries-all.yml new file mode 100644 index 0..73c9d96946e33 --- /dev/null +++ b/.github/workflows/release-binaries-all.yml @@ -0,0 +1,94 @@ +name: Release Binaries All + +permissions: + contents: read # Default everything to read-only + +on: + workflow_dispatch: +inputs: + release-version: +description: 'Release Version' +required: true +type: string + upload: +description: 'Upload binaries to the release page' +required: true +default: false +type: boolean + + workflow_call: +inputs: + release-version: +description: 'Release Version' +required: true +type: string + upload: +description: 'Upload binaries to the release page' +required: true +default: false +type: boolean + + pull_request: +types: + - opened + - synchronize + - reopened + # When a PR is closed, we still start this workflow, but then skip + # all the jobs, which makes it effectively a no-op. The reason to + # do this is that it allows us to take advantage of concurrency groups + # to cancel in progress CI jobs whenever the PR is closed. + - closed +paths: + - '.github/workflows/release-binaries-all.yml' + - '.github/workflows/release-binaries.yml' + - '.github/workflows/release-binaries-setup-stage/*' + - '.github/workflows/release-binaries-save-stage/*' + +concurrency: + group: ${{ github.workflow }}-${{ github.event.pull_request.number || 'dispatch' }} + cancel-in-progress: True + +jobs: + setup-variables: +if: >- + (github.event_name != 'pull_request' || github.event.action != 'closed') +runs-on: ubuntu-22.04 +outputs: + release-version: ${{ steps.vars.outputs.release-version }} + upload: ${{ steps.vars.outputs.upload }} +steps: + - shell: bash +id: vars +run: | + upload="${{ inputs.upload }}" + release_version="${{ inputs.release-version }}" + if [ "${{ github.event_name }}" = "pull_request" ]; then +upload="false" +release_version="" + fi + echo "release-version=$release_version" >> "$GITHUB_OUTPUT" + echo "upload=$upload" >> "$GITHUB_OUTPUT" + + release-binaries-all: +name: Build Release Binaries +needs: + - setup-variables +permissions: + contents: write # For release uploads + id-token: write # For artifact attestations + attestations: write # For artifact attestations +strategy: + fail-fast: false + matrix: +runs-on: + - ubuntu-22.04 + - windows-2022 + - macos-13 + - macos-14 + +uses: ./.github/workflows/release-binaries.yml +with: + release-version: "${{ needs.setup-variables.outputs.release-version }}" + upload: ${{ needs.setup-variables.outputs.upload == 'true'}} + runs-on: "${{ matrix.runs-on }}" + diff --git a/.github/workflows/release-binaries-save-stage/action.yml b/.github/workflows/release-binaries-save-stage/action.yml new file mode 100644 index 0..c2939d79ca0d0 --- /dev/null +++ b/.github/workflows/release-binaries-save-stage/action.yml @@ -0,0 +1,33 @@ +name: Save Stage +inputs: + build-prefix: +description: "Directory containing the build directory." +required: true +type: 'string' + +runs: + using
[libclc] [libclc] Fix cross in-tree builds (PR #97392)
https://github.com/tstellar approved this pull request. https://github.com/llvm/llvm-project/pull/97392 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][cmake] Fixes for PGO builds when invoking ninja twice (PR #92591)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/92591 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Use the TXZ cpack generator for binaries (PR #90138)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/90138 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Use the TXZ cpack generator for binaries (PR #90138)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/90138 >From 0d0484ac99affdc8ccb9bc3a1eff827cf996c51c Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 24 Apr 2024 07:54:41 -0700 Subject: [PATCH 1/3] [CMake][Release] Use the TGZ cpack generator for binaries --- clang/cmake/caches/Release.cmake | 1 + 1 file changed, 1 insertion(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index c164d5497275f..aa7e6f99e94d5 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -90,4 +90,5 @@ endif() # Final Stage Config (stage2) set_final_stage_var(LLVM_ENABLE_RUNTIMES "${LLVM_RELEASE_ENABLE_RUNTIMES}" STRING) set_final_stage_var(LLVM_ENABLE_PROJECTS "${LLVM_RELEASE_ENABLE_PROJECTS}" STRING) +set_final_stage_var(CPACK_GENERATOR "TGZ" STRING) >From 2bbc2c97a91ec98a57fdd12c373c0d17bd9eb023 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Fri, 17 May 2024 16:39:50 -0700 Subject: [PATCH 2/3] Switch to xz compression --- .github/workflows/release-binaries.yml | 2 +- clang/cmake/caches/Release.cmake | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/release-binaries.yml b/.github/workflows/release-binaries.yml index fc497a7de94f7..540675cc62d6a 100644 --- a/.github/workflows/release-binaries.yml +++ b/.github/workflows/release-binaries.yml @@ -214,7 +214,7 @@ jobs: - id: package-info run: | -filename="LLVM-${{ needs.prepare.outputs.release-version }}-Linux.tar.gz" +filename="LLVM-${{ needs.prepare.outputs.release-version }}-Linux.tar.xz" echo "filename=$filename" >> $GITHUB_OUTPUT echo "path=/mnt/build/tools/clang/stage2-bins/$filename" >> $GITHUB_OUTPUT diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index c6bcbdfebf999..24a45039c9dab 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -91,5 +91,5 @@ endif() # Final Stage Config (stage2) set_final_stage_var(LLVM_ENABLE_RUNTIMES "${LLVM_RELEASE_ENABLE_RUNTIMES}" STRING) set_final_stage_var(LLVM_ENABLE_PROJECTS "${LLVM_RELEASE_ENABLE_PROJECTS}" STRING) -set_final_stage_var(CPACK_GENERATOR "TGZ" STRING) +set_final_stage_var(CPACK_GENERATOR "TXZ" STRING) >From c8587febd7b7a318b9d5566ff540ca6881404bd7 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Thu, 6 Jun 2024 17:13:48 -0700 Subject: [PATCH 3/3] Use CPACK_ARCHIVE_THREADS --- clang/cmake/caches/Release.cmake | 1 + 1 file changed, 1 insertion(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index 24a45039c9dab..1dfb1bc535bf1 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -92,4 +92,5 @@ endif() set_final_stage_var(LLVM_ENABLE_RUNTIMES "${LLVM_RELEASE_ENABLE_RUNTIMES}" STRING) set_final_stage_var(LLVM_ENABLE_PROJECTS "${LLVM_RELEASE_ENABLE_PROJECTS}" STRING) set_final_stage_var(CPACK_GENERATOR "TXZ" STRING) +set_final_stage_var(CPACK_ARCHIVE_THREADS "0" STRING) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][cmake] Fixes for PGO builds when invoking ninja twice (PR #92591)
tstellar wrote: Ping. https://github.com/llvm/llvm-project/pull/92591 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [clang][ci] Move libc++ testing into the main PR pipeline (PR #93318)
https://github.com/tstellar approved this pull request. This seems OK to me at first glance. You may want to have someone double-checkout those new libcxx jobs. I'm not really familiar with those. https://github.com/llvm/llvm-project/pull/93318 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [clang][ci] Move libc++ testing into the main PR pipeline (PR #93318)
@@ -178,6 +194,15 @@ function check-targets() { cross-project-tests) echo "check-cross-project" ;; +libcxx) + echo "check-cxx" +;; +libcxxabi) + echo "check-cxxabi" +;; +libunwind) + echo "check-unwind" tstellar wrote: OK, makes sense. https://github.com/llvm/llvm-project/pull/93318 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [clang][ci] Move libc++ testing into the main PR pipeline (PR #93318)
https://github.com/tstellar commented: Can we also delete https://github.com/llvm/llvm-project/blob/main/clang/utils/ci/buildkite-pipeline.yml https://github.com/llvm/llvm-project/pull/93318 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [clang][ci] Move libc++ testing into the main PR pipeline (PR #93318)
@@ -178,6 +194,15 @@ function check-targets() { cross-project-tests) echo "check-cross-project" ;; +libcxx) + echo "check-cxx" +;; +libcxxabi) + echo "check-cxxabi" +;; +libunwind) + echo "check-unwind" tstellar wrote: Won't this also add these check targets to the main build? Do we need that if we have separate steps now for building and testing the runtimes? https://github.com/llvm/llvm-project/pull/93318 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [clang][ci] Move libc++ testing into the main PR pipeline (PR #93318)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/93318 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][ci] Remove unnecessary BuildKite jobs for Clang (PR #93233)
https://github.com/tstellar approved this pull request. LGTM. Thanks, this should help free up some builder resources. https://github.com/llvm/llvm-project/pull/93233 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][ci] Remove unnecessary BuildKite jobs for Clang (PR #93233)
https://github.com/tstellar commented: If we aren't running the tests any more, can we build clang with `-DLLVM_TARGETS_TO_BUILD=Native` https://github.com/llvm/llvm-project/pull/93233 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Use the TXZ cpack generator for binaries (PR #90138)
https://github.com/tstellar milestoned https://github.com/llvm/llvm-project/pull/90138 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][cmake] Fixes for PGO builds when invoking ninja twice (PR #92591)
https://github.com/tstellar milestoned https://github.com/llvm/llvm-project/pull/92591 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Use the TXZ cpack generator for binaries (PR #90138)
tstellar wrote: > Did you consider the parallel setting as well? Forgot about that part, I will make that change too. https://github.com/llvm/llvm-project/pull/90138 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Disable compiling Flang and running its tests on Clang changes (PR #92740)
https://github.com/tstellar approved this pull request. LGTM, thank you. https://github.com/llvm/llvm-project/pull/92740 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Disable compiling Flang and running its tests on Clang changes (PR #92740)
@@ -72,7 +72,7 @@ class PrintFunctionsConsumer : public ASTConsumer { *sema.LateParsedTemplateMap.find(FD)->second; sema.LateTemplateParser(sema.OpaqueParser, LPT); llvm::errs() << "late-parsed-decl: \"" << FD->getNameAsString() << "\"\n"; -} +} tstellar wrote: This seems like an unrelated change. https://github.com/llvm/llvm-project/pull/92740 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Disable compiling Flang and running its tests on Clang changes (PR #92740)
@@ -68,7 +68,7 @@ function compute-projects-to-test() { done ;; clang) - for p in clang-tools-extra compiler-rt flang lldb cross-project-tests; do + for p in clang-tools-extra compiler-rt lldb cross-project-tests; do tstellar wrote: Don't you also need to remove this from the llvm and mlir switch cases? Also, we are we doing this here and not in the `exclude-windows` function? https://github.com/llvm/llvm-project/pull/92740 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Use the TXZ cpack generator for binaries (PR #90138)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/90138 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][cmake] Fixes for PGO builds when invoking ninja twice (PR #92591)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/92591 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][cmake] Fixes for PGO builds when invoking ninja twice (PR #92591)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/92591 This fixes two problems with the 2-stage PGO builds. The first problem was that the stage2-instrumented and stage2 targets would not be built on the second ninja invocation. For example: # This would work as expected. $ ninja -v -C build stage2-instrumented-generate-profdata # Edit a file. $ touch llvm/lib/Support/StringExtras.cpp # This would rebuild stage1 only and not build stage2-instrumented or # regenerate the profile data. $ ninja -v -C build stage2-instrumented-generate-profdata The second problem was that in some cases, the profile data would be regenerated, but not all of the stage2 targets would be rebuilt. One common scenario where this would happen is: # This would work as expected. $ ninja -C build stage2-check-all # This would regenerate the profile data, but then only build the # targets that were needed for install, but weren't needed for # check-all. This would cause errors like: # ld.lld: error: Function Import: link error: linking module flags # 'ProfileSummary': IDs have conflicting values in ... # This is because the executibles being built for the install target # used the new profile data, but they were linking with libraries that # used the old profile data. $ ninja -C build stage2-install With this change we can re-enable PGO for the release builds. >From 33d373044e247a568d2fce7371a6ac1c0527f1a2 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Thu, 16 May 2024 13:49:40 + Subject: [PATCH] [clang][cmake] Fixes for PGO builds when invoking ninja twice This fixes two problems with the 2-stage PGO builds. The first problem was that the stage2-instrumented and stage2 targets would not be built on the second ninja invocation. For example: # This would work as expected. $ ninja -v -C build stage2-instrumented-generate-profdata # Edit a file. $ touch llvm/lib/Support/StringExtras.cpp # This would rebuild stage1 only and not build stage2-instrumented or # regenerate the profile data. $ ninja -v -C build stage2-instrumented-generate-profdata The second problem was that in some cases, the profile data would be regenerated, but not all of the stage2 targets would be rebuilt. One common scenario where this would happen is: # This would work as expected. $ ninja -C build stage2-check-all # This would regenerate the profile data, but then only build the # targets that were needed for install, but weren't needed for # check-all. This would cause errors like: # ld.lld: error: Function Import: link error: linking module flags # 'ProfileSummary': IDs have conflicting values in ... # This is because the executibles being built for the install target # used the new profile data, but they were linking with libraries that # used the old profile data. $ ninja -C build stage2-install With this change we can re-enable PGO for the release builds. --- clang/CMakeLists.txt | 18 ++ clang/cmake/caches/Release.cmake | 2 +- clang/utils/perf-training/CMakeLists.txt | 17 ++--- 3 files changed, 21 insertions(+), 16 deletions(-) diff --git a/clang/CMakeLists.txt b/clang/CMakeLists.txt index c20ce47a12abb..b4cf83d18c4e6 100644 --- a/clang/CMakeLists.txt +++ b/clang/CMakeLists.txt @@ -849,23 +849,17 @@ if (CLANG_ENABLE_BOOTSTRAP) set(CLANG_BOOTSTRAP_TARGETS check-llvm check-clang check-all) endif() foreach(target ${CLANG_BOOTSTRAP_TARGETS}) -# Install targets have side effects, so we always want to execute them. -# "install" is reserved by CMake and can't be used as a step name for -# ExternalProject_Add_Step, so we can match against "^install-" instead of -# "^install" to get a tighter match. CMake's installation scripts already -# skip up-to-date files, so there's no behavior change if you install to the -# same destination multiple times. -if(target MATCHES "^install-") - set(step_always ON) -else() - set(step_always OFF) -endif() ExternalProject_Add_Step(${NEXT_CLANG_STAGE} ${target} COMMAND ${CMAKE_COMMAND} --build --target ${target} COMMENT "Performing ${target} for '${NEXT_CLANG_STAGE}'" DEPENDEES configure - ALWAYS ${step_always} + # We need to set ALWAYS to ON here, otherwise these targets won't be + # built on a second invocation of ninja. The targets have their own + # logic to determine if they should build or not so setting ALWAYS ON + # here does not mean the targets will always rebuild it just means that + # they will check their dependenices and see if they need to be built. + ALWAYS ON EXCLUDE_FROM_MAIN ON USES_TERMINAL 1 ) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index c0bfcbdfc1c2a..d40a84cf2a5f0 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -30,7 +30,7 @@ endfunction(
[clang] [llvm] [X86][Driver] Do not add `-evex512` for `-march=native` when the target doesn't support AVX512 (PR #91694)
tstellar wrote: @phoebewang Can you add a release note for this on the PR for the release branch and then add the release:note label. https://github.com/llvm/llvm-project/pull/91694 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Enable CMAKE_POSITION_INDEPENDENT_CODE (PR #90139)
tstellar wrote: This was merged in another PR: b7e2397c54b7cddac8fa188e68073f78e895a57a. https://github.com/llvm/llvm-project/pull/90139 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-tools-extra] [compiler-rt] [libcxx] [lld] [lldb] [llvm] [mlir] [polly] fix(python): fix invalid escape sequences (PR #91856)
tstellar wrote: Two questions: * Did you use a script to generate this patch? * Why do we need to make this change? https://github.com/llvm/llvm-project/pull/91856 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Enable CMAKE_POSITION_INDEPENDENT_CODE (PR #90139)
tstellar wrote: /cherry-pick 53ff002c6f7ec64a75ab0990b1314cc6b4bb67cf https://github.com/llvm/llvm-project/pull/90139 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Enable CMAKE_POSITION_INDEPENDENT_CODE (PR #90139)
https://github.com/tstellar milestoned https://github.com/llvm/llvm-project/pull/90139 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Enable CMAKE_POSITION_INDEPENDENT_CODE (PR #90139)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/90139 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Enable CMAKE_POSITION_INDEPENDENT_CODE (PR #90139)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/90139 Set this in the cache file directly instead of via the test-release.sh script so that the release builds can be reproduced with just the cache file. >From 67f3d1ede686758238bb37a2ea50790750f7693f Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 24 Apr 2024 07:57:09 -0700 Subject: [PATCH] [CMake][Release] Enable CMAKE_POSITION_INDEPENDENT_CODE Set this in the cache file directly instead of via the test-release.sh script so that the release builds can be reproduced with just the cache file. --- clang/cmake/caches/Release.cmake | 1 + llvm/utils/release/test-release.sh | 3 +-- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index c164d5497275f3..c0bfcbdfc1c2ae 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -82,6 +82,7 @@ set(LLVM_ENABLE_PROJECTS ${STAGE1_PROJECTS} CACHE STRING "") # stage2-instrumented and Final Stage Config: # Options that need to be set in both the instrumented stage (if we are doing # a pgo build) and the final stage. +set_instrument_and_final_stage_var(CMAKE_POSITION_INDEPENDENT_CODE "ON" STRING) set_instrument_and_final_stage_var(LLVM_ENABLE_LTO "${LLVM_RELEASE_ENABLE_LTO}" STRING) if (LLVM_RELEASE_ENABLE_LTO) set_instrument_and_final_stage_var(LLVM_ENABLE_LLD "ON" BOOL) diff --git a/llvm/utils/release/test-release.sh b/llvm/utils/release/test-release.sh index 4314b565e11b03..050004aa08c493 100755 --- a/llvm/utils/release/test-release.sh +++ b/llvm/utils/release/test-release.sh @@ -353,8 +353,7 @@ function build_with_cmake_cache() { env CC="$c_compiler" CXX="$cxx_compiler" \ cmake -G "$generator" -B $CMakeBuildDir -S $SrcDir/llvm \ -C $SrcDir/clang/cmake/caches/Release.cmake \ - -DCLANG_BOOTSTRAP_PASSTHROUGH="CMAKE_POSITION_INDEPENDENT_CODE;LLVM_LIT_ARGS" \ --DCMAKE_POSITION_INDEPENDENT_CODE=ON \ + -DCLANG_BOOTSTRAP_PASSTHROUGH="LLVM_LIT_ARGS" \ -DLLVM_LIT_ARGS="-j $NumJobs $LitVerbose" \ $ExtraConfigureFlags 2>&1 | tee $LogDir/llvm.configure-$Flavor.log ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Use the TGZ cpack generator for binaries (PR #90138)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/90138 None >From 0d0484ac99affdc8ccb9bc3a1eff827cf996c51c Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Wed, 24 Apr 2024 07:54:41 -0700 Subject: [PATCH] [CMake][Release] Use the TGZ cpack generator for binaries --- clang/cmake/caches/Release.cmake | 1 + 1 file changed, 1 insertion(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index c164d5497275f3..aa7e6f99e94d52 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -90,4 +90,5 @@ endif() # Final Stage Config (stage2) set_final_stage_var(LLVM_ENABLE_RUNTIMES "${LLVM_RELEASE_ENABLE_RUNTIMES}" STRING) set_final_stage_var(LLVM_ENABLE_PROJECTS "${LLVM_RELEASE_ENABLE_PROJECTS}" STRING) +set_final_stage_var(CPACK_GENERATOR "TGZ" STRING) ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Refactor cache file and use two stages for non-PGO builds (PR #89812)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/89812 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Refactor cache file and use two stages for non-PGO builds (PR #89812)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/89812 >From 275979816a1de8b16a6c45b5ee2ef4e73c8828ba Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Tue, 23 Apr 2024 19:29:04 + Subject: [PATCH] [CMake][Release] Refactor cache file and use two stages for non-PGO builds Completely refactor the cache file to simplify it and remove unnecessary variables. The main functional change here is that the non-PGO builds now use two stages, so `ninja -C build stage2-package` can be used with both PGO and non-PGO builds. --- clang/cmake/caches/Release.cmake | 134 +++ 1 file changed, 67 insertions(+), 67 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index bd1f688d61a7ea..c164d5497275f3 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -1,93 +1,93 @@ # Plain options configure the first build. # BOOTSTRAP_* options configure the second build. # BOOTSTRAP_BOOTSTRAP_* options configure the third build. +# PGO Builds have 3 stages (stage1, stage2-instrumented, stage2) +# non-PGO Builds have 2 stages (stage1, stage2) -# General Options + +function (set_final_stage_var name value type) + if (LLVM_RELEASE_ENABLE_PGO) +set(BOOTSTRAP_BOOTSTRAP_${name} ${value} CACHE ${type} "") + else() +set(BOOTSTRAP_${name} ${value} CACHE ${type} "") + endif() +endfunction() + +function (set_instrument_and_final_stage_var name value type) + # This sets the varaible for the final stage in non-PGO builds and in + # the stage2-instrumented stage for PGO builds. + set(BOOTSTRAP_${name} ${value} CACHE ${type} "") + if (LLVM_RELEASE_ENABLE_PGO) +# Set the variable in the final stage for PGO builds. +set(BOOTSTRAP_BOOTSTRAP_${name} ${value} CACHE ${type} "") + endif() +endfunction() + +# General Options: +# If you want to override any of the LLVM_RELEASE_* variables you can set them +# on the command line via -D, but you need to do this before you pass this +# cache file to CMake via -C. e.g. +# +# cmake -D LLVM_RELEASE_ENABLE_PGO=ON -C Release.cmake set(LLVM_RELEASE_ENABLE_LTO THIN CACHE STRING "") set(LLVM_RELEASE_ENABLE_PGO OFF CACHE BOOL "") - +set(LLVM_RELEASE_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") +set(LLVM_RELEASE_ENABLE_PROJECTS "clang;lld;lldb;clang-tools-extra;bolt;polly;mlir;flang" CACHE STRING "") +# Note we don't need to add install here, since it is one of the pre-defined +# steps. +set(LLVM_RELEASE_FINAL_STAGE_TARGETS "clang;package;check-all;check-llvm;check-clang" CACHE STRING "") set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") -# Stage 1 Bootstrap Setup +# Stage 1 Options +set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") + +set(STAGE1_PROJECTS "clang") +set(STAGE1_RUNTIMES "") + if (LLVM_RELEASE_ENABLE_PGO) + list(APPEND STAGE1_PROJECTS "lld") + list(APPEND STAGE1_RUNTIMES "compiler-rt") set(CLANG_BOOTSTRAP_TARGETS generate-profdata -stage2 +stage2-package stage2-clang -stage2-distribution stage2-install -stage2-install-distribution -stage2-install-distribution-toolchain stage2-check-all stage2-check-llvm -stage2-check-clang -stage2-test-suite CACHE STRING "") -else() - set(CLANG_BOOTSTRAP_TARGETS -clang -check-all -check-llvm -check-clang -test-suite -stage3 -stage3-clang -stage3-check-all -stage3-check-llvm -stage3-check-clang -stage3-install -stage3-test-suite CACHE STRING "") -endif() +stage2-check-clang CACHE STRING "") -# Stage 1 Options -set(STAGE1_PROJECTS "clang") -set(STAGE1_RUNTIMES "") + # Configuration for stage2-instrumented + set(BOOTSTRAP_CLANG_ENABLE_BOOTSTRAP ON CACHE STRING "") + # This enables the build targets for the final stage which is called stage2. + set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS ${LLVM_RELEASE_FINAL_STAGE_TARGETS} CACHE STRING "") + set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED IR CACHE STRING "") + set(BOOTSTRAP_LLVM_ENABLE_RUNTIMES "compiler-rt" CACHE STRING "") + set(BOOTSTRAP_LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -if (LLVM_RELEASE_ENABLE_PGO) - list(APPEND STAGE1_PROJECTS "lld") - list(APPEND STAGE1_RUNTIMES "compiler-rt") +else() + if (LLVM_RELEASE_ENABLE_LTO) +list(APPEND STAGE1_PROJECTS "lld") + endif() + # Any targets added here will be given the target name stage2-${target}, so + # if you want to run them you can just use: + # ninja -C $BUILDDIR stage2-${target} + set(CLANG_BOOTSTRAP_TARGETS ${LLVM_RELEASE_FINAL_STAGE_TARGETS} CACHE STRING "") endif() +# Stage 1 Common Config set(LLVM_ENABLE_RUNTIMES ${STAGE1_RUNTIMES} CACHE STRING "") set(LLVM_ENABLE_PROJECTS ${STAGE1_PROJECTS} CACHE STRING "") -set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") - -# Stage 2 Bootstrap Setup -set(BOOTSTRAP_CLANG_ENABLE_BOOTSTRAP ON CACHE STRING "") -set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS - cl
[clang] [CMake][Release] Refactor cache file and use two stages for non-PGO builds (PR #89812)
tstellar wrote: I've been testing this with #89521. https://github.com/llvm/llvm-project/pull/89812 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Add stage2-package target (PR #89517)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/89517 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Refactor cache file and use two stages for non-PGO builds (PR #89812)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/89812 Completely refactor the cache file to simplify it and remove unnecessary variables. The main functional change here is that the non-PGO builds now use two stages, so `ninja -C build stage2-package` can be used with both PGO and non-PGO builds. >From 275979816a1de8b16a6c45b5ee2ef4e73c8828ba Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Tue, 23 Apr 2024 19:29:04 + Subject: [PATCH] [CMake][Release] Refactor cache file and use two stages for non-PGO builds Completely refactor the cache file to simplify it and remove unnecessary variables. The main functional change here is that the non-PGO builds now use two stages, so `ninja -C build stage2-package` can be used with both PGO and non-PGO builds. --- clang/cmake/caches/Release.cmake | 134 +++ 1 file changed, 67 insertions(+), 67 deletions(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index bd1f688d61a7ea..c164d5497275f3 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -1,93 +1,93 @@ # Plain options configure the first build. # BOOTSTRAP_* options configure the second build. # BOOTSTRAP_BOOTSTRAP_* options configure the third build. +# PGO Builds have 3 stages (stage1, stage2-instrumented, stage2) +# non-PGO Builds have 2 stages (stage1, stage2) -# General Options + +function (set_final_stage_var name value type) + if (LLVM_RELEASE_ENABLE_PGO) +set(BOOTSTRAP_BOOTSTRAP_${name} ${value} CACHE ${type} "") + else() +set(BOOTSTRAP_${name} ${value} CACHE ${type} "") + endif() +endfunction() + +function (set_instrument_and_final_stage_var name value type) + # This sets the varaible for the final stage in non-PGO builds and in + # the stage2-instrumented stage for PGO builds. + set(BOOTSTRAP_${name} ${value} CACHE ${type} "") + if (LLVM_RELEASE_ENABLE_PGO) +# Set the variable in the final stage for PGO builds. +set(BOOTSTRAP_BOOTSTRAP_${name} ${value} CACHE ${type} "") + endif() +endfunction() + +# General Options: +# If you want to override any of the LLVM_RELEASE_* variables you can set them +# on the command line via -D, but you need to do this before you pass this +# cache file to CMake via -C. e.g. +# +# cmake -D LLVM_RELEASE_ENABLE_PGO=ON -C Release.cmake set(LLVM_RELEASE_ENABLE_LTO THIN CACHE STRING "") set(LLVM_RELEASE_ENABLE_PGO OFF CACHE BOOL "") - +set(LLVM_RELEASE_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") +set(LLVM_RELEASE_ENABLE_PROJECTS "clang;lld;lldb;clang-tools-extra;bolt;polly;mlir;flang" CACHE STRING "") +# Note we don't need to add install here, since it is one of the pre-defined +# steps. +set(LLVM_RELEASE_FINAL_STAGE_TARGETS "clang;package;check-all;check-llvm;check-clang" CACHE STRING "") set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") -# Stage 1 Bootstrap Setup +# Stage 1 Options +set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") + +set(STAGE1_PROJECTS "clang") +set(STAGE1_RUNTIMES "") + if (LLVM_RELEASE_ENABLE_PGO) + list(APPEND STAGE1_PROJECTS "lld") + list(APPEND STAGE1_RUNTIMES "compiler-rt") set(CLANG_BOOTSTRAP_TARGETS generate-profdata -stage2 +stage2-package stage2-clang -stage2-distribution stage2-install -stage2-install-distribution -stage2-install-distribution-toolchain stage2-check-all stage2-check-llvm -stage2-check-clang -stage2-test-suite CACHE STRING "") -else() - set(CLANG_BOOTSTRAP_TARGETS -clang -check-all -check-llvm -check-clang -test-suite -stage3 -stage3-clang -stage3-check-all -stage3-check-llvm -stage3-check-clang -stage3-install -stage3-test-suite CACHE STRING "") -endif() +stage2-check-clang CACHE STRING "") -# Stage 1 Options -set(STAGE1_PROJECTS "clang") -set(STAGE1_RUNTIMES "") + # Configuration for stage2-instrumented + set(BOOTSTRAP_CLANG_ENABLE_BOOTSTRAP ON CACHE STRING "") + # This enables the build targets for the final stage which is called stage2. + set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS ${LLVM_RELEASE_FINAL_STAGE_TARGETS} CACHE STRING "") + set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED IR CACHE STRING "") + set(BOOTSTRAP_LLVM_ENABLE_RUNTIMES "compiler-rt" CACHE STRING "") + set(BOOTSTRAP_LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -if (LLVM_RELEASE_ENABLE_PGO) - list(APPEND STAGE1_PROJECTS "lld") - list(APPEND STAGE1_RUNTIMES "compiler-rt") +else() + if (LLVM_RELEASE_ENABLE_LTO) +list(APPEND STAGE1_PROJECTS "lld") + endif() + # Any targets added here will be given the target name stage2-${target}, so + # if you want to run them you can just use: + # ninja -C $BUILDDIR stage2-${target} + set(CLANG_BOOTSTRAP_TARGETS ${LLVM_RELEASE_FINAL_STAGE_TARGETS} CACHE STRING "") endif() +# Stage 1 Common Config set(LLVM_ENABLE_RUNTIMES ${STAGE1_RUNTIMES} CACHE STRING "
[clang] [llvm] [cmake] Remove custom linker flag check function (PR #86602)
tstellar wrote: @petrhosek Would you be able to add a comment to the llvm_check_linker_flag function explaining this so someone else doesn't try to do the same thing. https://github.com/llvm/llvm-project/pull/86602 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [cmake] Remove custom linker flag check function (PR #86602)
https://github.com/tstellar approved this pull request. LGTM. https://github.com/llvm/llvm-project/pull/86602 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Add stage2-package target (PR #89517)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/89517 This target will be used to generate the release binary package for uploading to GitHub. >From 79e4f21df778ede9238c66e06ca289d7599f9f5c Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Mon, 15 Apr 2024 12:03:33 -0700 Subject: [PATCH] [CMake][Release] Add stage2-package target This target will be used to generate the release binary package for uploading to GitHub. --- clang/cmake/caches/Release.cmake | 2 ++ 1 file changed, 2 insertions(+) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index bd1f688d61a7ea..fa972636553f1f 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -14,6 +14,7 @@ if (LLVM_RELEASE_ENABLE_PGO) set(CLANG_BOOTSTRAP_TARGETS generate-profdata stage2 +stage2-package stage2-clang stage2-distribution stage2-install @@ -57,6 +58,7 @@ set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(BOOTSTRAP_CLANG_ENABLE_BOOTSTRAP ON CACHE STRING "") set(BOOTSTRAP_CLANG_BOOTSTRAP_TARGETS clang + package check-all check-llvm check-clang CACHE STRING "") ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [polly] [clang-format] Correctly annotate braces in macros (PR #87953)
tstellar wrote: @owenca Is there a PR for the cherry-pick ? https://github.com/llvm/llvm-project/pull/87953 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Disable PGO (PR #88465)
tstellar wrote: PR is here: https://github.com/llvm/llvm-project/pull/89000 https://github.com/llvm/llvm-project/pull/88465 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Sema] Mark alias/ifunc targets used and consider mangled names (PR #87130)
https://github.com/tstellar commented: I'm fine with this as long as @AaronBallman approves. https://github.com/llvm/llvm-project/pull/87130 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Disable PGO (PR #88465)
tstellar wrote: /cherry-pick 0d2bb7f017f13ceae793fab7d83d3e67e8d8d8f8) https://github.com/llvm/llvm-project/pull/88465 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Disable PGO (PR #88465)
https://github.com/tstellar milestoned https://github.com/llvm/llvm-project/pull/88465 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Disable PGO (PR #88465)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/88465 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Disable PGO (PR #88465)
tstellar wrote: > I'm assuming this is a stop-gap, and the plan is to eventually fix the issue > and reenable PGO in the release cache? Yes, I would like to fix this at some point, but not sure when I'll get to it. https://github.com/llvm/llvm-project/pull/88465 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [CMake][Release] Disable PGO (PR #88465)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/88465 Running the test-release.sh script with PGO enabled causes build errors like: ld.lld: error: Function Import: link error: linking module flags 'ProfileSummary': IDs have conflicting values I believe this a build system bug due to the PGO profile data being generated unconditionally. If you run `ninja check-all` and then `ninja install` like we do in test-release.sh, then the profile data is regenerated during `ninja install` and some of the clang tools which are not test dependencies get build during the ninja install step with different profile data. When these tools link against the LLVM libraries, like libSupport, we end up with these errors. >From 9ebb7efc47a76902fcce078eca4e371d3cf99b68 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Fri, 12 Apr 2024 01:10:45 + Subject: [PATCH] [CMake][Release] Disable PGO Running the test-release.sh script with PGO enabled causes build errors like: ld.lld: error: Function Import: link error: linking module flags 'ProfileSummary': IDs have conflicting values I believe this a build system bug due to the PGO profile data being generated unconditionally. If you run `ninja check-all` and then `ninja install` like we do in test-release.sh, then the profile data is regenerated during `ninja install` and some of the clang tools which are not test dependencies get build during the ninja install step with different profile data. When these tools link against the LLVM libraries, like libSupport, we end up with these errors. --- clang/cmake/caches/Release.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/cmake/caches/Release.cmake b/clang/cmake/caches/Release.cmake index 1ca9138b980731..bd1f688d61a7ea 100644 --- a/clang/cmake/caches/Release.cmake +++ b/clang/cmake/caches/Release.cmake @@ -4,7 +4,7 @@ # General Options set(LLVM_RELEASE_ENABLE_LTO THIN CACHE STRING "") -set(LLVM_RELEASE_ENABLE_PGO ON CACHE BOOL "") +set(LLVM_RELEASE_ENABLE_PGO OFF CACHE BOOL "") set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Sema] Mark alias/ifunc targets used and consider mangled names (PR #87130)
https://github.com/tstellar requested changes to this pull request. I'm going to NAK this change for now. Given the recent [developments](https://www.openwall.com/lists/oss-security/2024/03/29/4) with the xz project and considering this feature was requested by a user associated with the project, I think we need to be extra careful with this change. I would like to see at least two trusted code owners review and approve this change before it is committed. https://github.com/llvm/llvm-project/pull/87130 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[libclc] [libclc] Make CMake messages better fit into LLVM (PR #86945)
https://github.com/tstellar approved this pull request. https://github.com/llvm/llvm-project/pull/86945 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][CodeGen] Allow memcpy replace with trivial auto var init (PR #84230)
tstellar wrote: Do we need to backport this to release/18.x? https://github.com/llvm/llvm-project/pull/84230 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[libclc] [libclc] Convert tabs to spaces in CMake (PR #85634)
tstellar wrote: > FWIW I don't think the build bots are actually building `libclc`. > > Locally, at least, it's not sufficient to specify > `LLVM_ENABLE_PROJECTS=libclc` - you _also_ need > `LLVM_EXTERNAL_PROJECTS=libclc`. Even then, it doesn't build for me in-tree, > because it assumes `CMAKE_SOURCE_DIR` is its own top-level directory, which > it isn't as it's `llvm/`. > > I had a patch locally to switch `CMAKE_SOURCE_DIR` to `PROJECT_SOURCE_DIR` > which I think is sufficient to cover both build styles. Should I contribute > this upstream? > > Another problem is that all `libclc` targets are enabled by default, but the > `spirv-mesa3d` and `spirv64-mesa3d` targets both need `llvm-spirv` which I > doubt our buildbots have available. Should those be disabled by default, or > should the buildbots disable them? I'm not aware of any bots building libclc. It would be nice if you could build libclc more easily, it does not fit in with the rest of the projects right now. Any improvements you want to make in this area would be welcome. https://github.com/llvm/llvm-project/pull/85634 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[libclc] [libclc] Convert tabs to spaces in CMake (PR #85634)
https://github.com/tstellar approved this pull request. https://github.com/llvm/llvm-project/pull/85634 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] Add missing clang to the monolithic pre-merge build (PR #85354)
https://github.com/tstellar approved this pull request. Good catch. https://github.com/llvm/llvm-project/pull/85354 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang][NFC] Regroup declarations in `Sema` (PR #82217)
tstellar wrote: This will make backports more difficult, but I think it's OK. If we wanted to avoid impacting backports, we'd have to wait ~10 weeks to merge this, but that doesn't seem practical. https://github.com/llvm/llvm-project/pull/82217 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang] Add 'CLANG_ALLOW_IMPLICIT_RPATH' to enable toolchain use of -rpath (PR #82004)
tstellar wrote: Could we have the upstream default be to install a clang.cfg file in bin/ which has -fimplicit-openmp-rpath (or whatever the hoption is)? https://github.com/llvm/llvm-project/pull/82004 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang] Add 'CLANG_ALLOW_IMPLICIT_RPATH' to enable toolchain use of -rpath (PR #82004)
tstellar wrote: > > > I believe right now the `rtlib-rpath` points to the path > > > `${CLANG_BINARY}../lib/${HOST_TRIPLE}/`. I think it's definitely > > > reasonable to not put this on system libraries if that's a solution, > > > since we can generally assume it's a global installation and already > > > covered by the user's library path. Does the Fedora policy only disallow > > > `rpath` on system installs then? > > > > > > Right, any official Fedora binaries we build can't have rpath in them, but > > users can still use rpath in their own applications if they want. > > I'd be okay with that solution. The majority of the issues come from people > on cluster machines with several conflicting versions of the libraries. The > only question is how to determine what a system library is. We could possible > just hardcode a few paths depending on the host triple's operating system. I'm not a huge fan of trying to auto-detect settings based on the host OS. What if we just added an option like --exclude-rpath-dir= which would let a user or a distro specify a list of directories that aren't allowed to be added as rpath. Now that we have config file support it would be easy for distros to add a config file with the list of its own system directories. https://github.com/llvm/llvm-project/pull/82004 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang] Add 'CLANG_ALLOW_IMPLICIT_RPATH' to enable toolchain use of -rpath (PR #82004)
tstellar wrote: > > I believe right now the `rtlib-rpath` points to the path > `${CLANG_BINARY}../lib/${HOST_TRIPLE}/`. I think it's definitely reasonable > to not put this on system libraries if that's a solution, since we can > generally assume it's a global installation and already covered by the user's > library path. Does the Fedora policy only disallow `rpath` on system installs > then? Right, any official Fedora binaries we build can't have rpath in them, but users can still use rpath in their own applications if they want. https://github.com/llvm/llvm-project/pull/82004 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang] Add 'CLANG_ALLOW_IMPLICIT_RPATH' to enable toolchain use of -rpath (PR #82004)
tstellar wrote: > We could make this logic more complicated by checking the system > automatically `execute_command(lsb_release)` or something. I think this is too complicated and fragile. If we want to do something specifically to support the distro use case, I think we should do this: https://reviews.llvm.org/D142174 https://github.com/llvm/llvm-project/pull/82004 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang] Add 'CLANG_ALLOW_IMPLICIT_RPATH' to enable toolchain use of -rpath (PR #82004)
tstellar wrote: > Does Fedora only use default configurations when building its packages? We try to stick to the defaults as much as possible. Having an implicit rpath by default causes issues when building other RPMs with clang, and typically, if we need to change something in order to build RPMS, we do it by adjusting the global C Flags used for building RPMS rather than patching the compiler. https://github.com/llvm/llvm-project/pull/82004 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [Clang] Add 'CLANG_ALLOW_IMPLICIT_RPATH' to enable toolchain use of -rpath (PR #82004)
tstellar wrote: I don't think a CMake option is the right way to go here. For example, I doubt we would actually use this CMake option in Fedora if this patch is committed. Does gcc use rpath for libgomp? https://github.com/llvm/llvm-project/pull/82004 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][PGO] Build Sema.cpp to generate profdata for PGO builds (PR #77347)
@@ -26,9 +30,23 @@ if(LLVM_BUILD_INSTRUMENTED) message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to llvm-profdata") else() add_custom_target(generate-profdata - COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} + COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ COMMENT "Merging profdata" DEPENDS generate-profraw) +if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR) + llvm_ExternalProject_Add(generate-profraw-external ${CLANG_PGO_TRAINING_DATA_SOURCE_DIR} + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw) + add_dependencies(generate-profdata generate-profraw-external) +else() + # Default to compiling a file from clang. This also builds all the + # dependencies needed to build this file, like TableGen. + set(generate_profraw_clang_sema tools/clang/lib/Sema/CMakeFiles/obj.clangSema.dir/Sema.cpp.o) + llvm_ExternalProject_Add(generate-profraw-clang ${CMAKE_CURRENT_SOURCE_DIR}/../../../llvm + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw + EXTRA_TARGETS generate_profraw_clang_sema tstellar wrote: I was compiling just a single file (SemaChecking.cpp). https://github.com/llvm/llvm-project/pull/77347 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-format] Fix a regression in dumping the config (PR #80628)
tstellar wrote: Tracking the backport request here: #81096 https://github.com/llvm/llvm-project/pull/80628 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [Github] Use building LLVM as perf-training for CI container (PR #80713)
https://github.com/tstellar approved this pull request. https://github.com/llvm/llvm-project/pull/80713 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [Github] Use building LLVM as perf-training for CI container (PR #80713)
@@ -0,0 +1,13 @@ +diff --git a/clang/cmake/caches/BOLT-PGO.cmake b/clang/cmake/caches/BOLT-PGO.cmake +index 1a04ca9a74e5..d092820e4115 100644 +--- a/clang/cmake/caches/BOLT-PGO.cmake b/clang/cmake/caches/BOLT-PGO.cmake +@@ -4,6 +4,8 @@ set(CLANG_BOOTSTRAP_TARGETS + stage2-clang-bolt + stage2-distribution + stage2-install-distribution ++ clang ++ lld tstellar wrote: OK, thanks for the explanation. https://github.com/llvm/llvm-project/pull/80713 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [Github] Use building LLVM as perf-training for CI container (PR #80713)
@@ -0,0 +1,13 @@ +diff --git a/clang/cmake/caches/BOLT-PGO.cmake b/clang/cmake/caches/BOLT-PGO.cmake +index 1a04ca9a74e5..d092820e4115 100644 +--- a/clang/cmake/caches/BOLT-PGO.cmake b/clang/cmake/caches/BOLT-PGO.cmake +@@ -4,6 +4,8 @@ set(CLANG_BOOTSTRAP_TARGETS + stage2-clang-bolt + stage2-distribution + stage2-install-distribution ++ clang ++ lld tstellar wrote: Do you need this because of the way you have split the stages? If you were to to just run `ninja stage2-clang-bolt` from the start would it still be building a lot of unnecessary targets prior to generating the profile data? I'm mainly wondering if this is a change that would benefit everyone using the BOLT-PGO cache. https://github.com/llvm/llvm-project/pull/80713 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [Github] Use building LLVM as perf-training for CI container (PR #80713)
@@ -0,0 +1,13 @@ +diff --git a/clang/cmake/caches/BOLT-PGO.cmake b/clang/cmake/caches/BOLT-PGO.cmake +index 1a04ca9a74e5..d092820e4115 100644 +--- a/clang/cmake/caches/BOLT-PGO.cmake b/clang/cmake/caches/BOLT-PGO.cmake +@@ -4,6 +4,8 @@ set(CLANG_BOOTSTRAP_TARGETS + stage2-clang-bolt + stage2-distribution + stage2-install-distribution ++ clang ++ lld tstellar wrote: Why is this needed? https://github.com/llvm/llvm-project/pull/80713 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-tools-extra] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/78879 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-tools-extra] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
@@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "") +set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") tstellar wrote: @aaupov How does this look now? https://github.com/llvm/llvm-project/pull/78879 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [clang-tools-extra] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/78879 >From 0719f49ecc6dd69ae4698c3e84dbf175a1bf2ed3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sun, 21 Jan 2024 00:25:47 + Subject: [PATCH 1/4] [CMake][PGO] Add libunwind to list of stage1 runtimes This fixes the build since 8f90e6937a1fac80873bb2dab5f382c82ba1ba4e which made libcxxabi use llvm's libunwind by default. --- clang/cmake/caches/PGO.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/cmake/caches/PGO.cmake b/clang/cmake/caches/PGO.cmake index e1d0585e453f8..15bc755d110d1 100644 --- a/clang/cmake/caches/PGO.cmake +++ b/clang/cmake/caches/PGO.cmake @@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "") +set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED ON CACHE BOOL "") >From 5c602233ef4e54e850f6c8a17c25968bc706b898 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sat, 6 Jan 2024 07:46:01 + Subject: [PATCH 2/4] [CMake][PGO] Add option for using an external project to generate profile data. The new CLANG_PGO_TRAINING_DATA_SOURCE_DIR allows users to specify a CMake project to use for generating the profile data. For example, to use the llvm-test-suite to generate profile data you would do: $ cmake -G Ninja -B build -S llvm -C /clang/cmake/caches/PGO.cmake \ -DBOOTSTRAP_CLANG_PGO_TRAINING_DATA_SOURCE_DIR= \ -DBOOTSTRAP_CLANG_PERF_TRAINING_DEPS=runtimes --- clang/utils/perf-training/CMakeLists.txt | 13 +++-- clang/utils/perf-training/perf-helper.py | 16 +--- llvm/docs/AdvancedBuilds.rst | 23 +++ 3 files changed, 43 insertions(+), 9 deletions(-) diff --git a/clang/utils/perf-training/CMakeLists.txt b/clang/utils/perf-training/CMakeLists.txt index c6d51863fb1b5..f9d673b2e92e7 100644 --- a/clang/utils/perf-training/CMakeLists.txt +++ b/clang/utils/perf-training/CMakeLists.txt @@ -1,6 +1,10 @@ +include(LLVMExternalProjectUtils) + set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" CACHE PATH "The path to a lit testsuite containing samples for PGO and order file generation" ) +set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source directory containing cmake project with source files to use for generating pgo data") +set(CLANG_PERF_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to build the PGO training data.") if(LLVM_BUILD_INSTRUMENTED) configure_lit_site_cfg( @@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED) ) add_custom_target(clear-profraw -COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} profraw +COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw COMMENT "Clearing old profraw data") if(NOT LLVM_PROFDATA) @@ -26,9 +30,14 @@ if(LLVM_BUILD_INSTRUMENTED) message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to llvm-profdata") else() add_custom_target(generate-profdata - COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} + COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ COMMENT "Merging profdata" DEPENDS generate-profraw) +if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR) + llvm_ExternalProject_Add(generate-profraw-external ${CLANG_PGO_TRAINING_DATA_SOURCE_DIR} + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw) + add_dependencies(generate-profdata generate-profraw-external) +endif() endif() endif() diff --git a/clang/utils/perf-training/perf-helper.py b/clang/utils/perf-training/perf-helper.py index 99d6ab6ef..844aa274f049a 100644 --- a/clang/utils/perf-training/perf-helper.py +++ b/clang/utils/perf-training/perf-helper.py @@ -30,26 +30,28 @@ def findFilesWithExtension(path, extension): def clean(args): -if len(args) != 2: +if len(args) < 2: print( -"Usage: %s clean \n" % __file__ +"Usage: %s clean \n" % __file__ + "\tRemoves all files with extension from ." ) return 1 -for filename in findFilesWithExtension(args[0], args[1]): -os.remove(filename) +for path in args[1:-1]: +for filename in findFilesWithExtension(path, args[-1]): +os.remove(filename) return 0 def merge(args):
[clang] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
@@ -11,11 +15,11 @@ if(LLVM_BUILD_INSTRUMENTED) add_lit_testsuite(generate-profraw "Generating clang PGO data" ${CMAKE_CURRENT_BINARY_DIR}/pgo-data/ EXCLUDE_FROM_CHECK_ALL -DEPENDS clang clear-profraw ${CLANG_PERF_TRAINING_DEPS} +DEPENDS clang clear-profraw ${CLANG_PGO_TRAINING_DEPS} ) add_custom_target(clear-profraw -COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} profraw +COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw tstellar wrote: @aaupov That's the directory where clang will write it's profiles to by default. https://github.com/llvm/llvm-project/pull/78879 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [SystemZ][z/OS] This change adds support for the PPA2 section in zOS (PR #68926)
@@ -0,0 +1,31 @@ +; RUN: llc -mtriple s390x-ibm-zos -mcpu=z15 -asm-verbose=true < %s | FileCheck %s +; REQUIRES: systemz-registered-target + +; CHECK:.section".ppa2" +; CHECK: @@PPA2: +; CHECK:.byte 3 +; CHECK:.byte 231 +; CHECK:.byte 34 +; CHECK:.byte 4 +; CHECK:.long CELQSTRT-@@PPA2 +; CHECK:.long 0 +; CHECK:.long @@DVS-@@PPA2 +; CHECK:.long 0 +; CHECK:.byte 129 +; CHECK:.byte 0 +; CHECK:.short 0 +; CHECK: @@DVS: +; CHECK:.ascii "\361\371\367\360\360\361\360\361\360\360\360\360\360\360" +; CHECK:.short 0 +; CHECK:.quad @@PPA2-CELQSTRT * A(PPA2-CELQSTRT) +; CHECK: @@PPA1_void_test_0: +; CHECK:.long @@PPA2-@@PPA1_void_test_0 * Offset to PPA2 +; CHECK:.section"B_IDRL" +; CHECK:.byte 0 +; CHECK:.byte 3 +; CHECK:.short 30 +; CHECK:.ascii "\323\323\345\324@@\361\370\360\360\361\371\367\360\360\361\360\361\360\360\360\360\360\360\360\360" tstellar wrote: Hi, Based on 4236d56141be3a421d9e7fe8bb7f1ba1663ec97a, it appears that this test encodes the llvm version into the test. Is there some way to change this test so it does not container the llvm version in it. It's not practical to have to update this every time there is a release. https://github.com/llvm/llvm-project/pull/68926 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/78879 >From 0719f49ecc6dd69ae4698c3e84dbf175a1bf2ed3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sun, 21 Jan 2024 00:25:47 + Subject: [PATCH 1/4] [CMake][PGO] Add libunwind to list of stage1 runtimes This fixes the build since 8f90e6937a1fac80873bb2dab5f382c82ba1ba4e which made libcxxabi use llvm's libunwind by default. --- clang/cmake/caches/PGO.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/cmake/caches/PGO.cmake b/clang/cmake/caches/PGO.cmake index e1d0585e453f825..15bc755d110d19a 100644 --- a/clang/cmake/caches/PGO.cmake +++ b/clang/cmake/caches/PGO.cmake @@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "") +set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED ON CACHE BOOL "") >From 5c602233ef4e54e850f6c8a17c25968bc706b898 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sat, 6 Jan 2024 07:46:01 + Subject: [PATCH 2/4] [CMake][PGO] Add option for using an external project to generate profile data. The new CLANG_PGO_TRAINING_DATA_SOURCE_DIR allows users to specify a CMake project to use for generating the profile data. For example, to use the llvm-test-suite to generate profile data you would do: $ cmake -G Ninja -B build -S llvm -C /clang/cmake/caches/PGO.cmake \ -DBOOTSTRAP_CLANG_PGO_TRAINING_DATA_SOURCE_DIR= \ -DBOOTSTRAP_CLANG_PERF_TRAINING_DEPS=runtimes --- clang/utils/perf-training/CMakeLists.txt | 13 +++-- clang/utils/perf-training/perf-helper.py | 16 +--- llvm/docs/AdvancedBuilds.rst | 23 +++ 3 files changed, 43 insertions(+), 9 deletions(-) diff --git a/clang/utils/perf-training/CMakeLists.txt b/clang/utils/perf-training/CMakeLists.txt index c6d51863fb1b5c2..f9d673b2e92e775 100644 --- a/clang/utils/perf-training/CMakeLists.txt +++ b/clang/utils/perf-training/CMakeLists.txt @@ -1,6 +1,10 @@ +include(LLVMExternalProjectUtils) + set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" CACHE PATH "The path to a lit testsuite containing samples for PGO and order file generation" ) +set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source directory containing cmake project with source files to use for generating pgo data") +set(CLANG_PERF_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to build the PGO training data.") if(LLVM_BUILD_INSTRUMENTED) configure_lit_site_cfg( @@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED) ) add_custom_target(clear-profraw -COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} profraw +COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw COMMENT "Clearing old profraw data") if(NOT LLVM_PROFDATA) @@ -26,9 +30,14 @@ if(LLVM_BUILD_INSTRUMENTED) message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to llvm-profdata") else() add_custom_target(generate-profdata - COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} + COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ COMMENT "Merging profdata" DEPENDS generate-profraw) +if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR) + llvm_ExternalProject_Add(generate-profraw-external ${CLANG_PGO_TRAINING_DATA_SOURCE_DIR} + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw) + add_dependencies(generate-profdata generate-profraw-external) +endif() endif() endif() diff --git a/clang/utils/perf-training/perf-helper.py b/clang/utils/perf-training/perf-helper.py index 99d6ab6ef08..844aa274f049aaa 100644 --- a/clang/utils/perf-training/perf-helper.py +++ b/clang/utils/perf-training/perf-helper.py @@ -30,26 +30,28 @@ def findFilesWithExtension(path, extension): def clean(args): -if len(args) != 2: +if len(args) < 2: print( -"Usage: %s clean \n" % __file__ +"Usage: %s clean \n" % __file__ + "\tRemoves all files with extension from ." ) return 1 -for filename in findFilesWithExtension(args[0], args[1]): -os.remove(filename) +for path in args[1:-1]: +for filename in findFilesWithExtension(path, args[-1]): +os.remove(filename) return 0 def
[clang] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
https://github.com/tstellar edited https://github.com/llvm/llvm-project/pull/78879 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][Release] Add option for enabling PGO to release cache file. (PR #78823)
https://github.com/tstellar closed https://github.com/llvm/llvm-project/pull/78823 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[clang] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
https://github.com/tstellar updated https://github.com/llvm/llvm-project/pull/78879 >From 0719f49ecc6dd69ae4698c3e84dbf175a1bf2ed3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sun, 21 Jan 2024 00:25:47 + Subject: [PATCH 1/3] [CMake][PGO] Add libunwind to list of stage1 runtimes This fixes the build since 8f90e6937a1fac80873bb2dab5f382c82ba1ba4e which made libcxxabi use llvm's libunwind by default. --- clang/cmake/caches/PGO.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/cmake/caches/PGO.cmake b/clang/cmake/caches/PGO.cmake index e1d0585e453f825..15bc755d110d19a 100644 --- a/clang/cmake/caches/PGO.cmake +++ b/clang/cmake/caches/PGO.cmake @@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "") +set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED ON CACHE BOOL "") >From 5c602233ef4e54e850f6c8a17c25968bc706b898 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sat, 6 Jan 2024 07:46:01 + Subject: [PATCH 2/3] [CMake][PGO] Add option for using an external project to generate profile data. The new CLANG_PGO_TRAINING_DATA_SOURCE_DIR allows users to specify a CMake project to use for generating the profile data. For example, to use the llvm-test-suite to generate profile data you would do: $ cmake -G Ninja -B build -S llvm -C /clang/cmake/caches/PGO.cmake \ -DBOOTSTRAP_CLANG_PGO_TRAINING_DATA_SOURCE_DIR= \ -DBOOTSTRAP_CLANG_PERF_TRAINING_DEPS=runtimes --- clang/utils/perf-training/CMakeLists.txt | 13 +++-- clang/utils/perf-training/perf-helper.py | 16 +--- llvm/docs/AdvancedBuilds.rst | 23 +++ 3 files changed, 43 insertions(+), 9 deletions(-) diff --git a/clang/utils/perf-training/CMakeLists.txt b/clang/utils/perf-training/CMakeLists.txt index c6d51863fb1b5c2..f9d673b2e92e775 100644 --- a/clang/utils/perf-training/CMakeLists.txt +++ b/clang/utils/perf-training/CMakeLists.txt @@ -1,6 +1,10 @@ +include(LLVMExternalProjectUtils) + set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" CACHE PATH "The path to a lit testsuite containing samples for PGO and order file generation" ) +set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source directory containing cmake project with source files to use for generating pgo data") +set(CLANG_PERF_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to build the PGO training data.") if(LLVM_BUILD_INSTRUMENTED) configure_lit_site_cfg( @@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED) ) add_custom_target(clear-profraw -COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} profraw +COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw COMMENT "Clearing old profraw data") if(NOT LLVM_PROFDATA) @@ -26,9 +30,14 @@ if(LLVM_BUILD_INSTRUMENTED) message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to llvm-profdata") else() add_custom_target(generate-profdata - COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} + COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ COMMENT "Merging profdata" DEPENDS generate-profraw) +if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR) + llvm_ExternalProject_Add(generate-profraw-external ${CLANG_PGO_TRAINING_DATA_SOURCE_DIR} + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw) + add_dependencies(generate-profdata generate-profraw-external) +endif() endif() endif() diff --git a/clang/utils/perf-training/perf-helper.py b/clang/utils/perf-training/perf-helper.py index 99d6ab6ef08..844aa274f049aaa 100644 --- a/clang/utils/perf-training/perf-helper.py +++ b/clang/utils/perf-training/perf-helper.py @@ -30,26 +30,28 @@ def findFilesWithExtension(path, extension): def clean(args): -if len(args) != 2: +if len(args) < 2: print( -"Usage: %s clean \n" % __file__ +"Usage: %s clean \n" % __file__ + "\tRemoves all files with extension from ." ) return 1 -for filename in findFilesWithExtension(args[0], args[1]): -os.remove(filename) +for path in args[1:-1]: +for filename in findFilesWithExtension(path, args[-1]): +os.remove(filename) return 0 def
[clang] [llvm] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
@@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED) ) tstellar wrote: @petrhosek CLANG_PERF_TRAINING_DEPS was an existing variable that's used here. I think it was meant to be added to cache files, but I added it to CMakeLists.txt file here so that it could be passed on the command line without modifying the cache files. https://github.com/llvm/llvm-project/pull/78879 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [CMake][PGO] Build Sema.cpp to generate profdata for PGO builds (PR #77347)
tstellar wrote: I split the new configuration option out into a separate PR: #78879, because it seems useful regardless of what the default is. We can continue to discuss the best default on this PR. https://github.com/llvm/llvm-project/pull/77347 ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
[llvm] [clang] [CMake][PGO] Add option for using an external project to generate profile data (PR #78879)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/78879 The new CLANG_PGO_TRAINING_DATA_SOURCE_DIR allows users to specify a CMake project to use for generating the profile data. For example, to use the llvm-test-suite to generate profile data you would do: $ cmake -G Ninja -B build -S llvm -C /clang/cmake/caches/PGO.cmake \ -DBOOTSTRAP_CLANG_PGO_TRAINING_DATA_SOURCE_DIR= \ -DBOOTSTRAP_CLANG_PERF_TRAINING_DEPS=runtimes >From 0719f49ecc6dd69ae4698c3e84dbf175a1bf2ed3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sun, 21 Jan 2024 00:25:47 + Subject: [PATCH 1/2] [CMake][PGO] Add libunwind to list of stage1 runtimes This fixes the build since 8f90e6937a1fac80873bb2dab5f382c82ba1ba4e which made libcxxabi use llvm's libunwind by default. --- clang/cmake/caches/PGO.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/cmake/caches/PGO.cmake b/clang/cmake/caches/PGO.cmake index e1d0585e453f825..15bc755d110d19a 100644 --- a/clang/cmake/caches/PGO.cmake +++ b/clang/cmake/caches/PGO.cmake @@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "") +set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED ON CACHE BOOL "") >From 5c602233ef4e54e850f6c8a17c25968bc706b898 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sat, 6 Jan 2024 07:46:01 + Subject: [PATCH 2/2] [CMake][PGO] Add option for using an external project to generate profile data. The new CLANG_PGO_TRAINING_DATA_SOURCE_DIR allows users to specify a CMake project to use for generating the profile data. For example, to use the llvm-test-suite to generate profile data you would do: $ cmake -G Ninja -B build -S llvm -C /clang/cmake/caches/PGO.cmake \ -DBOOTSTRAP_CLANG_PGO_TRAINING_DATA_SOURCE_DIR= \ -DBOOTSTRAP_CLANG_PERF_TRAINING_DEPS=runtimes --- clang/utils/perf-training/CMakeLists.txt | 13 +++-- clang/utils/perf-training/perf-helper.py | 16 +--- llvm/docs/AdvancedBuilds.rst | 23 +++ 3 files changed, 43 insertions(+), 9 deletions(-) diff --git a/clang/utils/perf-training/CMakeLists.txt b/clang/utils/perf-training/CMakeLists.txt index c6d51863fb1b5c2..f9d673b2e92e775 100644 --- a/clang/utils/perf-training/CMakeLists.txt +++ b/clang/utils/perf-training/CMakeLists.txt @@ -1,6 +1,10 @@ +include(LLVMExternalProjectUtils) + set(CLANG_PGO_TRAINING_DATA "${CMAKE_CURRENT_SOURCE_DIR}" CACHE PATH "The path to a lit testsuite containing samples for PGO and order file generation" ) +set(CLANG_PGO_TRAINING_DATA_SOURCE_DIR OFF CACHE STRING "Path to source directory containing cmake project with source files to use for generating pgo data") +set(CLANG_PERF_TRAINING_DEPS "" CACHE STRING "Extra dependencies needed to build the PGO training data.") if(LLVM_BUILD_INSTRUMENTED) configure_lit_site_cfg( @@ -15,7 +19,7 @@ if(LLVM_BUILD_INSTRUMENTED) ) add_custom_target(clear-profraw -COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} profraw +COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py clean ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ profraw COMMENT "Clearing old profraw data") if(NOT LLVM_PROFDATA) @@ -26,9 +30,14 @@ if(LLVM_BUILD_INSTRUMENTED) message(STATUS "To enable merging PGO data LLVM_PROFDATA has to point to llvm-profdata") else() add_custom_target(generate-profdata - COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} + COMMAND "${Python3_EXECUTABLE}" ${CMAKE_CURRENT_SOURCE_DIR}/perf-helper.py merge ${LLVM_PROFDATA} ${CMAKE_CURRENT_BINARY_DIR}/clang.profdata ${CMAKE_CURRENT_BINARY_DIR} ${CMAKE_BINARY_DIR}/profiles/ COMMENT "Merging profdata" DEPENDS generate-profraw) +if (CLANG_PGO_TRAINING_DATA_SOURCE_DIR) + llvm_ExternalProject_Add(generate-profraw-external ${CLANG_PGO_TRAINING_DATA_SOURCE_DIR} + USE_TOOLCHAIN EXLUDE_FROM_ALL NO_INSTALL DEPENDS generate-profraw) + add_dependencies(generate-profdata generate-profraw-external) +endif() endif() endif() diff --git a/clang/utils/perf-training/perf-helper.py b/clang/utils/perf-training/perf-helper.py index 99d6ab6ef08..844aa274f049aaa 100644 --- a/clang/utils/perf-training/perf-helper.py +++ b/clang/utils/perf-training/perf-helper.py @@ -30,26 +30,28 @@ def findFilesWithExtension(path, extension): def clean(args): -if len(args) != 2: +if len(args) < 2: print( -"Usage: %s clean
[clang] [CMake][PGO] Add libunwind to list of stage1 runtimes (PR #78869)
https://github.com/tstellar created https://github.com/llvm/llvm-project/pull/78869 This fixes the build since 8f90e6937a1fac80873bb2dab5f382c82ba1ba4e which made libcxxabi use llvm's libunwind by default. >From 0719f49ecc6dd69ae4698c3e84dbf175a1bf2ed3 Mon Sep 17 00:00:00 2001 From: Tom Stellard Date: Sun, 21 Jan 2024 00:25:47 + Subject: [PATCH] [CMake][PGO] Add libunwind to list of stage1 runtimes This fixes the build since 8f90e6937a1fac80873bb2dab5f382c82ba1ba4e which made libcxxabi use llvm's libunwind by default. --- clang/cmake/caches/PGO.cmake | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/clang/cmake/caches/PGO.cmake b/clang/cmake/caches/PGO.cmake index e1d0585e453f82..15bc755d110d19 100644 --- a/clang/cmake/caches/PGO.cmake +++ b/clang/cmake/caches/PGO.cmake @@ -2,7 +2,7 @@ set(CMAKE_BUILD_TYPE RELEASE CACHE STRING "") set(CLANG_ENABLE_BOOTSTRAP ON CACHE BOOL "") set(LLVM_ENABLE_PROJECTS "clang;lld" CACHE STRING "") -set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi" CACHE STRING "") +set(LLVM_ENABLE_RUNTIMES "compiler-rt;libcxx;libcxxabi;libunwind" CACHE STRING "") set(LLVM_TARGETS_TO_BUILD Native CACHE STRING "") set(BOOTSTRAP_LLVM_BUILD_INSTRUMENTED ON CACHE BOOL "") ___ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits