From: Deepesh Varatharajan <[email protected]> This patch series introduces a test framework to run selftests for target builds of Clang, LLVM, and LLD.
v6 changes: - Removed the workaround for the llvm-confid to fix Rust build failure as it is no longer required. https://git.openembedded.org/openembedded-core/commit/?id=8aace29b692a8f3902a675fb4c022c223ba49531 - Removed CLANG_ENABLE_TESTSUITE = "1" from the meta-selftest layer, as it caused the check-layer script to fail. Test suite enabling is now handled directly within the test suite script. - x86-32 bit testing for llvm-module is no longer broken. v5 changes: - Fix intermittent failure seen for arm64. v4 changes: - Fix intermittent failures. v3 changes: - llvm_sysroot_preprocess_testsuite function needs be added to llvm:class-target not llvm:class-nativesdk. v2 changes: - Enable required CMake flags only when running the testsuite. - Ensure default build time and package sizes remain unaffected. Summary of changes: -Patch 1: Disable clang-tools-extra tests, as Clang's CMake cannot detect the llvm-bcanalyzer target. -Patch 2: Enabled the necessary CMake flags for clang, llvm & lld tests when CLANG_ENABLE_TESTSUITE is set to "1". -Patch 3: Selftest script for Clang, LLVM, and LLD including exclude list Testing: 1. The test suite has been successfully validated on x86-64, x86, arm64, riscv64 and arm32 for all Clang, LLD & LLVM components. 2. Clang, LLVM & LLD all together ~80K tests are there and we currently ~250 tests to exclude list which are failing (0.32%). Impact on build time: These builds were executed in a controlled environment without poky-sstate enabled. Default(without changes): +-----------+------------+------------+------------+----------------+ | Target | Run 1 | Run 2 | Run 3 | Average | | --------- | ---------- | ---------- | ---------- | -------------- | | llvm | 22m18.441s | 22m31.905s | 22m23.718s | 22m24.688s | | clang | 30m09.774s | 30m22.680s | 30m15.312s | 30m15.922s | | lld | 1m26.994s | 1m30.415s | 1m29.006s | 1m28.805s | +-----------+------------+------------+------------+----------------+ With the testsuite changes applied: +-----------+------------+------------+------------+----------------+ | Target | Run 1 | Run 2 | Run 3 | Average | | --------- | ---------- | ---------- | ---------- | -------------- | | llvm | 23m32.104s | 23m45.887s | 23m38.157s | 23m38.716s | | clang | 30m31.402s | 30m46.198s | 30m39.445s | 30m39.015s | | lld | 1m29.845s | 1m34.212s | 1m31.925s | 1m31.994s | +-----------+------------+------------+------------+----------------+ Comparison: The additional overhead is minimal and primarily due to enabling test-related build configurations. +-----------+-------------+------------------+------------------+ | Target | Default Avg | With Changes Avg | **Diff** | | --------- | ----------- | ---------------- | ---------------- | | llvm | 22m24.688s | 23m38.716s | +1m14.028s | | clang | 30m15.922s | 30m39.015s | +0m23.093s | | lld | 1m28.805s | 1m31.994s | +0m3.189s | +-----------+-------------+------------------+------------------+ Impact on package size and Fix: The testsuite changes has a negative impact on certain package sizes, as below: +---------------------+----------+----------+-----------+------------+ | Package | Old (MB) | New (MB) | Diff (MB) | Change (%) | +=====================+==========+==========+===========+============+ | clang/clang-dbg | 44.99 | 59.79 | +14.81 | +32.92% | | clang/clang-dev | 30.22 | 30.22 | -0.00 | -0.00% | | clang/clang-tools | 5.05 | 29.58 | +24.53 | +485.87% | | llvm/llvm-bin | 16.40 | 71.77 | +55.36 | +337.50% | | llvm/llvm-dbg | 386.79 | 646.12 | +259.33 | +67.05% | | llvm/llvm-dev | 32.67 | 33.67 | +1.00 | +3.05% | | llvm/llvm-src | 387.54 | 389.44 | +1.90 | +0.49% | | llvm/llvm-staticdev | 609.68 | 616.80 | +7.12 | +1.17% | +---------------------+----------+----------+-----------+------------+ FIX: All testsuite-related changes are now gated by the CLANG_ENABLE_TESTSUITE variable (default "0"), ensuring: - Test-specific CMake flags are applied only during test execution. - Standard builds remain fully unaffected in both build time and package size. Deepesh Varatharajan (3): clang-tools-extra: disable tests clang: Enable cmake flags for llvm, clang, lld tests oeqa/selftest/clang: Add oe-selftests for Clang/LLVM/LLD test suites meta/lib/oeqa/selftest/cases/clang.py | 330 ++++++++++++++++++ .../0042-guard-clang-tools-extra-test.patch | 74 ++++ meta/recipes-devtools/clang/clang_git.bb | 8 + meta/recipes-devtools/clang/common.inc | 1 + meta/recipes-devtools/clang/lld_git.bb | 5 + meta/recipes-devtools/clang/llvm_git.bb | 18 +- 6 files changed, 435 insertions(+), 1 deletion(-) create mode 100644 meta/lib/oeqa/selftest/cases/clang.py create mode 100644 meta/recipes-devtools/clang/clang/0042-guard-clang-tools-extra-test.patch -- 2.49.0
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#240938): https://lists.openembedded.org/g/openembedded-core/message/240938 Mute This Topic: https://lists.openembedded.org/mt/120277562/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
