lux-QAQ wrote: I ran performance tests by following the LLVM Test Suite Guide and obtained the following results.
The `results_old` data represents the baseline performance using the **original, unmodified Clang**. The `results_new` data shows the performance using the **Clang with my modifications applied**. [results_new.json](https://github.com/user-attachments/files/20655379/results_new.json) [results_old.json](https://github.com/user-attachments/files/20655389/results_old.json) ``` shell (base) ***@DESKTOP-80T55HI:~/code/llvmleak$ llvm-test-suite/utils/compare.py -m compile_time results_old.json results_new.json Tests: 3327 Metric: compile_time Program compile_time results_old results_new diff SingleSour...tTests/2020-01-06-coverage-008 0.00 0.02 484.6% SingleSour...tTests/2010-05-24-BitfieldTest 0.00 0.01 265.9% SingleSour.../2006-01-29-SimpleIndirectCall 0.00 0.01 238.9% SingleSour...006-12-07-Compare64BitConstant 0.00 0.01 186.8% SingleSour...itTests/2003-07-06-IntOverflow 0.01 0.02 159.7% SingleSour...UnitTests/2002-05-02-CastTest1 0.00 0.01 117.9% tools/test/check_env 0.02 0.05 107.9% tools/test/ret0 0.02 0.05 107.9% tools/test/abrt 0.02 0.05 107.9% tools/test/ret1 0.02 0.05 107.9% tools/fpcmp-target 0.02 0.05 107.9% SingleSour...UnitTests/2006-01-23-UnionInit 0.01 0.02 88.5% SingleSour.../UnitTests/block-call-r7674133 0.01 0.01 76.2% SingleSour...UnitTests/2005-05-12-Int64ToFP 0.01 0.02 64.2% SingleSour.../UnitTests/SignlessTypes/cast2 0.01 0.02 62.4% Geomean difference -5.4% compile_time run results_old results_new diff count 3327.000000 3327.000000 454.000000 mean 0.197357 0.184994 -0.005927 std 2.103227 1.847044 0.393625 min 0.000000 0.000000 -0.797814 25% 0.000000 0.000000 -0.139927 50% 0.000000 0.000000 -0.031750 75% 0.000000 0.000000 0.055100 max 92.449400 75.736300 4.846154 (base) ***@DESKTOP-80T55HI:~/code/llvmleak$ llvm-test-suite/utils/compare.py --filter-short results_old.json results_new.json Tests: 3327 Short Running: 2390 (filtered out) Remaining: 937 Metric: exec_time Program exec_time results_old results_new SingleSour...ut-C++/Shootout-C++-sieve.test 0.00 1.05 SingleSour...marks/CoyoteBench/lpbench.test 0.00 1.29 MultiSourc...lt/CrossingThresholds-flt.test 0.00 1.21 SingleSour...ce/Benchmarks/Misc/perlin.test 0.00 1.20 MultiSourc...CI_Purple/SMG2000/smg2000.test 0.00 1.23 MicroBench...HMARK_ANISTROPIC_DIFFUSION/256 20620.89 48887.01 MicroBench..._MemCmp<31, LessThanZero, Mid> 294.71 677.21 MicroBench...HMARK_ANISTROPIC_DIFFUSION/128 6420.34 13718.01 MicroBench...MemCmp<5, LessThanZero, First> 501.73 1004.02 MicroBench...test:benchAutoVecForBigLoopTC8 1.44 2.88 MicroBench...test:benchAutoVecForBigLoopTC2 1.51 3.01 MicroBench...t:BENCHMARK_asinf_novec_float_ 109.42 217.25 MicroBench...rks.test:benchForIC4VW4LoopTC4 2.04 4.01 MicroBench...BENCHMARK_ORDERED_DITHER/128/2 37.37 72.65 MicroBench...BENCHMARK_acos_autovec_double_ 158.15 302.43 exec_time run results_old results_new count 937.000000 937.000000 mean 2723.168099 2608.870807 std 30514.292833 28486.676194 min 0.000000 0.000000 25% 2.978695 3.087209 50% 17.409600 17.785316 75% 281.905940 283.165699 max 666889.746452 622151.000000 ``` ### **Performance Test Results** Following the LLVM Test Suite Guide, I conducted performance tests to evaluate the impact of my changes. The results are encouraging. All tests passed successfully for both the baseline and modified versions. Furthermore, the preliminary data indicates a **compile-time reduction of 5.4%** with the modified Clang compared to the original. ### **Caveats and Future Work** However, I want to be cautious with these initial findings. It is important to note that this is based on a single test run, and I suspect this performance difference could be attributed to random chance or environmental factors. I am currently busy with my final exams, but as soon as I have time, I plan to conduct more rigorous testing. My plan is to repeat the experiments on my WSL setup and on a separate Mac to obtain more robust and reliable results. ### **Test Environment & Configuration** To ensure the test was reproducible, the following configuration was used. **Host Compiler:** The same host compiler (installed via `apt`) was used to build both Clang versions: ``` $ /usr/bin/clang --version Ubuntu clang version 18.1.3 (1ubuntu1) Target: x86_64-pc-linux-gnu Thread model: posix InstalledDir: /usr/bin ``` **Build Configuration:** The following environment and CMake command were used to build both the baseline (original) and the modified Clang, ensuring a consistent build configuration between the two. ```shell # Environment variables were set to ensure optimization unset CFLAGS unset CXXFLAGS unset LDFLAGS unset LD_LIBRARY_PATH unset CC unset CXX export CFLAGS="-O3 -march=native" export CXXFLAGS="-O3 -march=native" # CMake command cmake -G "Ninja" \ -S /home/***/llvmtestfix/llvm \ -B /home/***/llvmtestfix/build \ -DCMAKE_INSTALL_PREFIX=/home/***/fixtestinstall \ -DCMAKE_INSTALL_RPATH=/home/***/fixtestinstall/lib \ -DLLVM_ENABLE_PROJECTS="clang;lld" \ -DLLVM_ENABLE_RUNTIMES="compiler-rt;libcxx;libcxxabi;libunwind;openmp" \ -DCMAKE_C_COMPILER=/usr/bin/clang \ -DCMAKE_CXX_COMPILER=/usr/bin/clang++ \ -DCMAKE_CXX_COMPILER_TARGET=x86_64-pc-linux-gnu \ -DCMAKE_C_COMPILER_TARGET=x86_64-pc-linux-gnu \ -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-pc-linux-gnu \ -DCMAKE_LINKER=/usr/bin/ld.lld \ -DLLVM_ENABLE_LLD=ON \ -DLLVM_PROFILE_GENERATE=OFF \ -DLLVM_INCLUDE_DOCS=OFF \ -DLLVM_BUILD_DOCS=OFF \ -DLIBCXX_INSTALL_MODULES=ON \ -DCMAKE_AR=/usr/bin/llvm-ar \ -DCMAKE_RANLIB=/usr/bin/llvm-ranlib \ -DCMAKE_BUILD_TYPE=Release \ -DLLVM_BUILD_LLVM_DYLIB=ON \ -DLLVM_LINK_LLVM_DYLIB=ON \ -DLLVM_ENABLE_RTTI=ON \ -DLLVM_ENABLE_EH=ON \ -DLLVM_ENABLE_EXCEPTIONS=ON \ -DLLVM_TARGETS_TO_BUILD="X86" ``` https://github.com/llvm/llvm-project/pull/143142 _______________________________________________ cfe-commits mailing list cfe-commits@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
