[gem5-dev] Change in gem5/gem5[develop]: util,tests: Added compiler test script
Bobby R. Bruce has submitted this change. (
https://gem5-review.googlesource.com/c/public/gem5/+/30395 )
Change subject: util,tests: Added compiler test script
..
util,tests: Added compiler test script
This script runs a series of compilations on gem5. The following
compilers are tested:
clang-9
clang-8
clang-7
clang-6
clang-5
clang-4
clang-3.9
gcc-9
gcc-8
gcc-7
gcc-6
gcc-5
gcc-4.8 (to be dropped soon:
https://gem5.atlassian.net/browse/GEM5-218)
They are tested by building the following build targets:
ARM
ARM_MESI_Three_Level
Garnet_standalone
GCN3_X86
MIPS
NULL_MESI_Two_Level
NULL_MOESI_CMP_directory
NULL_MOESI_CMP_token
NULL_MOESI_hammer
POWER
RISCV
SPARC
X86
X86_MOESI_AMD_BASE
For each, ".opt" and ".fast" compiler build settings are tested.
clang-9 and gcc-9 are tested against all targets with each build
setting. For the remaining compilers, a random build target is
chosen. After the script has run, the output of the tests can be
found in "compile-test-out".
Docker is required to run this script.
Change-Id: Id3bf4c89b9d424c87e9409930ee2aceaef72cb29
Reviewed-on: https://gem5-review.googlesource.com/c/public/gem5/+/30395
Reviewed-by: Jason Lowe-Power
Reviewed-by: Daniel Carvalho
Maintainer: Jason Lowe-Power
Tested-by: kokoro
---
A util/compiler-tests.sh
1 file changed, 132 insertions(+), 0 deletions(-)
Approvals:
Jason Lowe-Power: Looks good to me, approved; Looks good to me, approved
Daniel Carvalho: Looks good to me, approved
kokoro: Regressions pass
diff --git a/util/compiler-tests.sh b/util/compiler-tests.sh
new file mode 100755
index 000..01db818
--- /dev/null
+++ b/util/compiler-tests.sh
@@ -0,0 +1,132 @@
+#!/bin/bash
+
+# This script will run all our supported compilers (see the "images" set)
+# against gem5. The "ubuntu-20.04_all-dependencies" and "clang-version-9"
+# images are run against all built targets. The remainder are evaluated
+# against a random shuffling of built targets.
+
+dir="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+gem5_root="${dir}/.."
+build_dir="${gem5_root}/build"
+num_cores=`nproc`
+
+# All Docker images in the gem5 testing GCR which we want to compile with.
+images=("ubuntu-20.04_all-dependencies" # Uses GCC-9
+"clang-version-9"
+"gcc-version-8"
+"gcc-version-7"
+"gcc-version-6"
+"gcc-version-5"
+"gcc-version-4.8"
+"clang-version-8"
+"clang-version-7"
+"clang-version-6.0"
+"clang-version-5.0"
+"clang-version-4.0"
+"clang-version-3.9" )
+
+# A subset of the above list: these images will build against every target,
+# ignoring builds_per_compiler.
+comprehensive=("ubuntu-20.04_all-dependencies"
+ "clang-version-9")
+
+# All build targets in build_opt/ which we want to build using each image.
+builds=("ARM"
+"ARM_MESI_Three_Level"
+"Garnet_standalone"
+"GCN3_X86"
+"MIPS"
+"NULL_MESI_Two_Level"
+"NULL_MOESI_CMP_directory"
+"NULL_MOESI_CMP_token"
+"NULL_MOESI_hammer"
+"POWER"
+"RISCV"
+"SPARC"
+"X86"
+"X86_MOESI_AMD_Base")
+
+# The optimizations to use for each build target.
+opts=(".opt"
+ ".fast")
+
+# The number of build targets to randomly pull from the build target list
for
+# each compiler. To perform a full comprehensive test which covers every
+# possible pair of compiler and build target, set builds_per_compiler
equal to
+# the expression ${#builds[@]}.
+builds_per_compiler=1
+
+# Base URL of the gem5 testing images.
+base_url="gcr.io/gem5-test"
+
+# Arguments passed into scons on every build target test.
+build_args="-j ${num_cores}"
+
+# Testing directory variables
+mkdir -p "${build_dir}" # Create the build directory if it doesn't exist.
+test_dir_final="${gem5_root}/compile-test-out"
+test_dir="${gem5_root}/.compile-test-out"
+exits="${test_dir}/exit-codes.csv"
+
+# Create the testing output directory and files
+rm -rf $test_dir_final
+rm -rf $test_dir
+mkdir $test_dir
+touch $exits
+echo "compiler,build_target,exit_code" >> $exits
+
+for compiler in ${images[@]}; do
+echo "Starting build tests with '$compiler'..."
+# Generate a randomized list of build targets
+build_permutation=($(shuf -i 0-$((${#builds[@]} - 1)) ))
+
+builds_count=$builds_per_compiler
+if [[ " ${comprehensive[@]} " =~ " $compiler " ]]; then
+echo "'$compiler' was found in the comprehensive tests. All ISAs
will be built."
+builds_count=${#builds[@]}
+fi
+
+# Slice the first $builds_count entries of the permutation to get our
+# targets for this test
+build_indices=(${build_permutation[@]:0:$builds_count})
+
+repo_name="${base_url}/${compiler}:latest"
+
+# Grab compiler image
+docker pull $repo_name >/dev/null
+
+mkdir "$test_dir/$compiler"
+
+for build_index in
[gem5-dev] Change in gem5/gem5[develop]: util,tests: Added compiler test script
Bobby R. Bruce has uploaded this change for review. (
https://gem5-review.googlesource.com/c/public/gem5/+/30395 )
Change subject: util,tests: Added compiler test script
..
util,tests: Added compiler test script
This script runs a series of compilations on gem5. The following
compilers are tested:
clang-9
clang-8
clang-7
clang-6
clang-5
clang-4
clang-3.9
gcc-9
gcc-8
gcc-7
gcc-6
gcc-5
gcc-4.8 (to be dropped soon:
https://gem5.atlassian.net/browse/GEM5-218)
They are tested by building the following build targets:
ARM
ARM_MESI_Three_Level
Garnet_standalone
GCN3_X86
HSAIL_X86
MIPS
NULL_MESI_Two_Level
NULL_MOESI_CMP_directory
NULL_MOESI_CMP_token
NULL_MOESI_hammer
POWER
RISCV
SPARC
X86
X86_MOESI_AMD_BASE
For each, ".opt" and ".fast" compiler build settings are tested.
clang-9 and gcc-9 are tested against all targets with each build
setting. For the remaining compilers, a random build target is
chosen. After the script has run, the output of the tests can be
found in "compile-test-out".
Docker is required to run this script.
Change-Id: Id3bf4c89b9d424c87e9409930ee2aceaef72cb29
---
A util/compiler-tests.sh
1 file changed, 131 insertions(+), 0 deletions(-)
diff --git a/util/compiler-tests.sh b/util/compiler-tests.sh
new file mode 100755
index 000..06d4f96
--- /dev/null
+++ b/util/compiler-tests.sh
@@ -0,0 +1,131 @@
+#!/bin/bash
+
+# This script will run all our supported compilers (see the "images" set)
+# against gem5. The "ubuntu-20.04_all-dependencies" and "clang-version-9"
+# images are run against all built targets. The remainder are evaluated
+# against a random shuffling of built targets.
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+GEM5_ROOT="${DIR}/.."
+NUM_CORES=`nproc`
+
+# All Docker images in the gem5 testing GCR which we want to compile with.
+images=("ubuntu-20.04_all-dependencies" # Uses GCC-9
+"clang-version-9"
+"gcc-version-8"
+"gcc-version-7"
+"gcc-version-6"
+"gcc-version-5"
+"gcc-version-4.8"
+"clang-version-8"
+"clang-version-7"
+"clang-version-6.0"
+"clang-version-5.0"
+"clang-version-4.0"
+"clang-version-3.9" )
+
+# A subset of the above list: these images will build against every target,
+# ignoring buildsPerCompiler.
+comprehensive=("ubuntu-20.04_all-dependencies"
+ "clang-version-9")
+
+# All build targets in build_opt/ which we want to build using each image.
+builds=("ARM"
+"ARM_MESI_Three_Level"
+"Garnet_standalone"
+"GCN3_X86"
+"HSAIL_X86"
+"MIPS"
+"NULL_MESI_Two_Level"
+"NULL_MOESI_CMP_directory"
+"NULL_MOESI_CMP_token"
+"NULL_MOESI_hammer"
+"POWER"
+"RISCV"
+"SPARC"
+"X86"
+"X86_MOESI_AMD_Base")
+
+# The optimizations to use for each build target.
+opts=(".opt"
+ ".fast")
+
+# The number of build targets to randomly pull from the build target list
for
+# each compiler. To perform a full comprehensive test which covers every
+# possible pair of compiler and build target, set buildsPerCompiler equal
to
+# the expression ${#builds[@]}.
+buildsPerCompiler=1
+
+# Base URL of the gem5 testing images.
+base_url="gcr.io/gem5-test"
+
+# Arguments passed into scons on every build target test.
+build_args="-j ${NUM_CORES}"
+
+# Testing directory variables
+test_dir_final="${GEM5_ROOT}/compile-test-out"
+test_dir="${GEM5_ROOT}/.compile-test-out"
+exits="${test_dir}/exit-codes.csv"
+
+# Create the testing output directory and files
+rm -rf $test_dir_final
+rm -rf $test_dir
+mkdir $test_dir
+touch $exits
+echo "compiler,build_target,exit_code" >> $exits
+
+for compiler in ${images[@]}; do
+echo "Starting build tests with '$compiler'..."
+# Generate a randomized list of build targets
+build_permutation=($(shuf -i 0-$((${#builds[@]} - 1)) ))
+
+buildsCount=$buildsPerCompiler
+if [[ " ${comprehensive[@]} " =~ " $compiler " ]]; then
+echo "'$compiler' was found in the comprehensive tests. All ISAs
will be built."
+buildsCount=${#builds[@]}
+fi
+
+# Slice the first $buildsCount entries of the permutation to get our
+# targets for this test
+build_indices=(${build_permutation[@]:0:$buildsCount})
+
+repo_name="$base_url/$compiler"
+
+# Grab compiler image
+docker pull $repo_name >/dev/null
+
+mkdir "$test_dir/$compiler"
+
+# Clean any builds, if present
+rm -rf "${GEM5_ROOT}/build"
+
+for build_index in ${build_indices[@]}; do
+for build_opt in ${opts[@]}; do
+build="${builds[$build_index]}"
+build_dir="build/$build/gem5$build_opt"
+build_stdout="$test_dir/$compiler/$build$build_opt.stdout"
+build_stderr="$test_dir/$compiler/$build$build_opt.stderr"
+
+touch $build_stdout
+touch
