Re: [PATCH v1 2/5] drm/ci: generate testlist from build
Hi Dmitry,
On 30/04/24 15:47, Dmitry Baryshkov wrote:
On Tue, Apr 30, 2024 at 02:41:18PM +0530, Vignesh Raman wrote:
Stop vendoring the testlist into the kernel. Instead, use the
testlist from the IGT build to ensure we do not miss renamed
or newly added tests.
Signed-off-by: Vignesh Raman
---
drivers/gpu/drm/ci/build-igt.sh | 23 +
drivers/gpu/drm/ci/igt_runner.sh |9 +-
drivers/gpu/drm/ci/testlist.txt | 2761 --
3 files changed, 28 insertions(+), 2765 deletions(-)
delete mode 100644 drivers/gpu/drm/ci/testlist.txt
diff --git a/drivers/gpu/drm/ci/build-igt.sh b/drivers/gpu/drm/ci/build-igt.sh
index 500fa4f5c30a..cedc62baba1e 100644
--- a/drivers/gpu/drm/ci/build-igt.sh
+++ b/drivers/gpu/drm/ci/build-igt.sh
@@ -26,6 +26,29 @@ meson build $MESON_OPTIONS $EXTRA_MESON_ARGS
ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
ninja -C build install
+set +ex
+export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/igt/lib64
+while read -r line; do
+if [ "$line" = "TESTLIST" ] || [ "$line" = "END TESTLIST" ]; then
+continue
+fi
+
+tests=$(echo "$line" | tr ' ' '\n')
+
+for test in $tests; do
+output=$(/igt/libexec/igt-gpu-tools/"$test" --list-subtests)
+
+if [ -z "$output" ]; then
+echo "$test"
+else
+echo "$output" | while read -r subtest; do
+echo "$test@$subtest"
+done
+fi
+done
+done < /igt/libexec/igt-gpu-tools/test-list.txt >
/igt/libexec/igt-gpu-tools/testlist.txt
+set -ex
Is the list in sync between x86 and arm/arm64 IGT builds? Is there a
chance of having a safety net here?
We need to handle arm/arm64 cases also. IGT is not generating test-list
for arm and it is fixed now with
https://gitlab.freedesktop.org/drm/igt-gpu-tools/-/commit/1cf83083f855894dd287d9cf84bbcc2952b52d02
Will uprev IGT to latest commit to include this fix. Thanks.
Regards,
Vignesh
+
mkdir -p artifacts/
tar -cf artifacts/igt.tar /igt
diff --git a/drivers/gpu/drm/ci/igt_runner.sh b/drivers/gpu/drm/ci/igt_runner.sh
index f1a08b9b146f..20026612a9bd 100755
--- a/drivers/gpu/drm/ci/igt_runner.sh
+++ b/drivers/gpu/drm/ci/igt_runner.sh
@@ -59,25 +59,26 @@ fi
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s ${FDO_HTTP_CACHE_URI:-}$PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd -v -x -C /
+TESTLIST="/igt/libexec/igt-gpu-tools/testlist.txt"
# If the job is parallel at the gitab job level, take the corresponding fraction
# of the caselist.
if [ -n "$CI_NODE_INDEX" ]; then
-sed -ni $CI_NODE_INDEX~$CI_NODE_TOTAL"p" /install/testlist.txt
+sed -ni $CI_NODE_INDEX~$CI_NODE_TOTAL"p" $TESTLIST
fi
# core_getversion checks if the driver is loaded and probed correctly
# so run it in all shards
-if ! grep -q "core_getversion" /install/testlist.txt; then
+if ! grep -q "core_getversion" $TESTLIST; then
# Add the line to the file
-echo "core_getversion" >> /install/testlist.txt
+echo "core_getversion" >> $TESTLIST
fi
set +e
igt-runner \
run \
--igt-folder /igt/libexec/igt-gpu-tools \
---caselist /install/testlist.txt \
+--caselist $TESTLIST \
--output /results \
$IGT_SKIPS \
$IGT_FLAKES \
Re: [PATCH v1 2/5] drm/ci: generate testlist from build
On Tue, Apr 30, 2024 at 02:41:18PM +0530, Vignesh Raman wrote:
> Stop vendoring the testlist into the kernel. Instead, use the
> testlist from the IGT build to ensure we do not miss renamed
> or newly added tests.
>
> Signed-off-by: Vignesh Raman
> ---
> drivers/gpu/drm/ci/build-igt.sh | 23 +
> drivers/gpu/drm/ci/igt_runner.sh |9 +-
> drivers/gpu/drm/ci/testlist.txt | 2761 --
> 3 files changed, 28 insertions(+), 2765 deletions(-)
> delete mode 100644 drivers/gpu/drm/ci/testlist.txt
>
> diff --git a/drivers/gpu/drm/ci/build-igt.sh b/drivers/gpu/drm/ci/build-igt.sh
> index 500fa4f5c30a..cedc62baba1e 100644
> --- a/drivers/gpu/drm/ci/build-igt.sh
> +++ b/drivers/gpu/drm/ci/build-igt.sh
> @@ -26,6 +26,29 @@ meson build $MESON_OPTIONS $EXTRA_MESON_ARGS
> ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
> ninja -C build install
>
> +set +ex
> +export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/igt/lib64
> +while read -r line; do
> +if [ "$line" = "TESTLIST" ] || [ "$line" = "END TESTLIST" ]; then
> +continue
> +fi
> +
> +tests=$(echo "$line" | tr ' ' '\n')
> +
> +for test in $tests; do
> +output=$(/igt/libexec/igt-gpu-tools/"$test" --list-subtests)
> +
> +if [ -z "$output" ]; then
> +echo "$test"
> +else
> +echo "$output" | while read -r subtest; do
> +echo "$test@$subtest"
> +done
> +fi
> +done
> +done < /igt/libexec/igt-gpu-tools/test-list.txt >
> /igt/libexec/igt-gpu-tools/testlist.txt
> +set -ex
Is the list in sync between x86 and arm/arm64 IGT builds? Is there a
chance of having a safety net here?
> +
> mkdir -p artifacts/
> tar -cf artifacts/igt.tar /igt
>
> diff --git a/drivers/gpu/drm/ci/igt_runner.sh
> b/drivers/gpu/drm/ci/igt_runner.sh
> index f1a08b9b146f..20026612a9bd 100755
> --- a/drivers/gpu/drm/ci/igt_runner.sh
> +++ b/drivers/gpu/drm/ci/igt_runner.sh
> @@ -59,25 +59,26 @@ fi
>
> curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s
> ${FDO_HTTP_CACHE_URI:-}$PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd
> -v -x -C /
>
> +TESTLIST="/igt/libexec/igt-gpu-tools/testlist.txt"
>
> # If the job is parallel at the gitab job level, take the corresponding
> fraction
> # of the caselist.
> if [ -n "$CI_NODE_INDEX" ]; then
> -sed -ni $CI_NODE_INDEX~$CI_NODE_TOTAL"p" /install/testlist.txt
> +sed -ni $CI_NODE_INDEX~$CI_NODE_TOTAL"p" $TESTLIST
> fi
>
> # core_getversion checks if the driver is loaded and probed correctly
> # so run it in all shards
> -if ! grep -q "core_getversion" /install/testlist.txt; then
> +if ! grep -q "core_getversion" $TESTLIST; then
> # Add the line to the file
> -echo "core_getversion" >> /install/testlist.txt
> +echo "core_getversion" >> $TESTLIST
> fi
>
> set +e
> igt-runner \
> run \
> --igt-folder /igt/libexec/igt-gpu-tools \
> ---caselist /install/testlist.txt \
> +--caselist $TESTLIST \
> --output /results \
> $IGT_SKIPS \
> $IGT_FLAKES \
--
With best wishes
Dmitry
[PATCH v1 2/5] drm/ci: generate testlist from build
Stop vendoring the testlist into the kernel. Instead, use the
testlist from the IGT build to ensure we do not miss renamed
or newly added tests.
Signed-off-by: Vignesh Raman
---
drivers/gpu/drm/ci/build-igt.sh | 23 +
drivers/gpu/drm/ci/igt_runner.sh |9 +-
drivers/gpu/drm/ci/testlist.txt | 2761 --
3 files changed, 28 insertions(+), 2765 deletions(-)
delete mode 100644 drivers/gpu/drm/ci/testlist.txt
diff --git a/drivers/gpu/drm/ci/build-igt.sh b/drivers/gpu/drm/ci/build-igt.sh
index 500fa4f5c30a..cedc62baba1e 100644
--- a/drivers/gpu/drm/ci/build-igt.sh
+++ b/drivers/gpu/drm/ci/build-igt.sh
@@ -26,6 +26,29 @@ meson build $MESON_OPTIONS $EXTRA_MESON_ARGS
ninja -C build -j${FDO_CI_CONCURRENT:-4} || ninja -C build -j 1
ninja -C build install
+set +ex
+export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/igt/lib64
+while read -r line; do
+if [ "$line" = "TESTLIST" ] || [ "$line" = "END TESTLIST" ]; then
+continue
+fi
+
+tests=$(echo "$line" | tr ' ' '\n')
+
+for test in $tests; do
+output=$(/igt/libexec/igt-gpu-tools/"$test" --list-subtests)
+
+if [ -z "$output" ]; then
+echo "$test"
+else
+echo "$output" | while read -r subtest; do
+echo "$test@$subtest"
+done
+fi
+done
+done < /igt/libexec/igt-gpu-tools/test-list.txt >
/igt/libexec/igt-gpu-tools/testlist.txt
+set -ex
+
mkdir -p artifacts/
tar -cf artifacts/igt.tar /igt
diff --git a/drivers/gpu/drm/ci/igt_runner.sh b/drivers/gpu/drm/ci/igt_runner.sh
index f1a08b9b146f..20026612a9bd 100755
--- a/drivers/gpu/drm/ci/igt_runner.sh
+++ b/drivers/gpu/drm/ci/igt_runner.sh
@@ -59,25 +59,26 @@ fi
curl -L --retry 4 -f --retry-all-errors --retry-delay 60 -s
${FDO_HTTP_CACHE_URI:-}$PIPELINE_ARTIFACTS_BASE/$ARCH/igt.tar.gz | tar --zstd
-v -x -C /
+TESTLIST="/igt/libexec/igt-gpu-tools/testlist.txt"
# If the job is parallel at the gitab job level, take the corresponding
fraction
# of the caselist.
if [ -n "$CI_NODE_INDEX" ]; then
-sed -ni $CI_NODE_INDEX~$CI_NODE_TOTAL"p" /install/testlist.txt
+sed -ni $CI_NODE_INDEX~$CI_NODE_TOTAL"p" $TESTLIST
fi
# core_getversion checks if the driver is loaded and probed correctly
# so run it in all shards
-if ! grep -q "core_getversion" /install/testlist.txt; then
+if ! grep -q "core_getversion" $TESTLIST; then
# Add the line to the file
-echo "core_getversion" >> /install/testlist.txt
+echo "core_getversion" >> $TESTLIST
fi
set +e
igt-runner \
run \
--igt-folder /igt/libexec/igt-gpu-tools \
---caselist /install/testlist.txt \
+--caselist $TESTLIST \
--output /results \
$IGT_SKIPS \
$IGT_FLAKES \
diff --git a/drivers/gpu/drm/ci/testlist.txt b/drivers/gpu/drm/ci/testlist.txt
deleted file mode 100644
index 3377f002f8c5..
--- a/drivers/gpu/drm/ci/testlist.txt
+++ /dev/null
@@ -1,2761 +0,0 @@
-core_auth@getclient-simple
-core_auth@getclient-master-drop
-core_auth@basic-auth
-core_auth@many-magics
-core_getclient
-core_getstats
-core_getversion
-core_setmaster_vs_auth
-drm_read@invalid-buffer
-drm_read@fault-buffer
-drm_read@empty-block
-drm_read@empty-nonblock
-drm_read@short-buffer-block
-drm_read@short-buffer-nonblock
-drm_read@short-buffer-wakeup
-gem_eio@throttle
-gem_eio@create
-gem_eio@create-ext
-gem_eio@context-create
-gem_eio@execbuf
-gem_eio@banned
-gem_eio@suspend
-gem_eio@hibernate
-gem_eio@in-flight-external
-gem_eio@in-flight-suspend
-gem_eio@reset-stress
-gem_eio@unwedge-stress
-gem_eio@wait-immediate
-gem_eio@wait-wedge-immediate
-gem_eio@in-flight-immediate
-gem_eio@in-flight-contexts-immediate
-gem_eio@in-flight-internal-immediate
-gem_eio@wait-1us
-gem_eio@wait-wedge-1us
-gem_eio@in-flight-1us
-gem_eio@in-flight-contexts-1us
-gem_eio@in-flight-internal-1us
-gem_eio@wait-10ms
-gem_eio@wait-wedge-10ms
-gem_eio@in-flight-10ms
-gem_eio@in-flight-contexts-10ms
-gem_eio@in-flight-internal-10ms
-gem_eio@kms
-kms_3d
-kms_addfb_basic@unused-handle
-kms_addfb_basic@unused-pitches
-kms_addfb_basic@unused-offsets
-kms_addfb_basic@unused-modifier
-kms_addfb_basic@clobberred-modifier
-kms_addfb_basic@invalid-smem-bo-on-discrete
-kms_addfb_basic@legacy-format
-kms_addfb_basic@no-handle
-kms_addfb_basic@basic
-kms_addfb_basic@bad-pitch-0
-kms_addfb_basic@bad-pitch-32
-kms_addfb_basic@bad-pitch-63
-kms_addfb_basic@bad-pitch-128
-kms_addfb_basic@bad-pitch-256
-kms_addfb_basic@bad-pitch-1024
-kms_addfb_basic@bad-pitch-999
-kms_addfb_basic@bad-pitch-65536
-kms_addfb_basic@invalid-get-prop-any
-kms_addfb_basic@invalid-get-prop
-kms_addfb_basic@invalid-set-prop-any
-kms_addfb_basic@invalid-set-prop
-kms_addfb_basic@master-rmfb
-kms_addfb_basic@addfb25-modifier-no-flag
-kms_addfb_basic@addfb25-bad-modifier
-kms_addfb_basic@addfb25-x-tiled-mismatch-legacy
-kms_addfb_basic@addfb25-x-tiled-legacy
-kms_addfb_basic@addfb25-framebuffer-vs-set-tiling
-kms_addfb_basic@basic-x-tiled-legacy
