Hi, On Thu, 9 Apr 2026 at 11:01, Nazir Bilal Yavuz <[email protected]> wrote: > On Tue, 7 Apr 2026 at 19:27, Nazir Bilal Yavuz <[email protected]> wrote: > > > Here is v2, which addresses the issues I mentioned earlier. > > I have updated the design since this will only be used for Meson > builds. I decided to invoke the script using 'meson compile > clear_testrun_folder', as this avoids the need to manually call > python3. This approach prevents issues on NetBSD, where the python3 > environment may not be found. > > I don’t expect the 'ci_meson_clear_testrun_folder' script to be used > outside of CI, so I placed it in the 'src/tools/ci/' directory. > However, if you think it could be useful beyond CI, we could consider > renaming it and moving it to 'src/tools/'.
Here is the v3, it is modified for Github Actions. I implemented it as another step which will run if the job fails. Alternatively, we can put 'the clearing command' into the meson_test_world step, this makes the change simpler but then this command will run although tests don't fail. With this change, the compressed size of the log decreases from ~8MB to ~1MB. Also, it is much easier to find which tests failed. You can see example GHA run, which intentionally failed: https://github.com/nbyavuz/postgres/actions/runs/27013199064 -- Regards, Nazir Bilal Yavuz Microsoft
From 810fc7b3a773e0011dc1768c490952fcca184466 Mon Sep 17 00:00:00 2001 From: Nazir Bilal Yavuz <[email protected]> Date: Wed, 8 Apr 2026 11:23:34 +0300 Subject: [PATCH v3] ci: Don't collect successful tests' logs Add a Python script that removes subdirectories containing a test.success file from a given testrun folder. This ensures that only failed tests' logs are kept when uploading CI artifacts, reducing artifact size and noise. This change affects only Meson CI builds as test.success file is created only on Meson builds. --- .github/workflows/pg-ci.yml | 18 ++++++++ meson.build | 4 ++ src/tools/ci/ci_meson_clear_testrun_folder | 50 ++++++++++++++++++++++ 3 files changed, 72 insertions(+) create mode 100755 src/tools/ci/ci_meson_clear_testrun_folder diff --git a/.github/workflows/pg-ci.yml b/.github/workflows/pg-ci.yml index 8560e9389f6..ad9e79171d6 100644 --- a/.github/workflows/pg-ci.yml +++ b/.github/workflows/pg-ci.yml @@ -362,6 +362,13 @@ jobs: meson test ${{env.MTEST_ARGS}} --num-processes ${{env.TEST_JOBS}} --no-suite setup ${{env.MTEST_TARGET}} + # Clear test folder so only failed tests are left + - &clear_test_folder_step + name: Clear test folder + if: failure() && !cancelled() + run: | + meson compile clear_testrun_folder -C build + - &linux_collect_cores_step name: Core backtraces if: failure() && !cancelled() @@ -566,6 +573,10 @@ jobs: build/tmp_install/usr/local/pgsql/bin/pg_ctl -D build/runningcheck stop + # Clear test folder so only failed tests are left + meson compile clear_testrun_folder -C build + + - *clear_test_folder_step - *linux_collect_cores_step - *upload_logs_step @@ -619,6 +630,7 @@ jobs: shell: *su_postgres_shell run: *meson_test_world_cmd + - *clear_test_folder_step - *linux_collect_cores_step - *upload_logs_step @@ -770,6 +782,8 @@ jobs: ADDITIONAL_SETUP: ulimit -n 1024 run: *meson_test_world_cmd + - *clear_test_folder_step + - name: Core backtraces if: failure() && !cancelled() run: src/tools/ci/cores_backtrace.sh macos "$HOME/cores" @@ -965,6 +979,8 @@ jobs: call "C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Auxiliary\Build\vcvarsall.bat" x64 run: *meson_test_world_cmd + - *clear_test_folder_step + # TODO: We need to collect crashlogs but for them to be generated, we'd # have to configure the JIT Debugger to do so. cdb.exe is installed on # the runner so that is possible. @@ -1082,6 +1098,8 @@ jobs: - name: Test world run: *meson_test_world_cmd + - *clear_test_folder_step + # TODO: We want to include crashlogs, but they are not yet # collected. cdb.exe is installed on the runner, so we can configure it # appropriately. diff --git a/meson.build b/meson.build index 568e0e150bf..d73c8874fac 100644 --- a/meson.build +++ b/meson.build @@ -4133,6 +4133,10 @@ add_test_setup('tmp_install', add_test_setup('running', exclude_suites: ['setup'] + install_suites) +clear_testrun_folder = files('src/tools/ci/ci_meson_clear_testrun_folder') +run_target('clear_testrun_folder', + command: [python, clear_testrun_folder, test_result_dir] +) ############################################################### diff --git a/src/tools/ci/ci_meson_clear_testrun_folder b/src/tools/ci/ci_meson_clear_testrun_folder new file mode 100755 index 00000000000..bb44066e1eb --- /dev/null +++ b/src/tools/ci/ci_meson_clear_testrun_folder @@ -0,0 +1,50 @@ +#!/usr/bin/env python3 +# +# After a test run, each test that passed contains a test.success file in its +# output directory. This script removes those directories so that only the logs +# of failed tests remain, making it easier to inspect CI artifacts. +# +# Usage: ci_meson_clear_testrun_folder <path> +# e.g. ci_meson_clear_testrun_folder build/testrun + +import argparse +import os +import shutil +import sys + +parser = argparse.ArgumentParser( + description="Remove subdirectories that contain a test.success file." +) +parser.add_argument( + "path", + help="Path to the directory to clear, e.g. 'build/testrun'", +) +args = parser.parse_args() + +testrun_dir = os.path.abspath(args.path) + +if not os.path.exists(testrun_dir): + print(f"Directory does not exist, nothing to do: {testrun_dir}") + sys.exit(0) + +if not os.path.isdir(testrun_dir): + print(f"Error: not a directory: {testrun_dir}") + sys.exit(1) + +print(f"Removing successful tests from {testrun_dir}:") +for dirpath, dirnames, filenames in os.walk(testrun_dir): + if "test.success" in filenames: + print(f"Removing: {dirpath}") + shutil.rmtree(dirpath) + dirnames.clear() +print("Done\n") + +print(f"Removing any empty directories from {testrun_dir}:") +for dirpath, _, _ in os.walk(testrun_dir, topdown=False): + # Don't remove ${testrun_dir} + if dirpath == testrun_dir: + continue + if not os.listdir(dirpath): + print(f"Removing empty dir: {dirpath}") + os.rmdir(dirpath) +print("Done\n") -- 2.47.3
