It was previously reported[1] and discussed that tests booting full blown distros and relying on TCG would take too much time to run, especially in the environments given by GitLab CI's shared runners.
This is an implementation of a proposal to exclude those tests from being run by default on `make check-avocado` invocations. To make it extra clear, all tests are still available, but those that are tagged with "accel:tcg" and "boots:distro", are filtered out by default on `make check-avocado`. This is the situation of the Avocado GitLab CI jobs with and without the changes in this PS: +------------------+--- ----------------+ | Now[2] | Before[3] | +------------------------+------------------+--------------------+ | Job | Length | Tests | Length | Tests | | Name | (mm:ss) | Run | (mm:ss) | Run | +------------------------+------------------+-------------------+ |avocado-system-alpine | 06:33 16 | 20:30 18 | |avocado-system-debian | 12:06 24 | 13:05 24 | |avocado-system-centos | 09:58 41 | 24:15 44 | |avocado-system-fedora | 08:50 35 | 08:59 35 | |avocado-system-opensuse | 08:09 38 | 27:21 42 | |avocado-system-ubuntu | 06:52 16 | 18:52 18 | |avocado-cfi-x86_64 | 05:43 27 | 15:07 29 | +------------------------+------------------+--------------------+ |TOTALS | 58:11 197 | 2:08:09 210 | +------------------------+------------------+--------------------+ Assuming the jobs run in parallel, the overall wait time for all the Avocado jobs to complete is now ~12 minutes. [1] https://lists.gnu.org/archive/html/qemu-devel/2021-07/msg07271.html [2] https://gitlab.com/cleber.gnu/qemu/-/pipelines/479720240 [3] https://gitlab.com/qemu-project/qemu/-/pipelines/478580581 Cleber Rosa (9): Avocado GitLab CI jobs: don't reset TARGETS and simplify commands Avocado tests: use logging namespace that is preserved in test logs Avocado migration test: adapt to "utils.network" API namespace change Avocado: bump to version 95.0 tests/avocado/linux_ssh_mips_malta.py: add missing accel (tcg) tag tests/avocado/virtiofs_submounts.py: shared_dir may not exist Avocado tests: improve documentation on tag filtering Avocado tests: classify tests based on what it's booted Avocado tests: don't run tests with TCG that boot full blown distros .gitlab-ci.d/buildtest-template.yml | 3 ++ .gitlab-ci.d/buildtest.yml | 9 ---- docs/devel/testing.rst | 22 +++++++++ tests/Makefile.include | 6 ++- tests/avocado/avocado_qemu/__init__.py | 10 ++--- tests/avocado/boot_linux.py | 4 ++ tests/avocado/boot_linux_console.py | 54 +++++++++++++++++++++++ tests/avocado/boot_xen.py | 3 ++ tests/avocado/hotplug_cpu.py | 1 + tests/avocado/intel_iommu.py | 1 + tests/avocado/linux_initrd.py | 5 ++- tests/avocado/linux_ssh_mips_malta.py | 5 +++ tests/avocado/machine_arm_canona1100.py | 1 + tests/avocado/machine_arm_integratorcp.py | 7 ++- tests/avocado/machine_arm_n8x0.py | 2 + tests/avocado/machine_avr6.py | 1 + tests/avocado/machine_m68k_nextcube.py | 1 + tests/avocado/machine_microblaze.py | 1 + tests/avocado/machine_mips_fuloong2e.py | 1 + tests/avocado/machine_mips_loongson3v.py | 1 + tests/avocado/machine_mips_malta.py | 6 ++- tests/avocado/machine_rx_gdbsim.py | 2 + tests/avocado/machine_s390_ccw_virtio.py | 4 ++ tests/avocado/machine_sparc64_sun4u.py | 1 + tests/avocado/machine_sparc_leon3.py | 1 + tests/avocado/migration.py | 4 +- tests/avocado/multiprocess.py | 4 ++ tests/avocado/ppc_405.py | 2 + tests/avocado/ppc_bamboo.py | 2 + tests/avocado/ppc_mpc8544ds.py | 1 + tests/avocado/ppc_prep_40p.py | 1 + tests/avocado/ppc_pseries.py | 1 + tests/avocado/ppc_virtex_ml507.py | 1 + tests/avocado/replay_kernel.py | 33 ++++++++++++-- tests/avocado/replay_linux.py | 6 +-- tests/avocado/reverse_debugging.py | 6 +-- tests/avocado/smmu.py | 1 + tests/avocado/tcg_plugins.py | 3 ++ tests/avocado/tesseract_utils.py | 6 +-- tests/avocado/virtio-gpu.py | 2 + tests/avocado/virtio_check_params.py | 3 +- tests/avocado/virtiofs_submounts.py | 8 ++-- tests/requirements.txt | 2 +- 43 files changed, 197 insertions(+), 41 deletions(-) -- 2.35.1