[PATCH v5 09/44] tests/functional: Set up logging

2024-08-30 Thread Thomas Huth
Create log files for each test separately, one file that contains the basic logging and one that contains the console output. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/functional/qemu_test/testcase.py | 27 +- 1 file changed, 26 insertions

[PATCH v5 14/44] tests/functional: add a module for handling asset download & caching

2024-08-30 Thread Thomas Huth
hilippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé [thuth: Drop sha1 support, use hash on file content for naming instead of URL, add the possibility to specify the cache dir via environment variable] Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 1 + t

[PATCH v5 13/44] tests/functional: Convert avocado tests that just need a small adjustment

2024-08-30 Thread Thomas Huth
ed-by: Daniel P. Berrangé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 3 + tests/functional/meson.build | 6 ++ .../test_info_usernet.py} | 11

[PATCH v5 02/44] tests/avocado/boot_xen.py: fetch kernel during test setUp()

2024-08-30 Thread Thomas Huth
by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-7-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/boot_xen.py | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/tests/avocado/boot_xen.py b/tests/avocado/boot_xen.py index f29bc58b9e..4

Re: [PATCH for-9.2 00/10] s390: Convert virtio-ccw, cpu to three-phase reset, and followup cleanup

2024-08-30 Thread Thomas Huth
On 30/08/2024 14.17, Peter Maydell wrote: On Fri, 30 Aug 2024 at 13:04, Peter Maydell wrote: On Fri, 30 Aug 2024 at 12:56, Nico Boehr wrote: Quoting Peter Maydell (2024-08-30 12:01:47) I ran overnight with none of the patchset applied, and it never failed (as expected). Running with just t

Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-30 Thread Thomas Huth
On 30/08/2024 09.42, Daniel P. Berrangé wrote: On Fri, Aug 30, 2024 at 09:38:17AM +0200, Thomas Huth wrote: On 29/08/2024 12.15, Daniel P. Berrangé wrote: On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote: On 27/08/2024 15.16, Thomas Huth wrote: On 23/08/2024 09.28, Philippe

Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-30 Thread Thomas Huth
On 29/08/2024 12.15, Daniel P. Berrangé wrote: On Tue, Aug 27, 2024 at 04:24:59PM +0200, Thomas Huth wrote: On 27/08/2024 15.16, Thomas Huth wrote: On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote: Hi, On 21/8/24 10:27, Thomas Huth wrote: From: Daniel P. Berrangé Many tests need to

Re: [PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests

2024-08-29 Thread Thomas Huth
On 29/08/2024 12.34, Daniel P. Berrangé wrote: On Wed, Aug 21, 2024 at 10:27:36AM +0200, Thomas Huth wrote: Document the new functional testing framework. The text is originally based on the Avocado documentation, but heavily modified to match the new framework. Signed-off-by: Thomas Huth

Re: [PATCH] iotests: fix expected output from gnutls in NBD test

2024-08-29 Thread Thomas Huth
effect of changing the output from one of the NBD tests. Reported-by: Thomas Huth Signed-off-by: Daniel P. Berrangé --- NB, wrt qemu-stable this will be for the 9.1 stable branch once that is created, no earlier releases will need this. tests/qemu-iotests/233.out | 12 ++-- 1 file

Re: [PATCH] iotests: fix expected output from gnutls in NBD test

2024-08-29 Thread Thomas Huth
effect of changing the output from one of the NBD tests. Reported-by: Thomas Huth Signed-off-by: Daniel P. Berrangé --- NB, wrt qemu-stable this will be for the 9.1 stable branch once that is created, no earlier releases will need this. tests/qemu-iotests/233.out | 12 ++-- 1 file

Re: [PULL 10/11] crypto: push error reporting into TLS session I/O APIs

2024-08-28 Thread Thomas Huth
On 27/08/2024 09.05, Markus Armbruster wrote: Daniel P. Berrangé writes: On Mon, Aug 12, 2024 at 05:38:41PM +0200, Thomas Huth wrote: On 24/07/2024 11.47, Daniel P. Berrangé wrote: The current TLS session I/O APIs just return a synthetic errno value on error, which has been translated from

Re: [PULL 10/11] crypto: push error reporting into TLS session I/O APIs

2024-08-28 Thread Thomas Huth
On 27/08/2024 09.05, Markus Armbruster wrote: Daniel P. Berrangé writes: On Mon, Aug 12, 2024 at 05:38:41PM +0200, Thomas Huth wrote: On 24/07/2024 11.47, Daniel P. Berrangé wrote: The current TLS session I/O APIs just return a synthetic errno value on error, which has been translated from

[OS-BUILD PATCH] [redhat] Disable CONFIG_S390_HYPFS in the zfcpdump kernel

2024-08-27 Thread Thomas Huth (via Email Bridge)
From: Thomas Huth [redhat] Disable CONFIG_S390_HYPFS in the zfcpdump kernel JIRA: https://issues.redhat.com/browse/RHEL-56069 Upstream Status: RHEL only CONFIG_S390_HYPFS has been split from CONFIG_S390_HYPFS_FS (with "_FS" suffix) in upstream commit 3325b4d857999 ("s390/hy

Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-27 Thread Thomas Huth
On 27/08/2024 15.16, Thomas Huth wrote: On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote: Hi, On 21/8/24 10:27, Thomas Huth wrote: From: Daniel P. Berrangé Many tests need to access assets stored on remote sites. We don't want to download these during test execution when run by

Re: [PATCH v4 7/7] tests/qtest: Delete previous boot file

2024-08-27 Thread Thomas Huth
On 26/08/2024 17.26, Peter Xu wrote: On Fri, Aug 23, 2024 at 03:13:12PM +0900, Akihiko Odaki wrote: A test run may create boot files several times. Delete the previous boot file before creating a new one. Signed-off-by: Akihiko Odaki Reviewed-by: Michael S. Tsirkin Acked-by: Thomas Huth I

Re: [PATCH v4 7/7] tests/qtest: Delete previous boot file

2024-08-27 Thread Thomas Huth
On 26/08/2024 17.26, Peter Xu wrote: On Fri, Aug 23, 2024 at 03:13:12PM +0900, Akihiko Odaki wrote: A test run may create boot files several times. Delete the previous boot file before creating a new one. Signed-off-by: Akihiko Odaki Reviewed-by: Michael S. Tsirkin Acked-by: Thomas Huth I

Re: [PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-27 Thread Thomas Huth
On 23/08/2024 09.28, Philippe Mathieu-Daudé wrote: Hi, On 21/8/24 10:27, Thomas Huth wrote: From: Daniel P. Berrangé Many tests need to access assets stored on remote sites. We don't want to download these during test execution when run by meson, since this risks hitting test timeouts

Re: [PATCH 0/7] pc-bios/s390-ccw: Merge the netboot loader into s390-ccw.img

2024-08-27 Thread Thomas Huth
On 26/08/2024 19.07, Jared Rossi wrote: Hi Thomas, I just wanted to get your thoughts on the status of the netboot loader merge. I see that the first patch from this series was merged, but not the others. Is there any issue with the rest of the changes? Hi Jared, there's no issue with that p

[PATCH 2/2] tests/functional: Convert the m68k Q800 Avocado test into a functional test

2024-08-27 Thread Thomas Huth
Just had to update the asset checksum to use SHA256 instead of SHA1, but apart from that it is a pretty much straightforward conversion. Signed-off-by: Thomas Huth --- MAINTAINERS | 1 + tests/avocado/boot_linux_console.py | 24 --- tests/functional

[PATCH 0/2] tests/functional: Add LinuxKernelTest class and convert Q800 test

2024-08-27 Thread Thomas Huth
t as an example here). Thomas Huth (2): tests/functional: Add the LinuxKernelTest for testing the Linux boot process tests/functional: Convert the m68k Q800 Avocado test into a functional test MAINTAINERS | 1 + tests/avocado/boot_linux_consol

[PATCH 1/2] tests/functional: Add the LinuxKernelTest for testing the Linux boot process

2024-08-27 Thread Thomas Huth
Copy the LinuxKernelTest from tests/acceptance/boot_linux_console.py to be able to convert the related tests to the functional test framework in the following patches. Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py| 1 + tests/functional/qemu_test/linuxkernel.py | 41

Re: [PATCH 1/4] tests/functional: Add a class containing Linux kernel helpers

2024-08-27 Thread Thomas Huth
On 27/08/2024 11.21, Philippe Mathieu-Daudé wrote: Hi Thomas, On 27/8/24 07:34, Thomas Huth wrote: On 27/08/2024 00.10, Philippe Mathieu-Daudé wrote: Use the 'linux_kernel' namespace to provide common helpers to functional tests booting a Linux kernel. Suggested-by: Thomas Huth

Re: [PATCH v3 2/7] tests/functional: Convert mips64el Fuloong2e avocado test (2/2)

2024-08-27 Thread Thomas Huth
On 27/08/2024 11.16, Philippe Mathieu-Daudé wrote: On 27/8/24 07:27, Thomas Huth wrote: On 26/08/2024 22.59, Philippe Mathieu-Daudé wrote: On 26/8/24 11:10, Thomas Huth wrote: On 24/08/2024 18.08, Philippe Mathieu-Daudé wrote: Straight forward conversion. Update the SHA1 hashes to SHA256

Re: [PATCH 1/4] tests/functional: Add a class containing Linux kernel helpers

2024-08-26 Thread Thomas Huth
On 27/08/2024 00.10, Philippe Mathieu-Daudé wrote: Use the 'linux_kernel' namespace to provide common helpers to functional tests booting a Linux kernel. Suggested-by: Thomas Huth Signed-off-by: Philippe Mathieu-Daudé --- tests/functional/qemu_test/linux_kern

Re: [PATCH v3 2/7] tests/functional: Convert mips64el Fuloong2e avocado test (2/2)

2024-08-26 Thread Thomas Huth
On 26/08/2024 22.59, Philippe Mathieu-Daudé wrote: On 26/8/24 11:10, Thomas Huth wrote: On 24/08/2024 18.08, Philippe Mathieu-Daudé wrote: Straight forward conversion. Update the SHA1 hashes to SHA256 hashes since SHA1 should not be used anymore nowadays. Add extract_from_deb() method in

Re: [PATCH v3] tests/functional: Convert Aarch64 Virt machine avocado tests

2024-08-26 Thread Thomas Huth
-aarch64_virt OK 30.44s 3 subtests passed ... Signed-off-by: Philippe Mathieu-Daudé --- Based-on: <20240822114146.86838-1-phi...@linaro.org> --- Reviewed-by: Thomas Huth

[PULL 1/3] gitlab-ci: Replace build_script -> step_script in Cirrus jobs

2024-08-26 Thread Thomas Huth
t-in-custom-executor [2] https://about.gitlab.com/releases/2024/05/16/gitlab-17-0-released/ Signed-off-by: Philippe Mathieu-Daudé Reviewed-by: Thomas Huth Message-ID: <20240816213203.18350-1-phi...@linaro.org> Signed-off-by: Thomas Huth --- .gitlab-ci.d/cirrus/build.yml | 2 +- 1 file

[PULL 3/3] tests/qtest: Delete previous boot file

2024-08-26 Thread Thomas Huth
From: Akihiko Odaki A test run may create boot files several times. Delete the previous boot file before creating a new one. Signed-off-by: Akihiko Odaki Reviewed-by: Michael S. Tsirkin Message-ID: <20240823-san-v4-7-a24c6dfa4...@daynix.com> Signed-off-by: Thomas Huth --- tests

[PULL 2/3] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-26 Thread Thomas Huth
failure of the qtests on MSYS2, we can revert this patch to get everything back. Suggested-by: Philippe Mathieu-Daudé Message-ID: <20240820170142.55324-1-th...@redhat.com> Signed-off-by: Thomas Huth --- .gitlab-ci.d/windows.yml | 26 ++ 1 file changed, 2 insert

[PULL 0/3] Optional fixes for 9.1.0-rc4

2024-08-26 Thread Thomas Huth
e previous boot file Philippe Mathieu-Daudé (1): gitlab-ci: Replace build_script -> step_script in Cirrus jobs Thomas Huth (1): .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job tests/qtest/migration-test.c | 18 +++--- .gitlab-ci.d/cirrus/build.yml |

Re: [PATCH v3 2/7] tests/functional: Convert mips64el Fuloong2e avocado test (2/2)

2024-08-26 Thread Thomas Huth
On 24/08/2024 18.08, Philippe Mathieu-Daudé wrote: Straight forward conversion. Update the SHA1 hashes to SHA256 hashes since SHA1 should not be used anymore nowadays. Add extract_from_deb() method in qemu_test.utils package. Should we maybe rather copy the whole LinuxKernelTest class into a n

Re: [PATCH v3 1/7] tests/functional: Convert mips64el Fuloong2e avocado test (1/2)

2024-08-26 Thread Thomas Huth
sts passed Signed-off-by: Philippe Mathieu-Daudé --- Reviewed-by: Thomas Huth

Re: [PATCH v8 6/8] rust: add crate to expose bindings and interfaces

2024-08-26 Thread Thomas Huth
On 26/08/2024 08.41, Manos Pitsidianakis wrote: On Mon, 26 Aug 2024 08:31, Junjie Mao wrote: +unsafe impl GlobalAlloc for QemuAllocator { +    unsafe fn alloc(&self, layout: Layout) -> *mut u8 { +    if layout.align() == 0 { +    g_malloc0(layout.size().try_into().unwrap()).cast::()

Re: [PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests

2024-08-26 Thread Thomas Huth
On 23/08/2024 14.54, Philippe Mathieu-Daudé wrote: On 21/8/24 10:27, Thomas Huth wrote: Provide a meson.build file for the upcoming python-based functional tests, and add some wrapper glue targets to the tests/Makefile.include file. We are going to use two "speed" modes for the functi

Re: [PATCH v2] tests/functional: Convert Aarch64 Virt machine avocado tests

2024-08-26 Thread Thomas Huth
On 23/08/2024 10.24, Philippe Mathieu-Daudé wrote: Straight forward conversion. Update the SHA1 hash to SHA256 since SHA1 should not be used anymore nowadays. Provide get_qemu_img() helper in qemu_test. $ make check-functional-aarch64 V=1 ... ▶ 1/4 test_aarch64_virt.Aarch64VirtMachine.

Re: [PATCH v2] tests/functional: Convert Aarch64 SBSA-Ref avocado tests

2024-08-22 Thread Thomas Huth
On 22/08/2024 16.39, Philippe Mathieu-Daudé wrote: On 22/8/24 16:06, Marcin Juszkiewicz wrote: On 22.08.2024 13:41, Philippe Mathieu-Daudé wrote:   # Timeouts for individual tests that can be slow e.g. with debugging enabled   test_timeouts = { +  'aarch64_sbsaref' : 180, What kind of machin

Re: [PATCH v2] tests/functional: Convert Aarch64 SBSA-Ref avocado tests

2024-08-22 Thread Thomas Huth
On 22/08/2024 16.06, Marcin Juszkiewicz wrote: On 22.08.2024 13:41, Philippe Mathieu-Daudé wrote:   # Timeouts for individual tests that can be slow e.g. with debugging enabled   test_timeouts = { +  'aarch64_sbsaref' : 180, What kind of machine is able to run those tests in 180s? I bumped th

Re: [PATCH 1/3] tests/functional: Add QemuUserTest class

2024-08-22 Thread Thomas Huth
user-mode emulation. Signed-off-by: Philippe Mathieu-Daudé --- tests/functional/qemu_test/__init__.py | 2 +- tests/functional/qemu_test/testcase.py | 17 + 2 files changed, 18 insertions(+), 1 deletion(-) Reviewed-by: Thomas Huth

Re: [PATCH 3/3] tests/avocado: Remove unused QemuUserTest class

2024-08-22 Thread Thomas Huth
+-- 1 file changed, 1 insertion(+), 18 deletions(-) Reviewed-by: Thomas Huth

Re: [PATCH] tests/functional: Convert Aarch64 Virt machine avocado tests

2024-08-22 Thread Thomas Huth
On 22/08/2024 14.07, Philippe Mathieu-Daudé wrote: Straight forward conversion. Update the SHA1 hash to SHA256 since SHA1 should not be used anymore nowadays. Use has_cmd() and run_cmd() methods from qemu_test. $ make check-functional-aarch64 V=1 ... ▶ 1/4 test_aarch64_virt.Aarch64VirtM

Re: [PATCH v2] tests/functional: Convert Aarch64 SBSA-Ref avocado tests

2024-08-22 Thread Thomas Huth
%) mode change 100644 => 100755 Thanks for your help with the conversions! Reviewed-by: Thomas Huth

Re: [PATCH] tests/functional: Convert ARM Integrator/CP avocado tests

2024-08-22 Thread Thomas Huth
rcp.py} | 56 ++- 3 files changed, 33 insertions(+), 26 deletions(-) rename tests/{avocado/machine_arm_integratorcp.py => functional/test_arm_integratorcp.py} (63%) mode change 100644 => 100755 Reviewed-by: Thomas Huth

Re: [PATCH v2 4/4] tests/avocado: Run STM32 bFLT busybox binary in current directory

2024-08-22 Thread Thomas Huth
/load_bflt.py:LoadBFLT.test_stm32: PASS (0.09 s) RESULTS: PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0 JOB TIME : 0.62 s Reported-by: Thomas Huth Signed-off-by: Philippe Mathieu-Daudé --- tests/avocado/load_bflt.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion

Re: [PATCH v2 3/3] tests/functional: Convert mipsel Malta YAMON avocado test

2024-08-22 Thread Thomas Huth
, 54 insertions(+), 55 deletions(-) delete mode 100644 tests/avocado/machine_mips_malta.py create mode 100755 tests/functional/test_mipsel_malta.py Reviewed-by: Thomas Huth

Re: [PATCH] tests/avocado: Allow running user-mode tests

2024-08-22 Thread Thomas Huth
from avocado.utils import ssh +from avocado.utils import process, ssh from avocado.utils.path import find_command Reviewed-by: Thomas Huth ... but is the test itself still working for you? When I run it, I get: ... (074/281) tests/avocado/linux_ssh_mips_malta.py:LinuxSSH.test_mips_malta

Re: [PATCH] tests/avocado: Move LinuxTest related code into a separate file

2024-08-21 Thread Thomas Huth
On 21/08/2024 17.11, Philippe Mathieu-Daudé wrote: On 19/7/24 11:50, Thomas Huth wrote: Only some few tests are using the LinuxTest class. Move the related code into a separate file so that this does not pollute the main namespace. Signed-off-by: Thomas Huth ---   tests/avocado/avocado_qemu

Re: [PATCH 1/2] tests/functional: Convert mips64el Malta I6400 avocado tests

2024-08-21 Thread Thomas Huth
ips64el -:avocado: tags=machine:malta -:avocado: tags=cpu:I6400 -:avocado: tags=mips:smp -:avocado: tags=flaky -""" -self.do_test_i6400_framebuffer_logo(8) - class MaltaMachine(QemuSystemTest): def do_test_yamon(self): diff --git a/tests/functional/meson.build b/tests/functional/meson.build index 8d28313a65..1823f88897 100644 --- a/tests/functional/meson.build +++ b/tests/functional/meson.build @@ -56,6 +56,7 @@ tests_mips64el_quick = [ ] tests_mips64el_thorough = [ + 'mips64el_malta', 'mips64el_loongson3v', ] Maybe keep the list in alphabetical order? Anyway: Reviewed-by: Thomas Huth

Re: [PATCH] tests/functional: Convert mips fuloong2e avocado test

2024-08-21 Thread Thomas Huth
On 21/08/2024 13.29, Philippe Mathieu-Daudé wrote: Straight forward conversion. Update the SHA1 hashes to SHA256 hashes since SHA1 should not be used anymore nowadays. Since the asset is expected locally and the test is guarded with QEMU_TEST_ALLOW_UNTRUSTED_CODE, keep it under the 'quick' categ

Re: [PATCH v4 26/35] tests/functional: Convert the ppc_hv avocado test into a standalone test

2024-08-21 Thread Thomas Huth
On 21/08/2024 11.43, Philippe Mathieu-Daudé wrote: On 21/8/24 10:27, Thomas Huth wrote: A straight forward conversion, we just also have to remove the decorator @skipUnless(os.getenv('SPEED')) since all non-trivial functional tests are running in SPEED=thorough mode now. Also make sur

Re: [PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py

2024-08-21 Thread Thomas Huth
On 21/08/2024 11.31, Philippe Mathieu-Daudé wrote: On 21/8/24 10:27, Thomas Huth wrote: Without this change, the new Avocado v103 fails to find the tests that are based on the LinuxTest class. Reorder before previous patch? The funny thing is that Avocado v88 breaks the other way round if

[PATCH v4 25/35] tests/functional: Convert the ppc_amiga avocado test into a standalone test

2024-08-21 Thread Thomas Huth
Use the Python standard zipfile module instead of avocado.utils for extracting the ZIP file that we download here, and use the standard subprocess module for running the "tail" command. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: T

[PATCH v4 07/35] python: Install pycotap in our venv if necessary

2024-08-21 Thread Thomas Huth
ere should not really be a problem). The wheel file has been obtained with: pip download --only-binary :all: --dest . --no-cache pycotap Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- python/wheels/pycotap-1.3.1-py3-none-any.whl | Bin 0 -> 5119 bytes py

[PATCH v4 10/35] tests/Makefile.include: Increase the level of indentation in the help text

2024-08-21 Thread Thomas Huth
The next patch is going to add some entries that need more space between the command and the help text, so let's increase the indentation here first. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Alex Bennée Signed-off-by: Thomas Huth ---

[PATCH v4 05/35] tests/avocado/avocado_qemu: Fix the "from" statements in linuxtest.py

2024-08-21 Thread Thomas Huth
Without this change, the new Avocado v103 fails to find the tests that are based on the LinuxTest class. Suggested-by: Cleber Rosa Signed-off-by: Thomas Huth --- tests/avocado/avocado_qemu/linuxtest.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/avocado

[PATCH v4 35/35] docs/devel/testing: Add documentation for functional tests

2024-08-21 Thread Thomas Huth
Document the new functional testing framework. The text is originally based on the Avocado documentation, but heavily modified to match the new framework. Signed-off-by: Thomas Huth --- docs/devel/testing/functional.rst | 269 ++ docs/devel/testing/index.rst

[PATCH v4 26/35] tests/functional: Convert the ppc_hv avocado test into a standalone test

2024-08-21 Thread Thomas Huth
and there are stale read-only files already around. Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/functional/meson.build | 2 + .../test_ppc64_hv.py} | 53 --- 3 files changed, 25 inserti

[PATCH v4 31/35] gitlab-ci: Add "check-functional" to the build tests

2024-08-21 Thread Thomas Huth
Now that we converted many tests from the "check-avocado" test suite to the "check-functional" test suite, we should make sure that these also get tested in the CI. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- .g

[PATCH v4 28/35] tests/functional: Convert the acpi-bits test into a standalone test

2024-08-21 Thread Thomas Huth
Mostly a straight-forward conversion. Looks like we can simply drop the avocado datadrainer stuff when not using the avocado framework anymore. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS

[PATCH v4 08/35] tests/functional: Add base classes for the upcoming pytest-based tests

2024-08-21 Thread Thomas Huth
will be set via meson.build later. Signed-off-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- MAINTAINERS| 5 + tests/functional/qemu_test/__init__.py | 13 ++ tests/functional/qemu_test/cmd.py | 178 + tests/functional/qemu_test

[PATCH v4 20/35] tests/functional: Convert the x86_cpu_model_versions test

2024-08-21 Thread Thomas Huth
Nothing thrilling in here, it's just a straight forward conversion. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 1 + tests/functional/meson.build

[PATCH v4 33/35] docs/devel/testing: Split the Avocado documentation into a separate file

2024-08-21 Thread Thomas Huth
The main testing documentation file got very overloaded already. Thus let's split the Avocado information into a separate file. Signed-off-by: Thomas Huth --- docs/devel/testing/avocado.rst | 581 + docs/devel/testing/index.rst | 1 + docs/devel/te

[PATCH v4 34/35] docs/devel/testing: Rename avocado_qemu.Test class

2024-08-21 Thread Thomas Huth
The avocado_qemu.Test class has been renamed a while back in commit 2283b627bc ("tests/avocado: Rename avocado_qemu.Test -> QemuSystemTest"), so we should reflect this now in the documentation, too. Signed-off-by: Thomas Huth --- docs/devel/testing/avo

[PATCH v4 30/35] tests/functional: Convert the linux_initrd avocado test into a standalone test

2024-08-21 Thread Thomas Huth
While we're at it, update the SHA1 hashes to SHA256 hashes since SHA1 should not be used anymore nowadays. Signed-off-by: Thomas Huth --- tests/functional/meson.build | 1 + .../test_linux_initrd.py} | 37 +++ 2 files changed, 22 inser

[PATCH v4 24/35] tests/functional: Convert most ppc avocado tests into standalone tests

2024-08-21 Thread Thomas Huth
Nothing thrilling in here, just straight forward conversions. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 9 +-- tests/functional/meson.build | 11 .../test_ppc64_powernv.py}| 56

[PATCH v4 19/35] tests/functional: Convert the s390x avocado tests into standalone tests

2024-08-21 Thread Thomas Huth
ymore nowadays. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 4 +- tests/functional/meson.build | 6 ++ tests/functional/qemu_test/utils.py | 14 +++ .../test_s390x_ccw_virtio.py}

[PATCH v4 29/35] tests/functional: Convert the rx_gdbsim avocado test into a standalone test

2024-08-21 Thread Thomas Huth
Provide a "gzip_uncompress" function based on the standard "gzip" module to avoid the usage of avocado.utils here. Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 2

[PATCH v4 22/35] tests/functional: Convert the riscv_opensbi avocado test into a standalone test

2024-08-21 Thread Thomas Huth
Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS| 1 + tests/avocado/riscv_opensbi.py | 63 -- tests/functional/meson.build | 8 tests/functional/test_riscv_opensbi.py | 36

[PATCH v4 11/35] tests/functional: Prepare the meson build system for the functional tests

2024-08-21 Thread Thomas Huth
ed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/Makefile.include | 11 ++ tests/functional/meson.build | 66 tests/meson.build| 1 + 3 files changed, 78 insertions(+) create mode 10064

[PATCH v4 17/35] tests/functional: Add a function for extracting files from an archive

2024-08-21 Thread Thomas Huth
with this functionality, so let's just add a nice wrapper function around that. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Tested-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/functional/qemu_test/utils.py | 21 ++

[PATCH v4 13/35] tests/functional: Convert avocado tests that just need a small adjustment

2024-08-21 Thread Thomas Huth
ed-by: Daniel P. Berrangé Tested-by: Philippe Mathieu-Daudé Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 3 + tests/functional/meson.build | 6 ++ .../test_info_usernet.py} | 11

[PATCH v4 12/35] tests/functional: Convert simple avocado tests into standalone python tests

2024-08-21 Thread Thomas Huth
em directly. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 5 ++ tests/functional/meson.build | 5 ++ .../test_cpu_queries.py}

[PATCH v4 23/35] tests/functional: Convert the virtio_gpu avocado test into a standalone test

2024-08-21 Thread Thomas Huth
Nothing thrilling in here, it's just a straight forward conversion. Reviewed-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- tests/functional/meson.build | 1 + .../test_virtio_gpu.py} | 64 +-- 2 files changed, 30 inser

[PATCH v4 27/35] tests/functional: Convert the m68k nextcube test with tesseract

2024-08-21 Thread Thomas Huth
thieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- MAINTAINERS | 1 + tests/avocado/tesseract_utils.py | 46 --- tests/functional/meson.build | 4 ++ tests/functional/qemu_test/tessera

[PATCH v4 21/35] tests/functional: Convert the microblaze avocado tests into standalone tests

2024-08-21 Thread Thomas Huth
Signed-off-by: Thomas Huth --- MAINTAINERS | 2 +- tests/avocado/machine_microblaze.py | 61 --- tests/functional/meson.build | 8 +++ .../functional/test_microblaze_s3adsp1800.py | 40

[PATCH v4 32/35] docs/devel: Split testing docs from the build docs and move to separate folder

2024-08-21 Thread Thomas Huth
Building and testing are two separate topics, so let's split the testing into a separate category and move the related files into a separate folder. Signed-off-by: Thomas Huth --- docs/devel/index-build.rst | 14 -- docs/devel/inde

[PATCH v4 02/35] tests/avocado/boot_xen.py: fetch kernel during test setUp()

2024-08-21 Thread Thomas Huth
by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-7-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/boot_xen.py | 13 ++--- 1 file changed, 6 insertions(+), 7 deletions(-) diff --git a/tests/avocado/boot_xen.py b/tests/avocado/boot_xen.py index f29bc58b9e..4

[PATCH v4 18/35] tests/functional: Convert some avocado tests that needed avocado.utils.archive

2024-08-21 Thread Thomas Huth
Instead of using the "archive" module from avocado.utils, switch these tests to use the new wrapper function that is based on the "tarfile" module instead. Reviewed-by: Daniel P. Berrangé Reviewed-by: Philippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé Signed

[PATCH v4 00/35] Convert avocado tests to normal Python unittests

2024-08-21 Thread Thomas Huth
FC Cleber Rosa (4): tests/avocado: machine aarch64: standardize location and RO access tests/avocado/boot_xen.py: fetch kernel during test setUp() tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image Bump avocado to 103.0 Daniel P. Berrangé (2): tests/functional: add

[PATCH v4 03/35] tests/avocado/machine_aarch64_sbsaref.py: allow for rw usage of image

2024-08-21 Thread Thomas Huth
ewicz Message-ID: <20240806173119.582857-9-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/machine_aarch64_sbsaref.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/avocado/machine_aarch64_sbsaref.py b/tests/avocado/machine_aarch64_sbsaref.py index

[PATCH v4 09/35] tests/functional: Set up logging

2024-08-21 Thread Thomas Huth
Create log files for each test separately, one file that contains the basic logging and one that contains the console output. Reviewed-by: Daniel P. Berrangé Signed-off-by: Thomas Huth --- tests/functional/qemu_test/testcase.py | 27 +- 1 file changed, 26 insertions

[PATCH v4 04/35] Bump avocado to 103.0

2024-08-21 Thread Thomas Huth
://avocado-framework.readthedocs.io/en/103.0/releases/lts/103_0.html Signed-off-by: Cleber Rosa Tested-by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-2-cr...@redhat.com> Signed-off-by: Thomas Huth --- pythondeps.toml| 2 +- tests/Makefile.include | 2 +- 2 files changed, 2 inse

[PATCH v4 16/35] tests/functional: Convert some tests that download files via fetch_asset()

2024-08-21 Thread Thomas Huth
Now that we've got the Asset class with pre-caching, we can convert some Avocado tests that use fetch_asset() for downloading their required files. Signed-off-by: Thomas Huth --- MAINTAINERS | 12 ++--- tests/avocado/machine_mips_loongson3v.py

[PATCH v4 14/35] tests/functional: add a module for handling asset download & caching

2024-08-21 Thread Thomas Huth
hilippe Mathieu-Daudé Tested-by: Philippe Mathieu-Daudé [thuth: Drop sha1 support, use hash on file content for naming instead of URL, add the possibility to specify the cache dir via environment variable] Signed-off-by: Thomas Huth --- tests/functional/qemu_test/__init__.py | 1 + t

[PATCH v4 15/35] tests/functional: enable pre-emptive caching of assets

2024-08-21 Thread Thomas Huth
ys done by make check-functional Signed-off-by: Daniel P. Berrangé Tested-by: Richard Henderson [thuth: Remove the duplicated "path = os.path.basename(...)" line] Signed-off-by: Thomas Huth --- tests/Makefile.include | 3 ++- tests/fu

[PATCH v4 01/35] tests/avocado: machine aarch64: standardize location and RO access

2024-08-21 Thread Thomas Huth
y: Cleber Rosa Tested-by: Marcin Juszkiewicz Message-ID: <20240806173119.582857-5-cr...@redhat.com> Signed-off-by: Thomas Huth --- tests/avocado/machine_aarch64_sbsaref.py | 2 +- tests/avocado/machine_aarch64_virt.py| 14 +++--- 2 files changed, 8 insertions(+), 8 deletions

[PATCH v4 06/35] tests/avocado/boot_linux_console: Remove the s390x subtest

2024-08-21 Thread Thomas Huth
py. Signed-off-by: Thomas Huth --- tests/avocado/boot_linux_console.py | 20 tests/avocado/machine_s390_ccw_virtio.py | 2 ++ 2 files changed, 2 insertions(+), 20 deletions(-) diff --git a/tests/avocado/boot_linux_console.py b/tests/avocado/boot_linux_console.py i

[PATCH v2] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-20 Thread Thomas Huth
failure of the qtests on MSYS2, we can revert this patch to get everything back. Suggested-by: Philippe Mathieu-Daudé Signed-off-by: Thomas Huth --- This decreases the runtime of the job from ~ 70 minutes to ~ 45 minutes for me. .gitlab-ci.d/windows.yml | 26 ++ 1

Re: [PULL 3/5] tests/avocado: apply proper skipUnless decorator

2024-08-20 Thread Thomas Huth
On 20/08/2024 10.03, Thomas Huth wrote: On 20/08/2024 00.31, Philippe Mathieu-Daudé wrote: On 16/8/24 09:22, Thomas Huth wrote: From: Cleber Rosa Commit 9b45cc993 added many cases of skipUnless for the sake of organizing flaky tests.  But, Python decorators *must* follow what they decorate

Re: [PULL 3/5] tests/avocado: apply proper skipUnless decorator

2024-08-20 Thread Thomas Huth
On 20/08/2024 00.31, Philippe Mathieu-Daudé wrote: On 16/8/24 09:22, Thomas Huth wrote: From: Cleber Rosa Commit 9b45cc993 added many cases of skipUnless for the sake of organizing flaky tests.  But, Python decorators *must* follow what they decorate, so the newlines added should *not* exist

Re: [PATCH] MAINTAINERS: Remove myself as reviewer

2024-08-19 Thread Thomas Huth
On 19/08/2024 17.00, Beraldo Leal wrote: Finally taking this off my to-do list. It’s been a privilege to be part of this project, but I am no longer actively involved in reviewing Python code here, so I believe it's best to update the list to reflect the current maintainers. Please, feel free to

Re: [PATCH] gitlab-ci: Build MSYS2 job using multiple CPUs

2024-08-19 Thread Thomas Huth
According to https://docs.gitlab.com/ee/ci/runners/hosted_runners/windows.html the Windows shared runner should have 2 vCPUs nowadays, indeed! Maybe worth to mention it in the patch description? Also, how much faster does the job now run for you? On 19/08/2024 13.21, Philippe Mathieu-Daudé

Re: [PATCH] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-19 Thread Thomas Huth
On 19/08/2024 12.21, Philippe Mathieu-Daudé wrote: On 19/8/24 07:30, Thomas Huth wrote: On 16/08/2024 19.18, Philippe Mathieu-Daudé wrote: On 16/8/24 18:40, Thomas Huth wrote: On 16/08/2024 18.34, Philippe Mathieu-Daudé wrote: On 16/8/24 17:37, Thomas Huth wrote: The qtests are broken since

Re: [PATCH] gitlab-ci: Replace build_script -> step_script in Cirrus jobs

2024-08-18 Thread Thomas Huth
--hard "$CI_COMMIT_SHA" - build_script: + step_script: - mkdir build - cd build - ../configure --enable-werror $CONFIGURE_ARGS Reviewed-by: Thomas Huth

Re: [PATCH] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-18 Thread Thomas Huth
On 16/08/2024 19.18, Philippe Mathieu-Daudé wrote: On 16/8/24 18:40, Thomas Huth wrote: On 16/08/2024 18.34, Philippe Mathieu-Daudé wrote: On 16/8/24 17:37, Thomas Huth wrote: The qtests are broken since a while in the MSYS2 job in the gitlab-CI, likely due to some changes in the MSYS2

Re: [PATCH] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-16 Thread Thomas Huth
On 16/08/2024 18.34, Philippe Mathieu-Daudé wrote: On 16/8/24 17:37, Thomas Huth wrote: The qtests are broken since a while in the MSYS2 job in the gitlab-CI, likely due to some changes in the MSYS2 environment. So far nobody has neither a clue what's going wrong here, nor an idea how t

Re: [PATCH] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-16 Thread Thomas Huth
On 16/08/2024 18.21, Alex Bennée wrote: Thomas Huth writes: The qtests are broken since a while in the MSYS2 job in the gitlab-CI, likely due to some changes in the MSYS2 environment. So far nobody has neither a clue what's going wrong here, nor an idea how to fix this (in fact most

[PATCH] .gitlab-ci.d/windows.yml: Disable the qtests in the MSYS2 job

2024-08-16 Thread Thomas Huth
ilable for properly analyzing this problem), so let's disable the qtests here again to get at least the test coverage for the compilation and unit tests back to the CI. Signed-off-by: Thomas Huth --- .gitlab-ci.d/windows.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitlab-ci.d/window

Re: [PATCH v2 3/9] tests/avocado: add cdrom permission related tests

2024-08-16 Thread Thomas Huth
On 06/08/2024 19.31, Cleber Rosa wrote: The "media=cdrom" parameter is also used on some Avocado tests as a way to protect files from being written. The tests here bring a more fundamental check that this behavior can be trusted. Signed-off-by: Cleber Rosa --- tests/avocado/cdrom.py | 41 +++

Re: [PULL for 9.1 00/21] Some fixes for 9.1-rc3 (build, replay, docs, plugins)

2024-08-16 Thread Thomas Huth
On 16/08/2024 14.54, Alex Bennée wrote: Richard Henderson writes: On 8/16/24 00:48, Alex Bennée wrote: The following changes since commit b0d6c037ea0a825192285f149af72a73a1baaa04: Update version for v9.1.0-rc2 release (2024-08-15 20:32:37 +1000) are available in the Git repository at:

Re: [PATCH v3] meson: Use -fno-sanitize=function when available

2024-08-16 Thread Thomas Huth
On 16/08/2024 10.27, Akihiko Odaki wrote: On 2024/08/16 17:24, Thomas Huth wrote: On 16/08/2024 10.21, Akihiko Odaki wrote: On 2024/08/16 17:03, Thomas Huth wrote: On 16/08/2024 09.30, Akihiko Odaki wrote: On 2024/08/16 16:27, Thomas Huth wrote: On 16/08/2024 09.12, Akihiko Odaki wrote: On

<    1   2   3   4   5   6   7   8   9   10   >