On 4/30/25 9:39 AM, Daniel P. Berrangé wrote:
On Wed, Apr 30, 2025 at 09:34:10AM -0700, Pierrick Bouvier wrote:
On 4/30/25 9:29 AM, Daniel P. Berrangé wrote:
On Wed, Apr 30, 2025 at 09:21:41AM -0700, Pierrick Bouvier wrote:
On 4/30/25 9:02 AM, Daniel P. Berrangé wrote:
On Wed, Apr 30, 2025 at 08:48:59AM -0700, Pierrick Bouvier wrote:
On 4/30/25 8:00 AM, Thomas Huth wrote:
On 30/04/2025 16.34, Pierrick Bouvier wrote:
Hi folks,
$ ninja -C build precache-functional
2025-04-30 07:23:20,382 - qemu-test - ERROR - Unable to download https://
archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/
gzimg/armv7.img.gz: HTTP error 503
2025-04-30 07:23:23,131 - qemu-test - ERROR - Unable to download https://
archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/
gzimg/armv7.img.gz: HTTP error 503
2025-04-30 07:23:25,870 - qemu-test - ERROR - Unable to download https://
archive.netbsd.org/pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/
gzimg/armv7.img.gz: HTTP error 503
2025-04-30 07:23:25,871 - qemu-test - ERROR - https://archive.netbsd.org/
pub/NetBSD-archive/NetBSD-9.0/evbarm-earmv7hf/binary/gzimg/armv7.img.gz:
Download retries exceeded: skipping asset precache
$ echo $?
0
Since we silently skip the asset precaching, how can we identify that an
asset is not available anymore (temporarily or not)?
Should we rely on test itself failing when trying to download again this asset?
The current logic fails hard for 404 errors, so if the asset is completely
gone, we should notice it. For other error codes, we assume that it is only
a temporary server problem that will hopefully be fixed on the server side
sooner or later.
Sounds good.
Should we replicate this semantic when running the test itself?
It would be more useful to skip it because an asset is missing instead of
reporting an error, except if it's a 404 error.
The tests already gracefully skip if one or more required assets
are not available. See the 'setUp' method of QemuBaseTest
if not self.assets_available():
self.skipTest('One or more assets is not available')
In the 404 case, the pre-cache step should fail and thus we shouldn't
even get to running the test.
This is not the behaviour I observe (error, with server returning 503) [1],
thus my original email.
Maybe something is missing in the associated test, or in our test
infrastructure?
Or... in my command :)
Nothing funky in the command line used, you can reproduce it with:
$ rm -rf ~/.cache/qemu build/
$ ./configure
$ ./build/pyvenv/bin/meson test -C build --setup thorough --suite func-quick
--suite func-thorough -t 5 --print-errorlogs func-ppc-ppc_40p
Oh, you're running meson test directly.
The behaviour I describe is wrt the official way of running tests via
'make check' or 'make check-functional'.
When you use 'make', we set 'QEMU_TEST_NO_DOWNLOAD=1' when the tests
themselves are run, so only the 'make precache-functional' will be
permitted to try downloading.
Oh thanks, that's what I was missing!
I'm running meson because the Makefile wrapper does not allow to pass any
additional parameters, or running specific test.
FWIW, if you want to run a specific test, personally don't use meson
or make, as you can just invoke the file directly:
$ QEMU_TEST_QEMU_BINARY=./build/qemu-system-x86_64 \
PYTHONPATH=./python \
./tests/functional/test_x86_cpu_model_versions.py
This was the key feature I wanted when we replaced avocado, as debugging
tests without a harness getting in the way is much simpler
Sounds good, thanks Daniel.
I usually find meson easier to run, since it will build the code also,
and allow to list all tests easily. I just think the behaviour of
missing assets relying on precache-functional is a bit awkward (it would
be more intuitive to gracefully skip the test without needing a special
env var), but since I'm running an undocumented workflow, I won't insist.
Thanks for all the information.
With regards,
Daniel
Regards,
Pierrick
