On Wed, Jul 26, 2023 at 3:34 AM Philippe Mathieu-Daudé <phi...@linaro.org> wrote: > > On 25/7/23 19:13, Peter Maydell wrote: > > Currently this CI job is failing: > > > > https://gitlab.com/qemu-project/qemu/-/jobs/4737819946 > > > > because: > > > > (05/59) > > tests/avocado/boot_linux_console.py:BootLinuxConsole.test_arm_exynos4210_initrd: > > INTERRUPTED: Missing asset > > https://snapshot.debian.org/archive/debian/20190928T224601Z/pool/main/l/linux/linux-image-4.19.0-6-armmp_4.19.67-2+deb10u1_armhf.deb\nRunner > > error occurred: Timeout reached\nOriginal status: CANCEL\n{'name': > > '05-tests/avocado/boot_linux_console... (90.67 s) > > > > Why is a "Missing asset" causing a timeout after 90 seconds, > > rather than being accounted as a "SKIP" ("missing requirements > > in the test environment" sounds like what we have here) ? > > Maybe something to report to upstream Avocado. > >
Hi Philippe, Please check my response to Peter's first message on this thread. It's a rather long answer, but I hope this behavior is understandable. > That said, we want CI to be reproducible. If we fetch assets from > unreliable sources, we can't be reproducible. If we are unable to > provide a assets cache, we'll keep hitting this problem. If we can > not find a way to have assets stored (requiring sysadmin time setting > up some infra, possibly only for GitLab), then I'd consider stopping > running tests depending on external assets on CI; otherwise at some > point we'll get tired to waste time figuring out the same problem. > Right, in an ideal world, we could have a master list of all the assets that every single job will need, and require an admin to make sure each and every one of them is cached before running a job. The current approach with Avocado's pre-job "fetch asset" plugin is to do as much as possible without having duplication of the assets URLs in such a "asset master list" and in the test code. Also, it will not abort a job if any of these assets fail to be fetched. It's a convenient choice that, on the other hand, yields lower reproducibility and reliability. > As a maintainer I'm happy to run the avocado tests using my local > assets cache, and I would rather keep using the framework. But then > my cache is likely different from others (users, maintainers, CI). > Similarly few users/maintainers end up having the same cache and > running the same set of tests. > > $ du -chs ~/avocado/data/cache/ > 5.7G /Users/philmd/avocado/data/cache/ > > Some files are older than 3 years, and I'm happy to still run the > tests depending on them (although they disappeared from their > original http server). > This is a well maintained cache! :) Can we rsync from it? :) Jokes aside, I'm open for ideas on how to better balance this convenience versus reliability question. Thanks, - Cleber.