On Tue, 8 Oct 2019 at 22:49, Cleber Rosa <cr...@redhat.com> wrote: > On Mon, Oct 07, 2019 at 05:28:49PM +0100, Peter Maydell wrote: > > Do our other acceptance tests download random third-party > > (ie "not a well-known distro") binaries for the tests ? > > It seems a bit hazardous for reproducability and trustability > > reasons...
> A quick and dirty grep shows (excluding comments and docs): > > boot_linux_console.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive/fedora' > boot_linux_console.py: deb_url = > ('http://snapshot.debian.org/archive/debian/' > boot_linux_console.py: deb_url = > ('http://snapshot.debian.org/archive/debian/' > boot_linux_console.py: deb_url = > ('http://snapshot.debian.org/archive/debian/' > boot_linux_console.py: initrd_url = > ('https://github.com/groeck/linux-build-test/raw/' > boot_linux_console.py: kernel_url = > ('https://mipsdistros.mips.com/LinuxDistro/nanomips/' > boot_linux_console.py: kernel_url = > ('https://mipsdistros.mips.com/LinuxDistro/nanomips/' > boot_linux_console.py: kernel_url = > ('https://mipsdistros.mips.com/LinuxDistro/nanomips/' > boot_linux_console.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive/fedora' > boot_linux_console.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive/fedora' > boot_linux_console.py: uboot_url = > ('https://raw.githubusercontent.com/' > boot_linux_console.py: spi_url = > ('https://raw.githubusercontent.com/' > boot_linux_console.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive' > boot_linux_console.py: kernel_url = > ('http://archive.debian.org/debian/dists/lenny/main/' > boot_linux_console.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive' > linux_initrd.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive/fedora/li' > linux_initrd.py: kernel_url = > ('https://archives.fedoraproject.org/pub/archive/fedora' > linux_ssh_mips_malta.py: 'be': {'image_url': > ('https://people.debian.org/~aurel32/qemu/mips/' > linux_ssh_mips_malta.py: 'le': {'image_url': > ('https://people.debian.org/~aurel32/qemu/mipsel/' > linux_ssh_mips_malta.py: kernel_url = > ('https://people.debian.org/~aurel32/qemu/mips/' > linux_ssh_mips_malta.py: kernel_url = > ('https://people.debian.org/~aurel32/qemu/mipsel/' > linux_ssh_mips_malta.py: kernel_url = > ('https://people.debian.org/~aurel32/qemu/mips/' > linux_ssh_mips_malta.py: kernel_url = > ('https://people.debian.org/~aurel32/qemu/mipsel/' > machine_m68k_nextcube.py: rom_url = > ('http://www.nextcomputers.org/NeXTfiles/Software/ROM_Files/' > > I find it hard to judge precisely how much of a third-party some of > these are. I remember Philippe mentioning that one of them, I guess > the images used on linux_ssh_mips_malta.py, were "as official as it > gets" (my words, from my often misleading memory). > > Reproducibility is definitely an issue, in the sense given that some > of these can indeed go away, but as long as they're available the hash > recorded in the test should guarantee that we're running the same > images. > > Do you think we should do something different here? I'm not sure, which is why I asked whether this new test was in line with what we've done previously. Since these are just test cases and we don't redistribute them to other people there's less of a traceability/reproducibility worry, and if we check hashes on download that cuts off a lot of "fail to notice if the image changes for some reason" possible problems. thanks -- PMM