On Tue, 21 Feb 2023 at 15:36, Dr. David Alan Gilbert <dgilb...@redhat.com> wrote: > > * Peter Maydell (peter.mayd...@linaro.org) wrote: > > On Tue, 21 Feb 2023 at 15:21, Dr. David Alan Gilbert > > <dgilb...@redhat.com> wrote: > > > Damn this is really going to impact the stability of migration if we > > > don't regularly test. > > > But fundamentally, I've never been able to debug much of the reports > > > that come from flakyness in gitlab ci; we're not getting the most basic > > > information like which subtest or where we're upto in the test which > > > makes it very very hard to debug. > > > > Right, but if you want more information you need to change the > > tests and/or test harness to provide it. > > I don't think the migration test is doing anything odd in that respect; > We've just got a bunch of qtest tests; having a test framework which > doesn't tell you which test failed is very difficult.
In the specific case here of the FreeBSD job it does, though you have to search backwards for 'migration' to find it: https://gitlab.com/qemu-project/qemu/-/jobs/3806090216 It prints ok 23 /aarch64/migration/multifd/tcp/plain/none # starting QEMU: exec ./qemu-system-aarch64 -qtest unix:/tmp/qtest-32469.sock -qtest-log /dev/null -chardev socket,path=/tmp/qtest-32469.qmp,id=char0 -mon chardev=char0,mode=control -display none -accel kvm -accel tcg -machine virt,gic-version=max -name source,debug-threads=on -m 150M -serial file:/tmp/migration-test-2A1201/src_serial -cpu max -kernel /tmp/migration-test-2A1201/bootsect 2>/dev/null -accel qtest # starting QEMU: exec ./qemu-system-aarch64 -qtest unix:/tmp/qtest-32469.sock -qtest-log /dev/null -chardev socket,path=/tmp/qtest-32469.qmp,id=char0 -mon chardev=char0,mode=control -display none -accel kvm -accel tcg -machine virt,gic-version=max -name target,debug-threads=on -m 150M -serial file:/tmp/migration-test-2A1201/dest_serial -incoming defer -cpu max -kernel /tmp/migration-test-2A1201/bootsect 2>/dev/null -accel qtest # starting QEMU: exec ./qemu-system-aarch64 -qtest unix:/tmp/qtest-32469.sock -qtest-log /dev/null -chardev socket,path=/tmp/qtest-32469.qmp,id=char0 -mon chardev=char0,mode=control -display none -accel kvm -accel tcg -machine virt,gic-version=max -name target,debug-threads=on -m 150M -serial file:/tmp/migration-test-2A1201/dest_serial -incoming defer -cpu max -kernel /tmp/migration-test-2A1201/bootsect -accel qtest Bail out! ERROR:../tests/qtest/migration-helpers.c:205:wait_for_migration_status: assertion failed: (g_test_timer_elapsed() < MIGRATION_STATUS_WAIT_TIMEOUT) so it has successfully run tests up to 23, and then crashed on test 24 (which will be migration/multifd/tcp/plain/cancel). The test framework also collects and logs any information the test case wants to print to stderr -- the reason there is no information there other than the assertion message is because that's the only thing the test emits. The local macos test log I have here also shows it failing on test 24. The macos tests are easy to provoke failures here during 'make check', it happens at least 1 time in 2. So if you want to write a patch to print out more information to the test log I'm happy to run it and collect the output. thanks -- PMM
