The recent set of regressions identified in the LUKS block driver re-inforced that despite having a hugely useful set of I/O tests, our CI coverage is still letting through too many bugs.
This series expands the meson suites / make targets for running I/O tests for more drivers, and then creates a CI job for each. While the first three patches are functionally usable as is, the last patch for gitlab is very much an RFC. A test pipeline https://gitlab.com/berrange/qemu/-/pipelines/2085907042 shows passes for qcow2, qed, vmdk & vpc, but failures for luks, nbd and raw. The luks failures all have patchs floating around pending merge. The nbd failures are problems with non-deterministic output ordering, or warnings from python deprecations that need addressing. The raw failures are problems with python deprecations, and insufficient permissions checks wrt device mapper acess. I'm unclear if there are other non-deterministic failures that would only appear if we start running the tests regularly. That is a chicken & egg problem though. At some point we have to accept a (hopefully short) period of instability to tease out the rare edge cases, in order to get a long term win of better code quality through CI coverage. Daniel P. Berrangé (4): tests: print reason when I/O test is skipped in TAP mode tests: rearrange suites for I/O tests tests: add nbd and luks to the I/O test suites gitlab: add jobs for thorough block tests .gitlab-ci.d/buildtest.yml | 63 ++++++++++++++++++++++++++++++++ tests/qemu-iotests/meson.build | 52 ++++++++++++++++++++------ tests/qemu-iotests/testrunner.py | 2 +- 3 files changed, 104 insertions(+), 13 deletions(-) -- 2.50.1
