On 01/05/2019 18.37, Alex Bennée wrote: > > Thomas Huth <[email protected]> writes: > >> Currently, all tests are in the "auto" group. This is a little bit pointless. >> OTOH, we need a group for the tests that we can automatically run during >> "make check" each time, too. Tests in this new group are supposed to run >> with every possible QEMU configuration, for example they must run with every >> QEMU binary (also non-x86), without failing when an optional features >> is > > I'm curious as to what tests fail on non-x86? I was under the naive > impression that at least for Linux all the file-system relevant bits > where fairly uniformly implemented. > >> missing (but reporting "skip" is ok), and be able to run on all kind of host >> filesystems and users (i.e. also as "nobody" or "root"). >> So let's use the "auto" group for this class of tests now. The initial >> list has been determined by running the iotests with non-x86 QEMU targets >> and with our CI pipelines on Gitlab, Cirrus-CI and Travis (i.e. including >> macOS and FreeBSD). >> >> Signed-off-by: Thomas Huth <[email protected]> > > > >> --- >> tests/qemu-iotests/group | 169 +++++++++++++++++++++------------------ >> 1 file changed, 91 insertions(+), 78 deletions(-) >> >> diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group >> index bae7718380..4d0b6ae256 100644 >> --- a/tests/qemu-iotests/group >> +++ b/tests/qemu-iotests/group >> @@ -1,8 +1,21 @@ >> # >> # QA groups control file >> # Defines test groups >> +# >> +# Some notes about the groups: >> +# >> # - do not start group names with a digit >> # >> +# - quick : Tests in this group should finish within some few seconds. >> +# >> +# - img : Tests in this group can be used to excercise the qemu-img tool. >> +# >> +# - auto : Tests in this group are used during "make check" and should be >> +# runnable in any case. That means they should run with every QEMU binary >> +# (also non-x86), with every QEMU configuration (i.e. must not fail if >> +# an optional feature is not compiled in - but reporting a "skip" is ok), >> +# and work all kind of host filesystems and users (e.g. "nobody" or >> "root"). >> +# >> >> # >> # test-group association ... one line per test >> @@ -32,11 +45,11 @@ >> 023 rw auto >> 024 rw backing auto quick >> 025 rw auto quick >> -026 rw blkdbg auto >> +026 rw blkdbg >> 027 rw auto quick >> 028 rw backing auto quick > > 028 failed on an armhf host for me. > >> 029 rw auto quick >> -030 rw auto backing >> +030 rw backing > <snip> >> +245 rw >> +246 rw quick >> 247 rw auto quick > > And 247
Ok, thanks, I'll remove them in v3. (We still can investigate and enable them later if we care). Thomas
