On Fri, Mar 03, 2023 at 08:53:32AM +0000, Daniel P. Berrangé wrote: > On Fri, Mar 03, 2023 at 09:30:39AM +0100, Thomas Huth wrote: > > On 02/03/2023 19.46, Daniel P. Berrangé wrote: > > > To just repeat the patch 5 description... > > > > > > Currently meson registers a single test that invokes an entire group of > > > I/O tests, hiding the test granularity from meson. There are various > > > downsides of doing this > > > > > > * You cannot ask 'meson test' to invoke a single I/O test > > > * The meson test timeout can't be applied to the individual > > > tests > > > * Meson only gets a pass/fail for the overall I/O test group > > > not individual tests > > > * If a CI job gets killed by the GitLab timeout, we don't > > > get visibility into how far through the I/O tests > > > execution got. > > > > > > This is not really specific to the I/O tests, the problem is common > > > to any case of us running a test which is in fact another test > > > harness which runs many tests. It would be nice to have meson have > > > the full view of all tests run. Adapting the I/O tests is as easy > > > win in this respect. > > > > > > This switches meson to perform test discovery by invoking 'check' in > > > dry-run mode. It then registers one meson test case for each I/O > > > test. Parallel execution remains disabled since the I/O tests do not > > > use self contained execution environments and thus conflict with > > > each other. > > > > Great to see some movement in this area again! > > > > Some questions/remarks: > > > > 1) Could you remove tests/check-block.sh now? See also: > > https://lore.kernel.org/all/20220209101530.3442837-9-th...@redhat.com/ > > Possibly, I wasn't sure if that was wanted as a general entry > point for humans, or was solely for meson ? > > > 2) With regards to parallel execution ... I think it should be > > possible nowadays - the "check" script is normally also run > > with the "-j" switch by the tests/check-block.sh script, so > > if you remove the possibility to run in parallel, it's a > > regression from the previous behavior! > > Hmmm, I got *masses* of test failures when running in parallel > but I'll check again to be sure.
Ooooh, the 'check' script only uses a unique sub-dir for tests when passing -j with a value >= 2. It also merely appends the test name, so it is still broken for anyone who runs multiple copies of 'check' in parallel for different configurations, even if they passed -j. It needs to just always use an isolated subdir no matter what, with process level uniqueness, not merely test. With regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
