On 02/10/2025 16.04, Kevin Wolf wrote:
Am 10.09.2025 um 17:37 hat Thomas Huth geschrieben:
From: Thomas Huth <th...@redhat.com>
When running the tests in thorough mode, e.g. with:
make -j$(nproc) check SPEED=thorough
we currently always get a huge amount of total tests that the test
runner tries to execute (2457 in my case), but a big bunch of them are
only skipped (1099 in my case, meaning that only 1358 got executed).
This happens because we try to run the whole set of iotests for multiple
image formats while a lot of the tests can only run with one certain
format only and thus are marked as SKIP during execution. This is quite a
waste of time during each test run, and also unnecessarily blows up the
displayed list of executed tests in the console output.
Thus let's try to be a little bit smarter: If the "check" script is run
with "-n" and an image format switch (like "-qed") at the same time (which
is what we do already for discovering the tests for the meson test runner),
only report the tests that likely support the given format instead of
providing the whole list of all tests. We can determine whether a test
supports a format or not by looking at the lines in the file that contain
a "supported_fmt" or "unsupported_fmt" statement. This is only heuristics,
of course, but it is good enough for running the iotests via "make
check-block" - I double-checked that the list of executed tests does not
get changed by this patch, it's only the tests that are skipped anyway that
are now not run anymore.
This way the amount of total tests drops from 2457 to 1432 for me, and
the amount of skipped tests drops from 1099 to just 74 (meaning that we
still properly run 1432 - 74 = 1358 tests as we did before).
Signed-off-by: Thomas Huth <th...@redhat.com>
---
tests/qemu-iotests/check | 35 ++++++++++++++++++++++++++++++++---
1 file changed, 32 insertions(+), 3 deletions(-)
+def dry_run_list(test_dir, imgfmt, testlist):
+ for t in testlist:
+ if not imgfmt:
+ print('\n'.join([os.path.basename(t)]))
+ continue
+ # If a format has been given, we look for the "supported_fmt"
+ # and the "unsupported_fmt" lines in the test and try to find out
+ # whether the format is supported or not. This is only heuristics,
+ # but it should be good enough for "make check-block"
I'm not completely sure if this is a good idea at all, but I think we
should at least mention the possible surprising cases where the
heuristics fails in this comment.
Yeah, it's not perfect, but I also failed to come up with a better
solution... do you have any other ideas?
If not, would you be fine with the patch if I respin it with the comment
here extended with some words about those surprising cases?
Thomas
