When there are many tests and/or many log lines, it is difficult to
quickly find out how the different tests went when running:

    make -C tools/testing/selftests run_tests

This run_tests target executes 'run_many' from runner.sh, which relies
on ktap_helpers.sh. At the end of the execution, the KTAP_CNT_* counters
have been populated, and the total can then be simply printed with the
ktap_print_totals helper.

Now a summary is printed at the end, e.g. here with the two last lines:

    $ make -sC tools/testing/selftests TARGETS=(...) run_tests
    # timeout set to (...)
    # selftests: (...)
    (...)
    ok 1 selftests: (...)
    # timeout set to (...)
    # selftests: (...)
    (...)
    ok 2 selftests: (...) # SKIP
    # timeout set to (...)
    # selftests: (...)
    (...)
    not ok 3 selftests: (...)
    # 1 skipped test(s) detected.  Consider enabling relevant config options to 
improve coverage.
    # Totals: pass:1 fail:1 xfail:0 xpass:0 skip:1 error:0

This summary is printed, using '#' as prefix to be treated as a comment
in the TAP format. The summary is also printed when summary=1 is used to
redirect log lines in different files because it still looks useful when
there are many tests, not to have to check each line.

Note that this 'make' command will continue to exit without errors
(returned code is 0) when the tests can be executed. I guess it is
better not to change this behaviour to be able to continue detecting
kselftests failures.

Signed-off-by: Matthieu Baerts (NGI0) <[email protected]>
---
 tools/testing/selftests/lib.mk | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index f02cc8a2e4ae..de15e63bc6b4 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -106,7 +106,8 @@ define RUN_TESTS
        if [ "X$(summary)" != "X" ]; then       \
                per_test_logging=1;             \
        fi;                                     \
-       run_many $(1)
+       run_many $(1);                          \
+       ktap_print_totals
 endef
 
 define INSTALL_INCLUDES

---
base-commit: fbb7ad31ab376c5101b2ac7205fad0344fd2de60
change-id: 20260625-sft-run-tests-print-totals-a18eea814bee

Best regards,
--  
Matthieu Baerts (NGI0) <[email protected]>


Reply via email to