Ping?
Nathan Lynch <[email protected]> writes: > How is the babeltrace test suite on the master and stable-2.0 branches > intended to be used? > > Effectively there are four test suites, one for each definition of TESTS > in the tests/ directory. If one of them has a failure, 'make check' > will not proceed to the next unless the -k/--keep-going flag is used. > Is this intended? I unwittingly was running only the tests under > tests/cli (some of which always fail) until I discovered this. > > Defining multiple test suites this way seems to be unusual; other > projects I've checked just have one top-level definition of TESTS. > > I'm aware of the rationale for doing it this way: > > commit 6ca1931cb32ca2eb33252896d2a42a4c48af436a > Author: Philippe Proulx <[email protected]> > Date: Fri May 5 16:45:11 2017 -0400 > > tests: put TESTS list in each Makefile.am > > So that you can do `make check` in any subdirectory to run only those > tests. > > But it still feels non-idiomatic to me, and I'm wondering if there's a > better way. There are other ways to limit the set of tests to run, > detailed here: > > https://www.gnu.org/software/automake/manual/html_node/Parallel-Test-Harness.html > > Here are my current results from running all test suites; let me know if > you want more detail and I'll follow up; there are likely several > different issues to investigate. > > $ find tests/ -name test-suite.log -exec head {} + > ==> tests/lib/test-suite.log <== > ===================================================== > babeltrace 2.0.0-pre1: tests/lib/test-suite.log > ===================================================== > > # TOTAL: 1200 > # PASS: 1195 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 1 > # XPASS: 0 > > ==> tests/bindings/python/bt2/test-suite.log <== > ===================================================================== > babeltrace 2.0.0-pre1: tests/bindings/python/bt2/test-suite.log > ===================================================================== > > # TOTAL: 11152 > # PASS: 0 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 11151 > # XPASS: 0 > > ==> tests/cli/test-suite.log <== > ===================================================== > babeltrace 2.0.0-pre1: tests/cli/test-suite.log > ===================================================== > > # TOTAL: 153 > # PASS: 143 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 7 > # XPASS: 0 > > ==> tests/plugins/test-suite.log <== > ========================================================= > babeltrace 2.0.0-pre1: tests/plugins/test-suite.log > ========================================================= > > # TOTAL: 2 > # PASS: 0 > # SKIP: 0 > # XFAIL: 0 > # FAIL: 0 > # XPASS: 0 > > (BTW, since the tests don't pass/skip/xfail 100%, 'make distcheck' is > failing.) _______________________________________________ lttng-dev mailing list [email protected] https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
