This is an automated email from the git hooks/post-receive script. nthykier pushed a commit to branch master in repository lintian.
commit 3bbb5b58210aae31a5c438da4c51d85478d935f6 Author: Niels Thykier <[email protected]> Date: Sun Mar 23 16:32:14 2014 +0100 doc: Update the test runner tutorial Signed-off-by: Niels Thykier <[email protected]> --- doc/tutorial/Lintian/Tutorial/TestSuite.pod | 73 ++++++++++++++++++++++++++--- 1 file changed, 66 insertions(+), 7 deletions(-) diff --git a/doc/tutorial/Lintian/Tutorial/TestSuite.pod b/doc/tutorial/Lintian/Tutorial/TestSuite.pod index 0a15cc0..cba56c5 100644 --- a/doc/tutorial/Lintian/Tutorial/TestSuite.pod +++ b/doc/tutorial/Lintian/Tutorial/TestSuite.pod @@ -14,21 +14,28 @@ up development, there are various options to limit the tests to run. =head1 DESCRIPTION The Lintian test suite is an extensive collection of various test -cases. The test suite is divided into 5 "sub-suites". The majority +cases. The test suite is divided into 4 "sub-suites". The majority of all tests are currently located in the "tests" sub-suite. To run the full suite in all of its glory, simply invoke: $ debian/rules runtests + OR + + $ mkdir -p debian/test-out + $ t/runtests -k --dump-logs t debian/test-out + But while writing a new tag (or check) you probably only want to run a particular (subset of the) test(s). See L</Running a subset of the tests> for the available options. -The test suite respects the "DEB_BUILD_OPTIONS=parallel=n", though "n" -denotes the amount of "worker" threads. The test runner will -generally have 2 threads more than that. Also each "worker" will run -lintian, which runs multiple unpacking jobs in parallel as well. +When run via I<debian/rules>, the test suite respects the +"DEB_BUILD_OPTIONS=parallel=N". When using I<t/runtests> directly, use +I<-jN> to choose the number of threads. Note that "N" denotes the +amount of "worker" threads. The test runner will generally have 2 +threads more than that. Also each "worker" will run lintian, which +runs multiple unpacking jobs in parallel as well. In case you are wondering about the 2 extra threads in the test runner, the first of them is the "coordinator"-thread (which will @@ -47,12 +54,20 @@ To run a single test by its name, use: $ debian/rules runtests onlyrun=$name + OR + + $ t/runtests --dump-logs t debian/test-out $name + =item Running all tests for a check To run all tests for a given check, use: $ debian/rules runtests onlyrun=$check + OR + + $ t/runtests --dump-logs -k t debian/test-out $check + $check must be the name of a check (it will test for checks/$check.desc) or "legacy". This will run all tests that starts with "$check-". @@ -66,6 +81,10 @@ To run all tests in a given sub-suite, use: $ debian/rules runtests onlyrun=suite:$suites + OR + + $ t/runtests --dump-logs -k t debian/test-out suite:$suites + $suites is a comma-separated list of names of sub-suites to run. Note: this cannot be used to influence the order, in which the sub-suites @@ -78,11 +97,51 @@ tag, use: $ debian/rules runtests onlyrun=tag:$tag -Alternatively: + OR - $ debian/rules check-tag tag=$tag + $ t/runtests --dump-logs -k t debian/test-out tag:$tag =back +=head2 Running tests under coverage + +It is possible to run most of the tests under L<Devel::Cover>. This is +done by passing I<--coverage> to I<t/runtests>. Example: + + $ t/runtests --coverage --dump-logs -j1 -k t debian/test-out + +Please note that L<Devel::Cover> does not seem to handle multiple +threads too well. You may see spurious warnings/errors if you run the +tests with 2 or more active worker threads. + +B<Caveat> 1: Coverage for collections (i.e. programs in I<collection/>) +does not seem to work at the moment. Therefore, they often end up with +(next to) zero coverage in the generated reports. + +B<Caveat> 2: L<Devel::Cover> sometimes changes the output of Lintian +or tools called by Lintian. Obviously, this leads to test +failures. Therefore, you may see weird test failures (or warnings) +when running under coverage. + +=head3 Collecting the coverage you want in a reasonable time + +Collecting coverage is excruciatingly slow. This is not helped by the +fact that it becomes unreliable when run under 2 or more threads. + +Fortunately, L<Devel::Cover> "appends" to its cover database. This +allows you to "slowly" build up the coverage database over multiple +runs. Example: + + $ t/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:scripts + $ t/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:debs + $ t/runtests --coverage --dump-logs -j1 -k t debian/test-out suite:source + ... + +Or: + + $ t/runtests --coverage --dump-logs -j1 -k t debian/test-out $check + $ t/runtests --coverage --dump-logs -j1 -k t debian/test-out legacy + + =cut -- Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/lintian/lintian.git -- To UNSUBSCRIBE, email to [email protected] with a subject of "unsubscribe". Trouble? Contact [email protected] Archive: https://lists.debian.org/[email protected]
