Module Name: src
Committed By: pooka
Date: Wed Aug 18 21:28:03 UTC 2010
Modified Files:
src/regress: README
Log Message:
new tests should use atf -> new tests must use atf
To generate a diff of this commit:
cvs rdiff -u -r1.2 -r1.3 src/regress/README
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/regress/README
diff -u src/regress/README:1.2 src/regress/README:1.3
--- src/regress/README:1.2 Mon Sep 14 17:15:32 2009
+++ src/regress/README Wed Aug 18 21:28:03 2010
@@ -1,61 +1,6 @@
-$NetBSD: README,v 1.2 2009/09/14 17:15:32 apb Exp $
+ $NetBSD: README,v 1.3 2010/08/18 21:28:03 pooka Exp $
-NOTE: New tests should use the ATF framework; see atf(7)
+New tests must use the ATF framework; see atf(7)
and the src/tests directory.
-------------------------------------------------------------------------
-This part of the source tree contains regression tests. There are special
-make targets and rules to follow. Most of these, however, are currently not
-enforced, and most tests available are not conforming.
-
-We hope to fix this someday. If you add new tests, please try to be conforming.
-
-What is a regression test?
-
- A regression test is run by a makefile in a test directory (see below).
- Each makefile may run multiple tests.
-
-What is a test directory?
-
- A directory in this part of the tree is a regression test directory. It
- contains a Makefile which implements the additional "regress" target,
- and runs all it's regression tests during this target.
-
-May the make progress be stopped on failures?
-
- No, the "make regress" target should succeed, unless some regression
- binaries could not be build, disk is full or other catastrophic failures
- outside of the tested subsystem happen. A failing regression test should
- log the failure (see below), but not make the target itself fail.
-
-What are the possible results of a regression test?
-
- A test may either
-
- - succeed, in which case it logs "PASSED" (see below for logging details)
- - fail, in which case it logs "FAILED"
- - not be able to run, in which case it logs "SKIPPED" and the reason
- for the skip in the comment field (see below)
-
- Typical reasons for tests to not being run are missing kernel options,
- or missing privileges (test needs root, "make regress" is invoked by
- mere mortal or vice versa). A test may not fail because of such
- environmental issues, it must detect and properly log the problem.
-
- If a test directory contains tests that may be skipped, it should have
- a README file explaining the prerequisites (e.g. needed kernel options)
-
- In future, we will mark affected makefiles and optimize run/skipped test
- during repeated runs with differing privileges - but currently there is no
- make framework in place to handle this.
-
-How and when does a test log results?
-
- If the make/environment variable ${REGRESS_LOG} is defined, the final
- results (and only those) should be logged to the file named by that
- variable. We will, in the future, add make targets for this purpose.
- The log format is line oriented, one line used per test. Each line
- consists of the directory where the Makefile lives, followed by the
- test name and the result (see above: PASSED, FAILED, SKIPPED).
- Following this an optional comment may be added. For SKIPPED tests the
- comment is not optional. Fields are separated by spaces.
+All exceptions need prior approval from core.
