This post is mostly directed to DMD contributors.
Currently most of the tests for DMD are end-to-end like tests. The test
invokes the compiler as a new process and the test asserts the exit code
and/or error messages outputted by the compiler. This means that
basically for each test file a new process is created which runs the
compiler. There are two problems with this: a lot of processes need to
be started and executed and since the compiler runs in a separate
process from the test, it's not possible to access the internals of the
compiler.
DMD also contains a few more of traditional unit tests (using the
`unittest` blocks) which tests individual functions. Mostly utility
functions.
I would like to announce that a new test runner has recently been added
for testing DMD. It's like a mix of both of the existing tests types. It
uses the `unittest` blocks and uses the compiler as a library. It runs
the test and the compiler in the same process. It allows to compile code
without starting a new process, then the test is free to assert whatever
it likes. For example, the structure of internal data that are only
available in the same process as the compiler.
The test runner compiles all files of this test type into a single
executable. This should hopefully speed up running the tests because
only a single executable is compiled and run, instead of hundreds.
This type of tests lives in the `test/unit` directory [1]. They're using
the `unittest` block to declare a test. It supports attaching a UDA to
filter tests. It also supports before and after hooks that are executed
before each `unittest` block within the same module and after each
`unittest` block (regardless of the test failed or passed). The test
runner will print a report at the end of which tests that failed.
The tests are executed by default when `make -C test` or `./test/run.d`
is executed. The runner supports a more finer grained control of running
the tests:
* To run all the unit test (without running the other tests), run:
`./test/run.d -u`
* To only run the unit tests in one or more specific files:
`./test/run.d -u test/unit/self_test.d`
* To only run a subset of the unit tests in a single file:
./test/run.d -u test/unit/self_test.d --filter "self test"
In the above example, the `--filter` flag will filter to only run the
tests with a UDA matching the given value, in this case `self test`.
An example can look like this [2]:
module self_test;
import support : afterEach, beforeEach, defaultImportPaths;
@beforeEach initializeFrontend()
{
import dmd.frontend : initDMD;
initDMD();
}
@afterEach deinitializeFrontend()
{
import dmd.frontend : deinitializeDMD;
deinitializeDMD();
}
@("self test")
unittest
{
import std.algorithm : each;
import dmd.frontend;
defaultImportPaths.each!addImport;
auto t = parseModule("test.d", q{
int a = 3;
});
assert(!t.diagnostics.hasErrors);
assert(!t.diagnostics.hasWarnings);
}
You're free to test whatever you like inside the `unittest` block, the
`parseModule` function is not required to use. Here's another example
that does not use it [3].
I highly recommend splitting up the `unittest` blocks to only test one
thing per `unittest` block.
I encourage all DMD contributors to give this a try. Using this kind of
test will hopefully speed up the DMD test suite and make it possible to
test things that are not tested today.
[1] https://github.com/dlang/dmd/tree/master/test/unit
[2] https://github.com/dlang/dmd/blob/master/test/unit/self_test.d
[3] https://github.com/dlang/dmd/blob/master/test/unit/deinitialization.d
--
/Jacob Carlborg