I actually thought about the whole "it should fail to build if
any of the unit tests fail" idea 2 or 3 weeks ago, so this sounds
good.
WRT to the error messages and their recognition by text editors,
a _massive_ improvement would be compiler-assisted formatting of
the assertion errors. This:
[email protected](2): Assertion failure
Is not useful when I wrote `assert(foo == 2)`. This, however, is:
tests.encode.testEncodeMoreThan8Bits:
tests/encode.d:166 - Expected: [158, 234, 3]
tests/encode.d:166 - Got: [158, 234]
In Python, my favourite testing framework is py.test. It reflects
on the test code itself and replaces `assert foo == 2` with its
own code so that it looks like this in the output:
def test_foo():
foo = 5
assert foo == 2
E assert 5 == 2
It also recognises things like `assert x in xs`, which is
obviously handy. Since Walter has mentioned the "specialness" of
assert before, maybe the compiler could recognise at least the
most common kinds and format accordingly (assert ==, assert in,
assert is null, assert !is null)?
The main reasons I wrote a unit testing library to begin with
were:
1. Better error messages when tests fail
2. Named unit tests and running them by name
3. Running unit tests in multiple threads
I'm address 1. above, 2. has its own thread currently and AFAIK
3. was only done my me in unit-threaded. There are other niceties
that I probably won't give up but those were the big 3.
Atila
On Monday, 30 March 2015 at 22:20:08 UTC, Andrei Alexandrescu
wrote:
This is a tooling issue.
D builds and messaging are rigged to consider unittests as a
special build followed by a special run of an application.
In brief: I'd like to transition to a model in which
unittesting is organically part of the build. After all, you
wouldn't want to deploy an application that's failing its
unittests.
Detail: Consider running a build vs. a unittest in one of the
supported editors/IDEs (emacs, vim, Code:Blocks, Visual D,
Xamarin...). During a build, an error will come in a standard
format, e.g.
std/array.d(39): Error: undefined identifier xyz
This format is recognized by the editor and allows the user to
click on it and go straight to the offending line etc.
In contrast, a failing unittest has a completely different
format. In fact, it's a format that's worse than useless
because it confuses the editor:
core.exception.AssertError@std/array.d(39): xyz
emacs will recognize the text as a filename and upon clicking
would ask the user to open the nonsense file
"core.exception.AssertError@std/array.d". This error line is
followed by a stacktrace, which is in no recognizable format.
It should be in the format of e.g. gcc remarks providing
additional information for an error, and again provide
file/line information so the user can click and see the call
stack in the source.
Where I want us to be is a place where unittests are considered
a part of the build; it should be trivial to set things up such
that unittesting is virtually indistinguishable from
compilation and linking.
This all is relatively easy to implement but might have a large
positive impact.
Please chime in before I make this into an issue. Anyone would
like to take this?
Andrei
