On 2012-01-01 01:57, Andrew Wiley wrote:
On Sat, Dec 31, 2011 at 2:56 PM, Jonathan M Davis<[email protected]> wrote:
On Saturday, December 31, 2011 16:06:49 Jacob Carlborg wrote:
On 2011-12-31 11:37, Jonathan M Davis wrote:
On Saturday, December 31, 2011 11:05:58 Tobias Pankrath wrote:
I think that the AssertError's message (which includes the file and
line number of the failure) and its stack trace are plenty. It's
exactly what you need and nothing else.
- Jonathan M Davis
I want to have such a summary.
I don't see any reason to put that in the standard library. There's
nothing wrong with 3rd party solutions which give additional
functionality, but D's unit test framework is designed to be
minimilistic, and I don't think that adding anything beyond what it
does now in terms of summary makes any sense. The only major issue in
that regard IMHO is the fact that no further unittest blocks within a
module are run after one fails. Even if it did, I still don't think
that a fancier summary would be worth having - especially in the
standard library.
BTW, what would be so wrong if the unit tests for the standard library
displayed a nice report when finished?
My primary issue here is that I don't think that we should be adding stuff to
Phobos which is essentially a new unit test framework on top of the built in
one. If 3rd party stuff wants to do that. Fine. But the standard library should
use the standard facilities. If the standard facilities aren't sufficient, then
they should be improved.
The counterargument is that the language doesn't really provide a
framework - it actually provides anonymous parameterless global
functions that will be run before main is invoked if code is compiled
with -unittest. That isn't considered a framework in any language I've
ever used, but it adds just enough functionality to allow a
well-integrated fully-featured library solution. Would making such a
library solution part of the standard library really be a problem?
I'm mostly ambivalent on this issue because I haven't had time to look
closely at the proposed framework, but your argument seems to be that
all unittesting functionality needs to be built into the language. I
don't think that should be necessary or required.
As for a "nice report," I don't see anything wrong with just using the stack
traces (which include the file, line number, and error message of the assertion
failure). That's all the information that's needed. Anything else is
superfluous IMHO. Now, if there were something nicer that could be generally
agreed upon and added to druntime such that the standard unit test facilities
used it, then fine. I don't see any point to it, but at least in that case, the
standard library is still using the standard unit test framework. What I
really don't want to see is Phobos essentially building a new unit test
framework on top of the existing one. Any issues that need to be addressed
with the unit test framework for the standard library should be addressed in
the standard framework. Any additional framework stuff should be left to 3rd
parties.
As I said above, I wouldn't consider what we have to be a framework,
but it's definitely enough to build an excellent library solution on
top of.
As for a report, the problem is that an assertion error isn't what you
want when you're running, say, a continuous integration server (or,
say, a pull request tester). What you really want is a detailed
explanation of what unittests broke, what the tests were testing, and
how the result differed from what was expected. You want to be able to
have a reasonable idea of what went wrong *without* having to look at
someone else's code and figure out exactly what they're testing every
time.
Exactly, I agree.
--
/Jacob Carlborg
