On Apr 28, 2010, at 5:45 PM, Andrei Alexandrescu <[email protected]>
wrote:
The problem is scale. Phobos is pretty large already. I did make
changes in unittests as you suggest above, but there are a lot of
those to make. Besides, when compiling Phobos as one unit and
unittesting everything, all unittests are run in one shot so you
don't know about modules and stuff.
I improved that by painstakingly changing the build process to print
and unittest one Phobos module at a time, while unittests are
disabled in all other modules. That does help but is still far from
perfect.
I think there will be a lot of opinions on what should be done:
• print failing test(s)
• Module-level status like what you did
• Test-level status (test name + pass/fail) like someone else
suggested
• Test with other log messages
• Level based logging for customizable post-mortem review
As a reference, cruise control with nunit outputs a set of web pages.
One is a top level showing each time the test suite was run and an
overall success/failure indicator. Clicking on one brings up a summary
page with number of tests, number passing tests, number of failed
tests, number of skipped tests. Further down is a summary of tests by
category. Each one is colored based on %success, %failure, and
%skipped. Clicking on one shows individual test results with failures
first and skips second. Clicking on one of those tests shows the logs
generated while running the test.
NUnit has a concept of test fixture setup, test setup, test teardown,
and test fixture teardown. A test fixture with N tests would run test
fixture and teardown once, but run test setup and teardown either 0 or
N times. Test fixture setup failure skips all N tests. Test setup
failure skips that one test.
I think providing overidable hooks for "starting test" and "finished
test" is a 90% solution to letting people do whatever they deem best.
_______________________________________________
dmd-internals mailing list
[email protected]
http://lists.puremagic.com/mailman/listinfo/dmd-internals
