I've tried submitting this as a launchpad answer but it expired
without further notice; I'm posting this here, I hope it's the right
place to discuss this.
I'm indirectly using zope.testrunner through zc.buildout zc.recipe.testrunner:
recipe = zc.recipe.testrunner
eggs = pydenji
defaults = [ "--auto-color", "-vv", "--tests-pattern", "^test_.*" ]
It works; but what happens if a test file matching the pattern has got
a serious failure, let's say an import or syntax error ?
Test-module import failures:
SyntaxError: invalid syntax
Running tests at level 1
Running zope.testrunner.layer.UnitTests tests:
Set up zope.testrunner.layer.UnitTests in 0.000 seconds.
[...] long test output [... ]
Ran 73 tests with 0 failures and 0 errors in 0.188 seconds.
Tearing down left over layers:
Tear down zope.testrunner.layer.UnitTests in 0.000 seconds.
Test-modules with import problems:
I can't show you such things here, but the "73" is green, which is the
colour for "all ok" - if any failure happens, the colors turn red,
while the "test modules with import problems" are just a tiny line
after that, and often gets overlooked.
Also, a test file matching the pattern but which not defines any test
is treated the very same way as a file with import problems, which is
probably not what it want.
The issues I can find are:
- I need to dig to the top in order to get the traceback; other
frameworks, like twisted's own trial, print all the tracebacks at the
bottom of the test run for easy debugging;
- test colors should not turn green if any test with import problem is
around; maybe an import/syntax error should count as a generic error.
- while an import issue is a serious fact - meaning the test can't be
run, and should be reported, a test module which does not define any
test could just issue a warning - it could just be a placeholder test,
or a not-yet-finished test case, and should not be a blocking issue.
Any thoughts on this?
contact me at pub...@[mysurname].eu
Zope-Dev maillist - Zope-Dev@zope.org
** No cross posts or HTML encoding! **
(Related lists -