: first place. Some things were wrapped in "multiple-failure" exception,
: which didn't seem right to me. When I wrote the dedicated runner there
: was a lot of guessing involved, including XML reports (this bit not
: even part of JUnit, but Apache Ant and Apache Maven).
...
: It's actually the listener that I wrote that emits this log. Which
: leads me to say that if it's more helpful we could emit a different
: type of report from the runner (in addition to what Jenkins consumes).
: Internally, listeners receive each and every exception separately so
: they can serialize them in a more palatable manner. The XML listener
: is just one implementation:
Interesting point ... but if we did generate a "new" structured report
from the runner, something would still need to be able to aggregate those
reports across multiple runs (ie: "ant test" being run in multiple dirs,
the repro logic re-running "ant test" later in the same jenkins job,
etc...) for it to be useful. As long as we stick w/the constraints of
the JUnit XML report we can leverage the ((ant|maven)+jenkins) testReport.
Thinking about it purely from the point of view of the failure rate
reports i've been generating, it would be nice if *every* TestClass
invocation consistently included a single psuedo-method entry indicating
either a PASS if there were no suite level initialization or tear down
problems, and a FAIL if there were -- something akin to the existing
"junit.framework.TestSuite" psuedo test method entries, but exactly one
per "suite instance" regardless of wether it succeeds or not (unlike the
way there can currently be multiple "junit.framework.TestSuite" instances
in a single class if it has multiple problems, but none if it doesn't) ...
that way the number of runs is just the exact number of entries with this
special name, and the number of suite instances that had 1 or more
problems is just the number of psuedo-mehtods that "FAILed"
It would make the xml files bigger in the "no problem" situation, but not
by much.
-Hoss
http://www.lucidworks.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
