Andreas Korneliussen wrote:
Kristian Waagan wrote:
Myrna van Lunteren wrote:
On 4/18/06, *John H. Embretsen (JIRA)* <[email protected]
<mailto:[email protected]>> wrote:
[snip - details and useprocess=false discussion]
JUnit and useprocess=false:
----------------------------------------------------
Some may be surprised to find that junit tests are run in a separate
process by default (instead of being skipped) when useprocess=false.
However, I think I see some advantages of doing it this way, so I
accept (a bit reluctantly, perhaps) this behavior for now.
In the patch I also find:
- if ( useprocess )
+ // Always create a process if testType is junit,
otherwise testRunner
+ // would exit a suite prematurely, and we can't debug
that anyway and
+ // they should run quick anyway.
+ if ( useprocess || (!useprocess &&
testType.equals("junit")))
I'm not sure I understand/remember exactly why junit tests cannot be
run with useprocess=false ( i.e. why "testRunner" (I assume this
means junit.*ui.TestRunner) would exit a suite prematurely). Feel
free to educate me on this subject (otherwise I guess I'll look into
it myself when time permits) ;)
Yes, I was referring to junit.*ui.TestRunner. I wanted to enable as
many test as possible when running useprocess=false, and I thought
skipping the junit test would be an increasingly bad idea as we'd get
more and more junit tests...
So I experimented running the junit test as a separate process (I
just copied how they're run when useprocess=true (the default)), the
first junit test ran successfully and that was all that ran, the
entore RunSuite process just stopped. I assumed that TestRunner must
be doing a System.exit() somewhere. Maybe I was off with that
conclusion/assumption...
I think you are correct with your conclusion/assumption. A quick grep
of the JUnit source code (3.8.1), shows matches for System.exit in all
of textui.TestRunner, swingui.TestRunner and awtui.TestRunner.
Since we would want a text/console based test runner, we may have to
write our own if we want to run JUnit tests with useprocess=false.
When looking at the source of textui.TestRunner, I don't think this
would be a very different task. It might even be possible to do a
quick hack, but I do not know if this is allowed according to the
license (CPL) or if we want to do this.
Can anyone with knowledge of CPL (Common Public License - v 1.0) shed
some light on what we can and can't do?
To make a new TestRunner, you may i.e make a new class which inherits
from junit.runner.BaseTestRunner.
In terms of legal, I do not see how this would be different from making
a class which inherits from junit.framework.TestCase, which we already
do with junit tests. However I am not sure what exactly you mean by
making a quick hack - did you mean modifying the existing
junit.runner.TestRunner class?
Yes, primarily to see how/if things work, before we spend time and
resources on writing our own TestRunner. But as I said, writing our own
runner should be pretty easy.
A new TestRunner should imo. do the following:
* when starting a Test, it should print that it starts a test, instead
of printing .
* when a test fails, it should immediately print the failure, instead of
waiting untilt the test completes.
In addition to the above, it should also print a summary with the number
of tests, number of failures and errors (as the textui.TestRunner does).
The output might require some thought, depending on how the tests are
run (ie. a single run of one big suite, or multiple runs of smaller suites).
As for integration with the harness, the messages printed from junit
tests / TestRunners, should be prefixed with something which can easily
be filtered by Sed - to avoid making diff files for junit tests.
I totally agree. Introducing diff files for JUnit tests would be a huge
regression in my opinion.
--
Kristian
Andreas
