[
http://jira.codehaus.org/browse/SUREFIRE-611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tycho Lamerigts updated SUREFIRE-611:
-------------------------------------
Attachment: tc-junit-crash.tar.gz
> surefire exit code should explicitly distinguish between crashed and failed
> unit test(s)
> ----------------------------------------------------------------------------------------
>
> Key: SUREFIRE-611
> URL: http://jira.codehaus.org/browse/SUREFIRE-611
> Project: Maven Surefire
> Issue Type: Improvement
> Reporter: Tycho Lamerigts
> Attachments: tc-junit-crash.tar.gz
>
>
> - If a unit test crashes, surefire skips all remaining tests, prints a
> message "[ERROR] There are test failures. ..." to the console, and does NOT
> print a test summary report.
> - If one or more unit tests fail, maven completes remaining tests, prints a
> message "[ERROR] There are test failures. ..." to the console, and DOES
> print a test summary report.
> The only distinction between crash and fail is the presence (or absence) of a
> summary report.
> This is a problem for some tools (like TeamCity), because they typically
> don't pick up on the fact that many testcases were not run at all in case of
> a crashed unit test. Instead in case of a crashed test they simply report,
> for example, 100 tests succeeded, 0 failed. They do not report "50 tests
> skipped due to a crash after test #100", and it is quite hard for those tools
> to recognize this situation. If surefire would report a crash more
> explicitly, other tools can act accordingly.
> I have attached a small maven project to reproduce the problem. By commenting
> and uncommenting the marked line in file ATest.java, you can reproduce both
> fail and crash scenarios. In the crash scenario, Maven will correctly report
> a failed build (unless you use the -Dmaven.test.failure.ignore flag, in which
> case it even won't do that), but it fails to indicate there was a crash which
> caused tests (in this case 3) to be skipped.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://jira.codehaus.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
