On 19/05/2009, at 11:50 AM, Ben Hutchison wrote:
Ive recently started using Surefire as part of a Maven project.
I was surprised how inconvenient it was to use, and how things about
it seemed, well... broken.
1. When running a suite of tests with "surefire:test <http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html
>", the main thing you want to see a summary of all failed tests. It
is THE most important and normal thing to look at. I was surprised
to discover Surefire doesnt produce this by default. Instead, it
creates many individual reports, one per test, whether they passed
or failed.
This is the output from one of my projects:
[INFO] [surefire:test]
[INFO] Surefire report directory: /Users/brett/scm/archiva/archiva/
archiva-modules/archiva-base/archiva-common/target/surefire-reports
-------------------------------------------------------
T E S T S
-------------------------------------------------------
Running org.apache.maven.archiva.common.utils.BaseFileTest
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.05 sec
Running org.apache.maven.archiva.common.utils.VersionComparatorTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.014
sec
Running org.apache.maven.archiva.common.utils.DateUtilTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.02 sec
Running org.apache.maven.archiva.common.utils.PathUtilTest
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016
sec
Results :
Tests run: 15, Failures: 0, Errors: 0, Skipped: 0
What are you looking for that is different to that?
2. Then, one has to run a second plugin and special goal just to get
the summary, which is what a typical user is going to be looking
for. It puts the summary in a different directory from the
individual files. And it doesnt produce this report as part of the
normal build.
I assume you are talking about the HTML report. That is in a separate
plugin because it is a separate function and people just wanting the
above don't want to drag down the report generation dependencies.
3. It generates the XML intermediate files that the report needs. It
also generates identical txt files, for no apparent reason.
Because the XML isn't very readable by humans. I always look at the
text files for failures.
And bizarrely, you can turn the XML off, but you cant turn the txt
off.
http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html#useFile
(yes, naming could be more consistent, the Xml one was probably added
later)
4. Im posting to main Maven Users list as its 19 months since
someone on the Surefire User list last received a reply. A pretty
good imitation of /dev/null. Maybe time to close that list down?
yes, we already indicated they'd fold into the main maven lists, just
haven't got around to doing it.
5. You get some situations where because the tests fail, the build
fails, and this actually stops the report generation which will tell
you what test failed :(
http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel
http://www.eclipse.org/newsportal/article.php?id=216&group=eclipse.technology.iam#216
That might be a regression in the embedded version of Maven.
Cheers,
Brett
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
