Ive recently started using Surefire as part of a Maven project.
I was surprised how inconvenient it was to use, and how things about it
seemed, well... broken.
1. When running a suite of tests with "surefire:test
<http://maven.apache.org/plugins/maven-surefire-plugin/test-mojo.html>",
the main thing you want to see a summary of all failed tests. It is THE
most important and normal thing to look at. I was surprised to discover
Surefire doesnt produce this by default. Instead, it creates many
individual reports, one per test, whether they passed or failed.
2. Then, one has to run a second plugin and special goal just to get the
summary, which is what a typical user is going to be looking for. It
puts the summary in a different directory from the individual files. And
it doesnt produce this report as part of the normal build.
3. It generates the XML intermediate files that the report needs. It
also generates identical txt files, for no apparent reason. And
bizarrely, you can turn the XML off, but you cant turn the txt off.
4. Im posting to main Maven Users list as its 19 months since someone on
the Surefire User list last received a reply. A pretty good imitation of
/dev/null. Maybe time to close that list down?
5. You get some situations where because the tests fail, the build
fails, and this actually stops the report generation which will tell you
what test failed :(
http://jira.codehaus.org/browse/SUREFIRE-507?page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel
http://www.eclipse.org/newsportal/article.php?id=216&group=eclipse.technology.iam#216
The whole thing seems like a case of programmer-centric, not
user-centric, thinking. This tool doesn't seem to have started with what
a typical user would want and then worked backwards to meet this goal.
Surefire is a widely used piece of infrastructure, and it should work
better than it does.
-Ben
--
*Ben Hutchison
Senior Developer
* Level 2 476 St Kilda Road Melbourne VIC 3004
T 613 8807 5252 | F 613 8807 5203 | M 0423 879 534 |
www.ibsglobalweb.com <http://www.ibsglobalweb.com/>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
