Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-18 Thread Matthew Adams
Oops. Yes, verify. On Fri, Nov 15, 2013 at 12:49 AM, Vincent Latombe vincent.lato...@gmail.com wrote: you mean 'verify', not 'validate' Vincent 2013/11/14 Matthew Adams matt...@matthewadams.me Here's a bit less philosophical, more practical description of Surefire v. Failsafe.

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-18 Thread Benson Margulies
Please don't conflate phases with function. As explained here, (http://maven.apache.org/surefire/maven-failsafe-plugin/) The Failsafe Plugin is designed to run integration tests while the Surefire Plugins is designed to run unit tests. The name (failsafe) was chosen both because it is a synonym

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-15 Thread James Green
On 15 November 2013 06:57, Vincent Latombe vincent.lato...@gmail.comwrote: maven-failsafe-plugin implies some specific steps are required *before* (pre-integration-test) and *after* (post-integration-test) the test execution, which allows to do fire up a container then turn it off after the

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-15 Thread Russell Gold
It appears that there are multiple use cases for failsafe. I tend to use it for slow tests that don’t necessarily need to pass at the moment, but for which I want to know the percentage passing. Accordingly, I bind the integration-test goal but NOT the verify goal. On Nov 15, 2013, at 1:57 AM,

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-15 Thread Mirko Friedenhagen
Almost same here, Russel. I always execute `mvn test` on my workstation, where these should be executed in less than a minute and let failsafe via `mvn deploy` (which comes after `verify`) in our CI system (for the Jenkins jobConfigHistory-plugin this takes about 10 minutes). I do not use

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-14 Thread Matthew Adams
On Wed, Nov 13, 2013 at 10:39 AM, Ron Wheeler rwhee...@artifact-software.com wrote: On 13/11/2013 11:16 AM, Matthew Adams wrote: I don't think timing should be the heuristic here. The fact that unit tests take less is a result of the fact that what you're testing, aka the unit, tends to be

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-14 Thread Matthew Adams
Here's a bit less philosophical, more practical description of Surefire v. Failsafe. Remember that if you use the maven-surefire-plugin, it's going to execute during the Maven test phase by default, and fail the build on errors _during that phase_ if any tests fail. The maven-failsafe-plugin

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-14 Thread Vincent Latombe
you mean 'verify', not 'validate' Vincent 2013/11/14 Matthew Adams matt...@matthewadams.me Here's a bit less philosophical, more practical description of Surefire v. Failsafe. Remember that if you use the maven-surefire-plugin, it's going to execute during the Maven test phase by default,

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-14 Thread Vincent Latombe
maven-failsafe-plugin implies some specific steps are required *before* (pre-integration-test) and *after* (post-integration-test) the test execution, which allows to do fire up a container then turn it off after the tests (for example). Then, only after these, the test results are evaluated

maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread James Green
I love the FAQ entry that states that it is intended for running integration tests. The next entry should read: What do you call an integration test? I've asked around and no-one comes up with a consistent answer. I guess it depends on what is executing the integration test. In this case maven

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Russell Gold
Hi James, Start with the basic Agile presumption about unit tests: they are very fast (you should be able to run about 100 unit tests in a second), so that you don’t mind running them with every compile, and must always run at 100% before checkin. Integration tests are pretty much anything

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Stephen Connolly
On 13 November 2013 15:20, James Green james.mk.gr...@gmail.com wrote: I love the FAQ entry that states that it is intended for running integration tests. The next entry should read: What do you call an integration test? Any test that takes more than 1 second to run is *not* a unit test.

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Ron Wheeler
+1 Very nice and concise summary of testing strategy. Thanks for taking the time to write this. Ron On 13/11/2013 10:35 AM, Russell Gold wrote: Hi James, Start with the basic Agile presumption about unit tests: they are very fast (you should be able to run about 100 unit tests in a second),

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Matthew Adams
I don't think timing should be the heuristic here. The fact that unit tests take less is a result of the fact that what you're testing, aka the unit, tends to be small. After all, a unit test should test a unit. An integration test, then, if I were defining it strictly, would be anything that's

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Ron Wheeler
+1 While Russell had a more sophisticated explanation, you can't beat the one second rule for clarity:-) Ron On 13/11/2013 10:59 AM, Stephen Connolly wrote: On 13 November 2013 15:20, James Green james.mk.gr...@gmail.com wrote: I love the FAQ entry that states that it is intended for

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Ron Wheeler
On 13/11/2013 11:16 AM, Matthew Adams wrote: I don't think timing should be the heuristic here. The fact that unit tests take less is a result of the fact that what you're testing, aka the unit, tends to be small. After all, a unit test should test a unit. So what is your definition? An

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread James Green
So where should one place a test that intends on exercising code against something real? We have bits here that involve http calls that pre-date soap and we therefore have no mock. A repeat of the second question from my original post: does the integrate test execute against the artefact produced

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread Ron Wheeler
You will probably get better answers. On 13/11/2013 12:09 PM, James Green wrote: So where should one place a test that intends on exercising code against something real? We have bits here that involve http calls that pre-date soap and we therefore have no mock. It depends on what you are

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread George Wilson
FWIW, I tend to think less about the particular definitions and semantics of testing types and think more about the needs associated with tests. For tests which depend on deployed resources, etc... I use failsafe. For tests which can be handled locally whether resources are mocked or called

Re: maven-failsafe-plugin: what is it actually intended for?

2013-11-13 Thread George Wilson
James, see my response. It sounds to me like you may need to use Failsafe- particularly if you need to deploy your artifact to your application server and access system resources. If you are doing simple HTTP requests and want to set it up as a unit test- my advise if capture a response and save