On Mon, Jun 27, 2011 at 12:09:29AM +0100, sebb wrote:
> On 26 June 2011 17:58, Gilles Sadowski <[email protected]> wrote:
> >>
> >> Some tools are smart enough to detect it, but not all of them.
> >>
> >> >At least, maven's conclusion is correct: It knows that there no tests to
> >> >run
> >> >(or rather that it should not try to instantiate an abstract class)
> >> >despite
> >> >the fact that there are methods marked with the "@Test" annotation.
> >>
> >> Yes, but it find it after having failed to launch these tests. It
> >> does not check it beforehand.
> >
> > How do you see that?
> > When I run "mvn test" here, the output does not show "BaseSecantSolverTest"
> > at all, and the build is successful. [Otherwise I wouldn't have committed
> > that class.]
>
> Yes, because Surefire detects that the class is abstract.
Thus, do you know whether this tool could be used from ant too?
> Did you try "ant test" ?
Not before committing the offending class, if this what you ask.
I usually run "mvn clean site", look at the output of the tests, and at the
CheckStyle-generated HTML page.
> JUnit does not detect it until it tries to run it.
Thus Junit does not analyse the code as Surefire does, but there is an
annotation ("@Ignore") to tell it not to run any tests within the annotated
class. As a side-effect, it does not try to instantiate it.
> >> >
> >> >If it's possible, it is certainly more robust to rely on a Java keyword
> >> >("abstract") rather than on an external convention (a name ending with
> >> >"AbstractTest"). [Or, more precisely, even if the name does not end with
> >> >"AbstracTest", the information exists that can prevent generating an
> >> >error.]
> >>
> >> It would be a good thing, I didn't check if there was a feature
> >> request on the tools we use for this (ant, maven, surefire, gump,
> >> continuum, eclipse ...).
> >>
> >> >The introduction of the "@Test" annotation is a progress, going in that
> >> >direction; that is, it is not mandatory anymore that the name of a test
> >> >method starts with "test". Similarly, it should not be mandatory that a
> >> >base class for tests ends with "AbstractTest".
> >>
> >> I don't know all the annotations Junit uses. Perhaps there is
> >> something for the complete class.
> >
> > I'll check whether instanitation could be disabled at that level; that would
> > indeed be the best option.
>
> No. the best option is to use the proper naming convention, so the
> tools don't even try to run the class as a test.
It used to be the only option because the tools were not smart enough. Now
it is not necessary anymore (in the same way that test methods do _have_ to
start with "test").
Before it was necessary, now it's a matter of taste.
> >> >
> >> >>>Maven also relies
> >> >>>on the surefire plugin to run the tests, and surefire relies on Junit.
> >> >>>So
> >> >>>there are a lot of intermediate steps between a build system (Gump,
> >> >>>Continuum, direct use of ant, direct use of Maven, Eclipse, Eclipse with
> >> >>>maven plugin ...) and the low level Junit runs. This may explain the
> >> >>>differences.
> >> >>
> >> >>Indeed.
> >> >>
> >> >>>I have already noticed that many tools do not have the same algorithm to
> >> >>>select classes to test. A recent example was the performance tests for
> >> >>>FastMath. Maven skip these tests because they do not end in "Test", but
> >> >>>Eclipse for example does not skip them because it directly look inside
> >> >>>the
> >> >>>class and find the @Test annotations.
> >> >>
> >> >>[Yes, that Eclipse behaviour is a nuisance!]
> >> >>
> >> >>It *could* have been that the Ant build file and Maven Pom had
> >> >>different configs for the test file names. However, I've just
> >> >>checked, and the files agree.
> >> >>
> >> >>The difference is due to the way that Surefire currently processes
> >> >>test-classes before handing them to JUnit.
> >> >
> >> >Do you mean that it's Surefire that handles the things correctly?
> >> >Then, could it be used also by Gump?
> >>
> >> It does not handle it correctly, it still tries to launch the tests
> >> and when it fails, there is an error message.
> >
> > Then I don't understand why it works locally, but not on the build server...
> > [No error message here.]
>
> Because you are using Maven, not Ant.
Well, I started use maven especially for building commons-math because I was
advised to do so, ant being not well supported (at the time). Did it change?
By the way, I noticed that, in order to run a single test case, the
property definition is not the same with ant and maven:
ant -Dtest.entry=org.apache.commons.math.analysis.solvers.SomeTest test
vs.
mvn -Dtest=SomeTest test
[With ant, the fully-qualified name of the class is required.]
> >> >
> >> >>>There is clearly no ideal solution, but I think having different build
> >> >>>systems to suit several users needs is better. Having different tools
> >> >>>help
> >> >>>finding different bugs.
> >> >>
> >> >>Indeed - in this case, the wrong naming convention for an abstract test
> >> >>class.
> >> >
> >> >If it's impossible to properly configure Gump, I'll change the name...
> >>
> >> I think this is the better short term solution.
> >>
> >
> > Probably, if there is no Junit annotation.
>
> Using the correct naming convention is the solution.
It's not _the_ correct solution, it's one solution; using "@Ignore" is now
another which arguably is more robust since the behaviour will be the same,
whatever tool calls Junit, without relying on out-of-band conventions or
intermediate tools.
Regards,
Gilles
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
