[
https://issues.apache.org/jira/browse/SUREFIRE-1470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16418905#comment-16418905
]
Alexey Subach commented on SUREFIRE-1470:
-----------------------------------------
Before trying to fix the issue I wanted to get feedback from your side, which I
had to wait for for quite long time :)
The "correct" result also seems a but strange - if test passes on second run,
why the total number of tests run is reported to be 3?
Regarding using surefire-junit47 provider, how do I set it? If I follow the
guidance at
http://maven.apache.org/surefire/maven-surefire-plugin/examples/providers.html
and set it like this:
{code}
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${surefire.version}</version>
<configuration>
<includes>
<include>**/*.java</include>
</includes>
<groups>TestGroup</groups>
<rerunFailingTestsCount>1</rerunFailingTestsCount>
</configuration>
<dependencies>
<dependency>
<groupId>org.apache.maven.surefire</groupId>
<artifactId>surefire-junit47</artifactId>
<version>${surefire.version}</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
{code}
The resultant TXT report is still incorrect and the same as it was without
explicit dependency specification.
Am I doing something incorrectly, or is it another bug?
> Unexpected report result when using rerunFailingTestsCount with groups filter
> in Surefire
> -----------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1470
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1470
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.x support, Maven Surefire Plugin
> Reporter: Alexey Subach
> Priority: Major
> Attachments: test.zip
>
>
> When the following plugin configuration is used:
> {code:java}
> <plugin>
> <groupId>org.apache.maven.plugins</groupId>
> <artifactId>maven-surefire-plugin</artifactId>
> <version>${surefire.version}</version>
> <configuration>
> <includes>
> <include>**/*.java</include>
> </includes>
> <groups>TestGroup</groups>
> <rerunFailingTestsCount>1</rerunFailingTestsCount>
> </configuration>
> </plugin>{code}
> And there is a test that fails first time it's run, the resultant *txt*
> report is unexpected and only contains the result of last rerun.
> Suppose we have the following test class:
> {code:java}
> @Category(TestGroup.class)
> public class CustomTest {
> @Test
> public void alwaysPassingTest() {
> Assert.assertEquals(0, 0);
> }
> private static volatile int x = 0;
> @Test
> public synchronized void secondAttemptPassingTest() throws
> InterruptedException {
> Thread.sleep(5000);
> if (x++ == 0) {
> Assert.fail();
> }
> }
> }
> {code}
> The resultant xml report is all right, but the txt one is
> {code}
> -------------------------------------------------------------------------------
> Test set: CustomTest
> -------------------------------------------------------------------------------
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.063 s - in
> CustomTest
> {code}
> Which is basically incorrect. The *txt* file is regenerated several times
> during the test run.
> The issue is present when groups filter (either exclusion or inclusion) is
> present. When there is no group filter and only rerunFailingTestsCount is
> configured, then the resultant report is the following one:
> {code}
> -------------------------------------------------------------------------------
> Test set: CustomTest
> -------------------------------------------------------------------------------
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.065 s <<<
> FAILURE! - in CustomTest
> secondAttemptPassingTest(CustomTest) Time elapsed: 0.009 s <<< FAILURE!
> java.lang.AssertionError
> at CustomTest.secondAttemptPassingTest(CustomTest.java:18)
> {code}
> When no group filter is configured, Junit4Provider instead of JunitCore
> provider is picked up and they seem to handle those cases differently.
> The situation is worsened by the fact that TeamCity is not able to parse the
> results correctly when the groups are configured along with
> rerunFailingTestsCount and it marks the tests that passed after rerun as
> failed, although the overall Maven result is SUCCESS in both cases.
> The simple Maven project to reproduce the issue is attached.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
