[
https://jira.codehaus.org/browse/SUREFIRE-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=351698#comment-351698
]
Jon Todd commented on SUREFIRE-654:
-----------------------------------
I'm running into this issue as well and its quit frustrating. This makes it
very cumbersome to test probabilistic code with surefire. This is an example
I'd expect to pass:
{code:java}
@Test(invocationCount = 100, successPercentage = 25)
void testRandom() {
// ~50% chance of generating 0 & 1
int num = new Random().nextInt(2);
Assert.assertEquals(num, 1);
}
{code}
Though there are no test failures, each of the test cases where this ends up
with an assertion failure surefire counts this as an error and fails the run.
I'd be happy to code up a fix if the patch provided on this isn't the desired
way to handle this but, as it's stands there's is very useful functionality in
TestNG which we simply can't use via Surefire.
> Surefire does not recognize successful tests using TestNG with the
> invocationCount and successPercentage parameters on the @Test annotation
> -------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-654
> URL: https://jira.codehaus.org/browse/SUREFIRE-654
> Project: Maven Surefire
> Issue Type: Bug
> Components: TestNG support
> Affects Versions: 2.6
> Environment: Apache Maven 2.2.1 (rdebian-4)
> Java version: 1.6.0-18
> Java home: /usr/lib/jvm/java-6-openjdk/jre
> Default locale: en_US, platform encoding: UTF-8
> OS name: "linux" version: "2.6.35-22-generic" arch: "amd64" Family: "unix"
> Ubuntu 10.10
> Reporter: Steve Stodola
> Attachments: bugtest.tar.gz, surefire654.diff
>
>
> Surefire does not recognize successful tests using TestNG with the
> invocationCount and successPercentage parameters on the @Test annotation.
> BugTest.java in the attached file has an invocationCount = 100 with a
> successPercentage = 1. 1 test in 100 needs to pass for the test to be
> considered successful. However when running *{{mvn clean test}}* Surefire
> reports the tests as failed. The generated report shows the tests as passing.
> Surefire should be identifying this test as passing.
--
This message was sent by Atlassian JIRA
(v6.1.6#6162)
