Tibor Digana commented on SUREFIRE-1751:

Thank you for the link.
I meant something else. Normally we want the users to make the fix. So 
therefore I asked you whether you wanted to help us making the fix.
The developers open a pullrequest on https://github.com/apache/maven-surefire 
and the fix is usually discussed and reviewed.
No worries if you are not convinced about the fix. We will help you with the 
I am almost totally convinced that the bug is in 
It would be fine for you to write a new test method in 
 and you can add you XML report in {{src/test/resources}} - just see the test 
method *successfulSurefireTestReport*.
Your test will prove the bug is in 
 because the result of the testcase cannot be determined by parsing case by 
case in the switch. It should real all cases and then the result should go to a 
missing logic which specifically looks for flakes. This logic does not exist 
and this should be the fix.

> Surefire report shows flaky tests as failures
> ---------------------------------------------
>                 Key: SUREFIRE-1751
>                 URL: https://issues.apache.org/jira/browse/SUREFIRE-1751
>             Project: Maven Surefire
>          Issue Type: Bug
>    Affects Versions: 3.0.0-M4
>            Reporter: Anu
>            Priority: Critical
>         Attachments: Test.zip, image-2020-02-13-11-41-03-906.png
> We have been using maven-surefire-plugin in 2.22.1 and Junit4.13 for our 
> testing. Now we are trying to upgrade junit to junit5 and saw that the 
> r{color:#6a8759}erunFailingTestsCount parameter isn't working with surefire 
> plugin version. I upgraded surefire to 3.0.0.M4 and verified that the failing 
> tests are now being retried. But the issue is flaky tests show up in the 
> report as Failures.
> {color}
> {color:#6a8759}ie. the log shows
> {color}
> [WARNING] Tests run: 10455, Failures: 0, Errors: 0, Skipped: 993, Flakes: 10
> But the actual surefire report shows like below
> !image-2020-02-13-11-41-03-906.png!
> This is blocking our junit5 upgrade task. Can you please provide a fix for 
> this at the earliest? Please let me know if you need anything else from me to 
> help with the investigation.

This message was sent by Atlassian Jira

Reply via email to