[
https://issues.apache.org/jira/browse/SUREFIRE-2274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17907016#comment-17907016
]
Elliotte Rusty Harold commented on SUREFIRE-2274:
-------------------------------------------------
Possibly related: SUREFIRE-1644
> failOnFlakeCount does not work for failsafe
> -------------------------------------------
>
> Key: SUREFIRE-2274
> URL: https://issues.apache.org/jira/browse/SUREFIRE-2274
> Project: Maven Surefire
> Issue Type: Bug
> Components: Maven Failsafe Plugin
> Affects Versions: 3.5.0
> Reporter: Daniel Fesenmeyer
> Priority: Major
>
> The property failOnFlakeCount seems not to work for the failsafe plugin. At
> least not as it is documented here:
> [https://maven.apache.org/surefire/maven-failsafe-plugin/verify-mojo.html]
> h2. Expected behavior
> When using the failsafe configuration property {{{}failOnFlakeCount{}}}, I'd
> expect that the build fails in case there are more than the specified count
> of flakes.
> h2. Actual behavior
> The build succeeds, even when there are more than the specified count of
> flakes. On the console, and the _TEST-***.xml_ report files, flakes are
> correctly reported. However, the report {_}failsafe-summary.xml{_}, does not
> contain a {{flakes}} property, and thus the verify step fails.
> h2. How to reproduce
> I've created a reproducer here:
> [https://github.com/danielFesenmeyer/failsafe-flakecount-reproducer]
> Just run the integration tests for this simple maven project:
> mvn verify
> The build shows the following output, which is correct, except for the
> information that the build succeeded (and status code 0):
> {code:java}
> ...
> [INFO] Running de.fesenmeyer.reproducer.ReproducerIT
> [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003
> s -- in de.fesenmeyer.reproducer.ReproducerIT
> [INFO]
> [INFO] Results:
> [INFO]
> [WARNING] Flakes:
> [WARNING] de.fesenmeyer.reproducer.ReproducerIT.test1
> [ERROR] Run 1: ReproducerIT.test1:12 » IllegalState Try 0 failed
> [ERROR] Run 2: ReproducerIT.test1:12 » IllegalState Try 1 failed
> [INFO] Run 3: PASS
> [INFO]
> [WARNING] de.fesenmeyer.reproducer.ReproducerIT.test2
> [ERROR] Run 1: ReproducerIT.test2:17 » IllegalState Try 0 failed
> [ERROR] Run 2: ReproducerIT.test2:17 » IllegalState Try 1 failed
> [INFO] Run 3: PASS
> [INFO]
> [INFO]
> [WARNING] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Flakes: 2
> [INFO]
> [INFO]
> [INFO] --- failsafe:3.5.0:verify (default) @ failsafe-flakecount-reproducer
> ---
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> ...
> {code}
> h2. Hint
> Probably the Feature currently works for Surefire, it seems like it has been
> only tested with Surefire.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
