[
https://issues.apache.org/jira/browse/SUREFIRE-1643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17677333#comment-17677333
]
Zoltan Meze commented on SUREFIRE-1643:
---------------------------------------
I had this same issue with xmls, it was causing reporting problem with
rerunFailingTestsCount in parallel runs. Sometimes first run and rerun ended up
in different xml files, and another report generator treating them as two
separate tests.
Small workaround was to use custom JUnit5Xml30StatelessReporter with custom
StatelessXmlReporter.
This custom reporter then removes results that doesn't belong to the completed
test set (class) and stores them in internal cache for later. If cache contains
something that belongs to current completed test set then it just restores that
entry from internal cache. Report entries can be modified easily, reference is
available from TestSetStats. Completed, failed, skipped count fields can be
modified using reflection. If anyone is interested I can share some code, but
it's a bit hacky solution.
> JUnit 5 in parallel execution mode confuses Surefire reports
> ------------------------------------------------------------
>
> Key: SUREFIRE-1643
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1643
> Project: Maven Surefire
> Issue Type: Bug
> Components: JUnit 5.x support
> Affects Versions: 3.0.0-M3
> Reporter: Christian Stein
> Assignee: Tibor Digana
> Priority: Major
> Labels: triaged
> Fix For: 3.0.0-M9
>
>
> Hi I hope somebody here can help me out. I am observing something strange
> while using JUNIT 5 parallel execution. I have two test classes TagTests, and
> TagTypeTests. When I set the execution mode to CONCURRENT on both classes,
> and run them, the test report shows the test cases on the wrong test class.
> For example, the test case testA belongs to class TagTests, but it is shown
> in the TagTypeTests test report. TagTests has 11 test cases, and TagType has
> 13 test cases. But the test report shows there are 22 test cases in
> TagTypeTests, and 2 in TagTests. Is this a known issue of parallelism? How to
> get around the issue?
> Sam Brannen @sbrannen Mar 01 18:00
> How is the test report generated? In an IDE? Using a build tool?
> James Zhou @jazhou Mar 01 19:47
> @sbrannen I am using maven surefire plugin(ver 3.0.0-M3) to run the tests.
> Here is the results:
> [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.42 s
> - in org.aaa.test.functional.tests.TagCrudTests
> [INFO] Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.148
> s - in org.aaa.test.functional.tests.TagTypeTests
> [INFO] Results:
> [INFO]
> [INFO] Tests run: 24, Failures: 0, Errors: 0, Skipped: 0
> @sbrannen there are actually 11 tests under TagCrudTests. Somehow only one
> test is shown in the report generated by surefire plugin
> Christian Stein @sormuras Mar 01 20:31
> @jazhou Do you have a minimal (failing) example project to share?
> James Zhou @jazhou Mar 01 22:56
> @sormuras Sure. Here is the code:
> https://github.com/jazhou/junit5-parallel-test
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
