[
https://jira.codehaus.org/browse/SUREFIRE-813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=286919#comment-286919
]
Kristian Rosenvold commented on SUREFIRE-813:
---------------------------------------------
The root cause of this is a ConcurrentModificationException in LogicalStream
line 81. While avoding the exception itself is simple, we again have somewhat
of a halting problem, since there is no way to determined when any forked
threads are finished. At this moment the only "solution" I can come up with is
to detach further output from the test-cases in question and just mix it in
with the single global stream.
Additionally, we could iterate all the streams on termination and maybe append
"left over" output. This stuff isn't too easy to get right, so any suggestions
are welcome ;)
> Randomly wrong tests count and empty summary files with JUnitProvider47 when
> forking the tests.
> -----------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-813
> URL: https://jira.codehaus.org/browse/SUREFIRE-813
> Project: Maven Surefire
> Issue Type: Bug
> Components: Junit 4.7+ (parallel) support
> Affects Versions: 2.10, 2.11, 2.12
> Environment: test case checked on windows 7 64 bits; issues on Linux.
> jdk1.6.0_24
> Reporter: nkeywal
> Assignee: Kristian Rosenvold
> Attachments: pom.xml, Test6.java
>
>
> When running the test suite, with forkMode set and junit provider 47, we can
> have some test results missing. The test class is executed, but:
> - the summary file is empty (exists, but as a zero length file)
> - the global test count is not increased.
> It's random, and does not occurs all the time. The occurrence seems increased
> when the machine is loaded, but it's more an impression. In our case, with
> ~250 test classes, it occurs on average once per run (so it's a real
> problem). It seems to occur more if we activate 'perthread' parallelization.
>
> I added some log lines, and it seems that ForkClient does not receive all the
> test events when the problem occurs. It receives:
> - ForkingRunListener.BOOTERCODE_TESTSET_STARTING
> - ForkingRunListener.BOOTERCODE_TEST_STARTING
>
> But not the final:
> - ForkingRunListener.BOOTERCODE_TEST_SUCCEEDED
> - ForkingRunListener.BOOTERCODE_TESTSET_COMPLETED
>
> The original issue is on Linux, java 1.6, with a lot of tests running. I
> wrote a test case, hopefully reproducing the real issue with surefire 2.10
> and 2.11. It's still random, but fails 50% of the time on my machine (laptop
> 2 cores, Windows 7, 64 bits, jdk1.6.0_24).
> In this example, we create a thread writing 1000 lines of
> 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" in stdout in the tear down method.
> That's an example of success with surefire 2.11. You see that exactly two
> lines of 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' has been written to the console. On
> some other successes it can be zero lines or more.
> {noformat}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Concurrency config is parallel='classes', perCoreThreadCount=true,
> threadCount=2, useUnlimitedThreads=false
> Running Test6
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
> 658:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> 659:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> Results :
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 2.724s
> [INFO] Finished at: Tue Dec 27 11:46:03 CET 2011
> [INFO] Final Memory: 10M/109M
> [INFO]
> ------------------------------------------------------------------------
> {noformat}
> Error case, with surefire 2.11. In this one there is no
> 'AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA' in the output, but you can have errors with
> multiple such lines.
> You see that the number of tests run is zero.
> {noformat}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Concurrency config is parallel='classes', perCoreThreadCount=true,
> threadCount=2, useUnlimitedThreads=false
> Running Test6
> Results :
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD FAILURE
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 2.820s
> [INFO] Finished at: Tue Dec 27 11:48:02 CET 2011
> [INFO] Final Memory: 10M/109M
> [INFO]
> ------------------------------------------------------------------------
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.11:test (default-test) on
> project surefireparametizedbug: No tests were executed! (Set
> -DfailIfNoTests=false to ignore this error.) -> [Help 1]
> [ERROR]
> {noformat}
> With junit4 instead of junit47 it seems it never fails, but you can still
> have some AAA lines in the final output:
> {noformat}
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running Test6
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.07 sec
> 18:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> 19:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> 20:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> 21:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> 22:AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
> Results :
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
> [INFO]
> ------------------------------------------------------------------------
> [INFO] BUILD SUCCESS
> [INFO]
> ------------------------------------------------------------------------
> [INFO] Total time: 2.667s
> [INFO] Finished at: Tue Dec 27 11:54:16 CET 2011
> [INFO] Final Memory: 10M/114M
> [INFO]
> ------------------------------------------------------------------------
> {noformat}
> It can fail as well (but less often it seems) if redirectTestOutputToFile is
> set to false.
> It fails as well with forkMode=once (on surefire 2.11 at least).
> parallel is set to "classes"; but if fails as well if it's not set (still on
> surefire 2.11 at least).
> Note that I write 1000 lines in the test case, it can fail with much less as
> well. It's just a simplification of the initially complex issue.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://jira.codehaus.org/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
