[
https://issues.apache.org/jira/browse/SUREFIRE-1169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Falko Modler updated SUREFIRE-1169:
-----------------------------------
Attachment: SUREFIRE-1169_forksExecuteTestsOneByOne_FIXED.patch
The original patch has a serious bug for projects/modules that have only one
test. Such a test will silently ignored!
The the new version {{SUREFIRE-1169_forksExecuteTestsOneByOne_FIXED.patch}}
fixes this.
> JUnit / Arquillian lifecycle friendly test execution with forkCount > 1 and
> reuseForks = true
> ---------------------------------------------------------------------------------------------
>
> Key: SUREFIRE-1169
> URL: https://issues.apache.org/jira/browse/SUREFIRE-1169
> Project: Maven Surefire
> Issue Type: Improvement
> Components: process forking
> Affects Versions: 2.18.1
> Reporter: Falko Modler
> Attachments: SUREFIRE-1169_forksExecuteTestsOneByOne.patch,
> SUREFIRE-1169_forksExecuteTestsOneByOne_FIXED.patch
>
>
> The current approach to "stream" each test class name to a fork via
> {{TestProvidingInputStream}} yields a good "load balancing" accross the forks
> but it *triggers the entire test lifecycle for each test*.
> With {{forkCount = 1}}, all tests are executed in one set but with
> {{forkCount = n}} (n > 1) each test is a separate "set" (so to say).
> This is very problematic in case you or a test framwork you are using relies
> on a certain lifecycle.
> [Arquillian|http://arquillian.org/] for example ties various "events" to
> JUnit's lifecycle methods, like {{AfterSuite}} to
> {{RunListener.testRunFinished(...)}} which triggers the shutdown of the
> managed server etc.
> When using multiple forks, {{RunListener.testRunFinished(...)}} is called for
> *every* single test class, triggering {{AfterSuite}} for every single test,
> although the fork will receive further tests after that which should "reuse"
> the server.
> This is just an example. In fact the whole JUnit / Arquillian lifecycle is
> inconsistent when using multiple forks.
> From a user perspective I wouldn't expect this behaviour:
> As {{forkCount = 1}} (and {{reusableForks = true}}) triggers
> {{RunListener.testRunFinished(...)}} *once*, {{forkCount = n}} (and
> {{reusableForks = true}}) should trigger that method *n* times, not *x* times.
> To be fair, the
> [documentation|https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html]
> *does* contain a pointer to that problem by saying:
> {quote}
> When using reuseForks=true and a forkCount value larger than one, test
> classes are handed over to the forked process one-by-one.
> {quote}
> But the consequences remain very unclear.
> *(Possible) Solution:*
> I took a stab at this and implemented an "eager test distribution" to the
> forks in {{ForkStart.java}} and disabled streaming. Please see attached patch
> (to be applied against project root, 2.18.1).
> Patch Details:
> - New config property: {{ForkConfiguration.forksExecuteTestsOneByOne}}, set
> via Mojo (default is true for backward compatibility, name is debatable)
> - When {{forksExecuteTestsOneByOne}} ist set to false, the {{messageQueue}}
> in {{ForkStarter.runSuitesForkOnceMultiple(...)}} is *not* wrapped in fork
> specific {{TestProvidingInputStream}} instances to be consumed bit by bit
> later on.
> Instead, the queue is consumed directly and each test class name is assigned
> to the respective fork by creating a copy of the {{providerProperties}} which
> is filled individually for each fork.
> E.g. for three forks and eight tests:
> -- fork 1 executes test 1, 4 and 7
> -- fork 2 executes test 2, 5 and 8
> -- fork 3 executes test 3 and 6
> - To have a clean {{providerProperties}} template I had to move
> {{DefaultScanResult.writeTo(...)}} to the respective private methods.
> Otherwise the properties would have contained *all* tests already.
> - I refactored some methods in {{ForkStarter}} to enhance readability and to
> reduce code duplication.
> - The patch does *not* contain a test for the new behaviour but all existing
> tests passed.
> {{forksExecuteTestsOneByOne = false}} now leads to a consistent lifecycle.
> This solution has one downside: One or more forks could be overloaded while
> other forks could "underloaded" because you cannot say how long each test
> will take. runOrder=balanced could help here but has yet to be implemented
> for forks.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
