If you use forkCount > 1, the Surefire loads test classes via load balancer. If you use default forkCount = 0, all the classes are run eagerly as a suite via JUnit5 Launcher in one shot.
If you are aiming for Arquillian, testing the applications in the application server, you should use maven-failsafe-plugin which has another testing model. See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin Documentation (apache.org) <https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html> https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html There are Maven phases: pre-integration-test integration-test post-integration-test verify Therefore you should start the application server in the phase pre-integration-test. Accordingly, you should stop it in the phase post-integration-test. Then use Failsafe plugin in the phases integration-test and verify. Cheers Tibor On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij <emond.papega...@gmail.com> wrote: > Hi all, > > First of all, sorry for the lengthy post. I decided to add some context to > explain things a bit, but it resulted in quite a long e-mail. For the past > few weeks I've been trying to come up with a solution for the issue I > filled under SUREFIRE-1935, but I'm getting stuck and starting to feel like > the issue cannot be solved with the current JUnit Platform API. To give > some perspective into why this issue is important for us, I first have to > explain a bit about our setup. > > We write our tests in the Spock framework and use Arquillian to run the > tests in the application container. Some of our tests, especially the > Selenium based tests, require quite some setup and tear down. Prior to > starting the test, Arquillian cube builds and starts several docker > containers and the tests are run against these containers. We currently use > the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to > Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For this, > I've already started working on a Spock extension that integrates Spock 2 > in the Arquillian test life cycle [1]. This extension is inspired by the > (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global > registration to keep track of the state managed by Arquilllian. This will > end up somewhere at the root of the TestPlan (for example see [3]). > > Because our tests are quite extensive, with a total run time of 6 hours, we > run them with a forkCount of 8, greatly reducing the total duration. > However, this is where SUREFIRE-1935 comes into play. With a forkCount > 1, > the entire test life cycle is started over and over again for every test > class. This happens in JUnitPlatformProvider at line 197 [4]. This results > in the entire Arquillian suite being torn down and setup for every class, > in our case adding several minutes to the execution of every test class > because the docker setup is done over and over again. > > To overcome this issue, the state from one test class execution has to be > carried to the next. It seems the LauncherSession (introduced in JUnit 5.8) > is meant to close this gap. However, this would mean my extension would > also need to implement a LauncherSessionListener, and I'm not sure if > extensions are supposed to integrate with the launcher as well. Also, for > this surefire would need to start a session prior to the tests, and close > it when done. I think this is a good idea anyway when running on platform > 1.8 or higher. > > Another solution could be a (sort of) dynamic test that produces the tests > to be run one by one. However, here my knowledge of JUnit really falls > short. I've got no idea of this is even possible. > > I hope someone can help me out on this one and point me in the right > direction, as we like to upgrade our test frameworks and this is blocking > us at the moment. > > Best regards, > Emond Papegaaij > > [1] Arquillian extension for Spock Framework 2: > > https://github.com/topicusonderwijs/arquillian-testrunner-spock/tree/spock-2.0-junit5 > [2] Arquillian module for JUnit 5: > https://github.com/arquillian/arquillian-core/tree/master/junit5 > [3] Registration in root store: > > https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/JUnitJupiterTestClassLifecycleManager.java#L20 > [4] JUnitPlatformProvider launching the tests: > > https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L197 >
