Hi all, First of all, sorry for the lengthy post. I decided to add some context to explain things a bit, but it resulted in quite a long e-mail. For the past few weeks I've been trying to come up with a solution for the issue I filled under SUREFIRE-1935, but I'm getting stuck and starting to feel like the issue cannot be solved with the current JUnit Platform API. To give some perspective into why this issue is important for us, I first have to explain a bit about our setup.
We write our tests in the Spock framework and use Arquillian to run the tests in the application container. Some of our tests, especially the Selenium based tests, require quite some setup and tear down. Prior to starting the test, Arquillian cube builds and starts several docker containers and the tests are run against these containers. We currently use the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For this, I've already started working on a Spock extension that integrates Spock 2 in the Arquillian test life cycle [1]. This extension is inspired by the (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global registration to keep track of the state managed by Arquilllian. This will end up somewhere at the root of the TestPlan (for example see [3]). Because our tests are quite extensive, with a total run time of 6 hours, we run them with a forkCount of 8, greatly reducing the total duration. However, this is where SUREFIRE-1935 comes into play. With a forkCount > 1, the entire test life cycle is started over and over again for every test class. This happens in JUnitPlatformProvider at line 197 [4]. This results in the entire Arquillian suite being torn down and setup for every class, in our case adding several minutes to the execution of every test class because the docker setup is done over and over again. To overcome this issue, the state from one test class execution has to be carried to the next. It seems the LauncherSession (introduced in JUnit 5.8) is meant to close this gap. However, this would mean my extension would also need to implement a LauncherSessionListener, and I'm not sure if extensions are supposed to integrate with the launcher as well. Also, for this surefire would need to start a session prior to the tests, and close it when done. I think this is a good idea anyway when running on platform 1.8 or higher. Another solution could be a (sort of) dynamic test that produces the tests to be run one by one. However, here my knowledge of JUnit really falls short. I've got no idea of this is even possible. I hope someone can help me out on this one and point me in the right direction, as we like to upgrade our test frameworks and this is blocking us at the moment. Best regards, Emond Papegaaij [1] Arquillian extension for Spock Framework 2: https://github.com/topicusonderwijs/arquillian-testrunner-spock/tree/spock-2.0-junit5 [2] Arquillian module for JUnit 5: https://github.com/arquillian/arquillian-core/tree/master/junit5 [3] Registration in root store: https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/JUnitJupiterTestClassLifecycleManager.java#L20 [4] JUnitPlatformProvider launching the tests: https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L197
