We have to dig into Junit 5. Surefire is a streamer of classes across the forks which is our load balancer. Therefore each class is running separately, pre and post fixtures. If the Junit 5 used Java Streamer including dome kind of control of fixtures then web would have this issue.
Dňa pi 10. 9. 2021, 18:03 Emond Papegaaij <emond.papega...@gmail.com> napísal(a): > On Fri, Sep 10, 2021 at 5:06 PM Tibor Digana <tibordig...@apache.org> > wrote: > > > If you use forkCount > 1, the Surefire loads test classes via load > > balancer. > > If you use default forkCount = 0, all the classes are run eagerly as a > > suite via JUnit5 Launcher in one shot. > > > > Yes, this is what I saw in the JUnitPlatformProvider. The cause of my > issues lies in the difference in events triggered by Arquillian due to the > classes executed one by one. The demo-project attached to the ticket > demonstrates this in just a few lines of code. > > If you are aiming for Arquillian, testing the applications in the > > application server, you should use maven-failsafe-plugin which has > another > > testing model. > > See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin > > Documentation (apache.org) > > < > https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html> > > https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html > > There are Maven phases: > > pre-integration-test > > integration-test > > post-integration-test > > verify > > > > Therefore you should start the application server in the phase > > pre-integration-test. > > Accordingly, you should stop it in the phase post-integration-test. > > > > Then use Failsafe plugin in the phases integration-test and verify. > > > > Unfortunately, this is not how arquillian is setup. You can run tests > against running servers (which you can then start in pre-integration-test), > but using arquillian cube, these containers can be started as part of the > test lifecycle. Also, Arquillian is just a single incarnation of the > underlying issue. Every extension that takes more than a few seconds to > start and is built to only initialize once for a suite will trigger this > same problem (as can be seen by the demo project). As surefire and failsafe > use the same JUnitPlatformProvider, the problem will also be the same. > > What do you think about the solution of using a LauncherSession to bind the > multiple executions together? This will require JUnit Platform 1.8 and > changes in the JUnitPlatformProvider. However, I'm not sure what the > intended audience of LauncherSessionListener is. > > Best regards, > Emond > > On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij < > emond.papega...@gmail.com> > > wrote: > > > > > Hi all, > > > > > > First of all, sorry for the lengthy post. I decided to add some context > > to > > > explain things a bit, but it resulted in quite a long e-mail. For the > > past > > > few weeks I've been trying to come up with a solution for the issue I > > > filled under SUREFIRE-1935, but I'm getting stuck and starting to feel > > like > > > the issue cannot be solved with the current JUnit Platform API. To give > > > some perspective into why this issue is important for us, I first have > to > > > explain a bit about our setup. > > > > > > We write our tests in the Spock framework and use Arquillian to run the > > > tests in the application container. Some of our tests, especially the > > > Selenium based tests, require quite some setup and tear down. Prior to > > > starting the test, Arquillian cube builds and starts several docker > > > containers and the tests are run against these containers. We currently > > use > > > the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to > > > Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For > this, > > > I've already started working on a Spock extension that integrates > Spock 2 > > > in the Arquillian test life cycle [1]. This extension is inspired by > the > > > (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global > > > registration to keep track of the state managed by Arquilllian. This > will > > > end up somewhere at the root of the TestPlan (for example see [3]). > > > > > > Because our tests are quite extensive, with a total run time of 6 > hours, > > we > > > run them with a forkCount of 8, greatly reducing the total duration. > > > However, this is where SUREFIRE-1935 comes into play. With a forkCount > > > > 1, > > > the entire test life cycle is started over and over again for every > test > > > class. This happens in JUnitPlatformProvider at line 197 [4]. This > > results > > > in the entire Arquillian suite being torn down and setup for every > class, > > > in our case adding several minutes to the execution of every test class > > > because the docker setup is done over and over again. > > > > > > To overcome this issue, the state from one test class execution has to > be > > > carried to the next. It seems the LauncherSession (introduced in JUnit > > 5.8) > > > is meant to close this gap. However, this would mean my extension would > > > also need to implement a LauncherSessionListener, and I'm not sure if > > > extensions are supposed to integrate with the launcher as well. Also, > for > > > this surefire would need to start a session prior to the tests, and > close > > > it when done. I think this is a good idea anyway when running on > platform > > > 1.8 or higher. > > > > > > Another solution could be a (sort of) dynamic test that produces the > > tests > > > to be run one by one. However, here my knowledge of JUnit really falls > > > short. I've got no idea of this is even possible. > > > > > > I hope someone can help me out on this one and point me in the right > > > direction, as we like to upgrade our test frameworks and this is > blocking > > > us at the moment. > > > > > > Best regards, > > > Emond Papegaaij > > > > > > [1] Arquillian extension for Spock Framework 2: > > > > > > > > > https://github.com/topicusonderwijs/arquillian-testrunner-spock/tree/spock-2.0-junit5 > > > [2] Arquillian module for JUnit 5: > > > https://github.com/arquillian/arquillian-core/tree/master/junit5 > > > [3] Registration in root store: > > > > > > > > > https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/JUnitJupiterTestClassLifecycleManager.java#L20 > > > [4] JUnitPlatformProvider launching the tests: > > > > > > > > > https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L197 > > > > > >
