Hi Tibor, Another popular principle out there is do one thing well. The provider principle is already in play with Maven's plugin architecture.
I would much rather have a test plugin purpose built for the test framework I use. Its fine until you want to do something a bit extra, https://github.com/apache/maven-surefire/issues/3146 Without knowing anything about the internals, my perception is that surefire is hugely complicated with a lot of needless ceremony. I don't feel that way about any other plugin. You talk about separation of concerns, but at least from the outside it seems so many concerns are overlapping. Configuration, test discovery. Threading is configurable in all three, junit, surefire, and maven, not to mention the unit under test. The gold standard is if test frameworks provide their own plugin so there is clear responsibility for it, and maybe a plugin that shares its namesake will encourage some buy-in. If the lineage of developers maintaining the plugin is patchy, what hope is there for supporting a particular test framework integration, or looking into deeper issues, or just having some time to respond? I probably shouldn't talk about contributing code, but I think designing a plugin that people would want to contribute should be pretty high on the list of priorities. Keeping it lean and fit for purpose should keep the number of releases down too. Its a busy plugin. Probably the busiest? Delany On Wed, 8 Oct 2025 at 21:30, Tibor Digana <[email protected]> wrote: > @Martin Desruisseaux, > > >> Even if Maven wanted to be independent of JUnit 5... > I know Surefire and JUnit5 quite well, no need to tell me, but I was not > talking about technical things. I am talking about totally different > topics, especially about good Surefire purpose and design with Providers, > risk management and the whole globe of users. > JUnit5 is not a principal shift. > > - Supporting only JUnit5 f/w, deleting Provider API, deleting JUnit3 + > 4 + TestNG is intentionally a bad idea. > The thing happening now goes against the historical continuity because > previous two developers left ASF. > - The Surefire originators who designed the architecture had a good > reasons to introduce Provider API, it is a good design. It separates > concerns of the plugin from the concerns of test frameworks. Although > the > test frameworks have very similar Listeners, they are cohesive with > Surefire Listener and this enables Maven + plugin to be independent of > test > framework implementation across vendors and versions and third party > policies (pricing, license, SDLC) and this provides us with certain > freedom > meaning no coupling (technical coupling and team coupling) between Maven > plugin and third party frameworks. > > >> Keeping direct support of JUnit 3 (as opposed to indirect support > through JUnit 5 API) has a cost. > No, this is not true. > I was a developer of Surefire for many years and JUnit3 + POJO *Provider* > was stable which means I did not need to touch it for quite a long time. > This Provider is pretty simple for simple tests, and the positive is that > the POJO does not require JUnit, this is good for lightweight test > projects. > > > >> Of course there is a risk that we break something. ... difficult to > implement. > It is not difficult to implement. I know Surefire like my shoes. > My problem is that I left during Covid and the historical continuity with > new developers was broken, that's my fault, I am honest now. If this did > not happen then most probably Slawomir would continue straight ahead and > others maybe too. > > Cheers > Tibor > > On Tue, Oct 7, 2025 at 8:22 PM Martin Desruisseaux > <[email protected]> wrote: > > > Hello Tibor > > > > Even if Maven wanted to be independent of JUnit 5, Maven does not need > > to invent its own test API. JUnit5 has a separation between API and > > implementations, so it would be possible to use only the API part if > > necessary (I don't think that it would be necessary, just mentioning > > that we have this possibility). Therefore, a full switch to JUnit 5 > > would not necessarily make us fully prisoner of JUnit 5. > > > > Keeping direct support of JUnit 3 (as opposed to indirect support > > through JUnit 5 API) has a cost. For example, I saw requests on this > > mailing list for testing the different versions of a multi-release JAR > > file. Currently, any classes in META-INF/versions/* will be ignored at > > testing time, because Surefire runs the tests with the `*.class` files > > on the class-path while multi-release works only with JAR files. Last > > Saturday, I wanted to start looking for the possibility of adding an > > option for running the tests many times with different > > `META-INF/versions/*` directories added to the class-path. It was not > > easy to find my way in the Surefire code, and I rely a lot on the > > simplification proposed by Romain before to make any other attempt. > > > > Of course there is a risk that we break something. But the example given > > in the previous email (tests as POJO) should be easy to port. The > > alternative (not removing direct JUnit 3 support) is more like freezing > > the Surefire plugin: no accidental lost of feature, but also less new > > features (e.g. easier testing of multi-release projects) because they > > would be more difficult to implement. > > > > Martin > > > > > > Le 07/10/2025 à 19:16, Tibor Digana a écrit : > > > Romain, > > > > > > to be honest the JUnit guys always wanted to make their own business > and > > > monopoly over the testing phase in Maven, which means using JUnit5 > only, > > > that's it. > > > > > > I have been a JUnit4 developer since cca 2011 or so, and we know each > > other. > > > If I say business then I really mean business and not only the word. > > > Defining report schema and forking mechanism JVMs in JUnit5, these are > > > still the same competition problems between us, and I say it's about > the > > > monopoly, which means who takes the control over these things takes > full > > > control over Maven testing and then Maven becomes completely dependent > on > > > some test framework which is the risk for Maven. > > > > > > This is the main problem but you wouldn't see this unless you have > spent > > > years and years developing both parties as I have. > > > > > > Making Maven strictly dependent only on JUnit5 is the worst decision > > > ever, I am telling you! > > > Btw, experienced Maven guys must know what a strict dependency means > and > > > the consequences too. > > > We are still in the loop for years with deleting something because > > somebody > > > has got a crazy idea in the morning, found out github/dev-factory or > > > anything else, some tool or technology as many other, those tools which > > > come and go every year. > > > > > > On Mon, Sep 29, 2025 at 9:09 AM Romain Manni-Bucau < > > [email protected]> > > > wrote: > > > > > >> Hi all, > > >> > > >> I'd like to start a thread about potentially dropping surefire > totally. > > >> The rational is that surefire (and failsafe) are mainly an abstraction > > >> layer on top of main test providers. > > >> However, since JUnit5 the platform/engine is itself such an > abstraction > > >> layer and a runner. > > >> > > >> On another side, testng and junit4 are slowly getting abandonned - > even > > EE > > >> TCK started to move. > > >> > > >> In terms of additional features we do have the maven site integratoin > - > > but > > >> I doubt it is much used outside and to be honest it can be replaced > > with a > > >> github/dev-factory link with more benefit these days. > > >> > > >> So overall I think we can converge by dropping surefire plugin in > favor > > of > > >> a thin wrapper of junit5 console runner ([1]). > > >> > > >> Short terms I'm sure Christian could help us getting something fast > > based > > >> on its implementation ([2] - including a small surefire compatibility > > mode) > > >> and long term it will reduce the maintenance cost we do have for a > very > > >> poor gain in current world (site and remoting are no more key features > > >> thanks the CI and doc evolution). > > >> > > >> Wdyt? Is maven 4 the mometum to do it? > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > >
