Hi, On Thu, 6 Nov 2025 at 01:32, Jon Harper <[email protected]> wrote: > > Hi, > reading this discussion, it made me think of one of the (I think very > important) missing features when using junit5 and surefire: you can't > run a single parameterized test on the command line (-Dtest=pattern) > > For junit4, there was a syntax like this (see > https://maven.apache.org/surefire/maven-surefire-plugin/test-mojo.html#test > "The Parameterized JUnit runner describes test methods using an index > in brackets, so the non-regex method pattern would become: > #testMethod[*]. If using @Parameters(name="{index}: fib({0})={1}") and > selecting the index e.g. 5 in pattern, the non-regex method pattern > would become #testMethod[5:*]." > > But last time I looked, the parameterized patterns were just never > implemented for junit5, surefire just checks the pattern against the > methodsource name: > https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/TestMethodFilter.java#L51-L53 > . > Personally this forces me to load the project in eclipse just to be > able to rightclick on one of the parameterized tests. > > Do you think this problem would go away as part of this migration ? I > never found an issue for this but maybe someone knows if there's one > already ? Otherwise I can create one if that helps. >
This will not go away and need to be implemented for JUnit 5 :) So yes if you can created an issues this would be great (and add a very simple reproducer might help even more) > (By the way I see that the docs are outdated "-Dtest=MyTest#myMethod. > This is supported for junit 4.x and TestNg." but this is also > supported by junit5 tests currently. Do you want me to do a PR for the I will fix this quickly in the migration PR > docs, or can someone else quickly udpate them ? Mentionning that > junit5 doesn't support the parameterized tests pattern explicitly > would also help I think, I remember losing a few hours desperately > trying to get the junit4 patterns to work with junit5 parameterized > tests out of ignorance) > > Cheers, > Jon > > On Wed, Oct 15, 2025 at 4:24 AM Olivier Lamy <[email protected]> wrote: > > > > Thanks, Delany. Your message perfectly reflects my own opinion, even > > as a long-time committer on the project. > > > > We never really closed this thread with a clear conclusion or next > > steps, so I’d like to summarize the current state and propose a path > > forward. > > > > From the discussion so far, there seems to be a general consensus on > > simplifying Surefire’s codebase by migrating test execution to the > > JUnit Platform engine. This would be done in a moderately conservative > > way, as described earlier in this thread: > > https://lists.apache.org/thread/vovj7srtpjlk67z7ocowzbz3g3z3878t > > > > Quick Summary of the current situation: > > Surefire currently runs tests through multiple provider modules (JUnit > > 3, 4, 5, TestNG, etc.). Each provider implements similar interfaces > > and classes, which means that any change to reporters or listeners > > requires parallel updates across all providers. > > This architecture made sense in the past, when there wasn’t a unified > > way to run JUnit 3, JUnit 4, TestNG, and JUnit 5 tests. However, it > > now results in significant code duplication and added complexity > > whenever changes are made. > > > > Proposed approach: > > A few years ago, the JUnit team introduced the JUnit Platform, which > > can run not only JUnit 5 tests (as used by the junit-platform > > provider) but also JUnit 3, 4, and TestNG tests. > > To simplify the codebase, we propose removing the dedicated providers > > for JUnit 3, 4, and TestNG, and using the junit-platform provider as > > the single unified test runner. Please note the provider architecture > > is still supported so everybody is free to implement/run their own but > > we need to stop maintaining so many. > > > > > > This change would mean that JUnit versions older than 4.12 will no > > longer be supported. > > However, existing JUnit 3 and JUnit 4 tests (≤ 4.11) can still be > > executed without modification by simply upgrading the JUnit dependency > > to at least version 4.12. > > > > Reference: > > A work-in-progress PR is available here: > > https://github.com/apache/maven-surefire/pull/3179 > > > > Even if it’s a significant PR with still work to do (Thanks, > > Sebastian, for his help), which removes a considerable amount of > > legacy code, it seems to be the way to go and represents some > > consensus reached with this thread. > > > > Onward > > Olivier > > > > > > On Thu, 9 Oct 2025 at 17:23, Delany <[email protected]> wrote: > > > > > > Hi Tibor, > > > > > > Another popular principle out there is do one thing well. > > > The provider principle is already in play with Maven's plugin > > > architecture. > > > > > > I would much rather have a test plugin purpose built for the test > > > framework > > > I use. > > > Its fine until you want to do something a bit extra, > > > https://github.com/apache/maven-surefire/issues/3146 > > > > > > Without knowing anything about the internals, my perception is that > > > surefire is hugely complicated with a lot of needless ceremony. > > > I don't feel that way about any other plugin. > > > You talk about separation of concerns, but at least from the outside it > > > seems so many concerns are overlapping. > > > Configuration, test discovery. Threading is configurable in all three, > > > junit, surefire, and maven, not to mention the unit under test. > > > > > > The gold standard is if test frameworks provide their own plugin so there > > > is clear responsibility for it, and maybe a plugin that shares its > > > namesake > > > will encourage some buy-in. > > > If the lineage of developers maintaining the plugin is patchy, what hope > > > is > > > there for supporting a particular test framework integration, > > > or looking into deeper issues, or just having some time to respond? > > > > > > I probably shouldn't talk about contributing code, but I think designing a > > > plugin that people would want to contribute should be pretty high on the > > > list of priorities. > > > Keeping it lean and fit for purpose should keep the number of releases > > > down > > > too. Its a busy plugin. Probably the busiest? > > > > > > Delany > > > > > > On Wed, 8 Oct 2025 at 21:30, Tibor Digana <[email protected]> wrote: > > > > > > > @Martin Desruisseaux, > > > > > > > > >> Even if Maven wanted to be independent of JUnit 5... > > > > I know Surefire and JUnit5 quite well, no need to tell me, but I was not > > > > talking about technical things. I am talking about totally different > > > > topics, especially about good Surefire purpose and design with > > > > Providers, > > > > risk management and the whole globe of users. > > > > JUnit5 is not a principal shift. > > > > > > > > - Supporting only JUnit5 f/w, deleting Provider API, deleting JUnit3 > > > > + > > > > 4 + TestNG is intentionally a bad idea. > > > > The thing happening now goes against the historical continuity > > > > because > > > > previous two developers left ASF. > > > > - The Surefire originators who designed the architecture had a good > > > > reasons to introduce Provider API, it is a good design. It separates > > > > concerns of the plugin from the concerns of test frameworks. Although > > > > the > > > > test frameworks have very similar Listeners, they are cohesive with > > > > Surefire Listener and this enables Maven + plugin to be independent > > > > of > > > > test > > > > framework implementation across vendors and versions and third party > > > > policies (pricing, license, SDLC) and this provides us with certain > > > > freedom > > > > meaning no coupling (technical coupling and team coupling) between > > > > Maven > > > > plugin and third party frameworks. > > > > > > > > >> Keeping direct support of JUnit 3 (as opposed to indirect support > > > > through JUnit 5 API) has a cost. > > > > No, this is not true. > > > > I was a developer of Surefire for many years and JUnit3 + POJO > > > > *Provider* > > > > was stable which means I did not need to touch it for quite a long time. > > > > This Provider is pretty simple for simple tests, and the positive is > > > > that > > > > the POJO does not require JUnit, this is good for lightweight test > > > > projects. > > > > > > > > > > > > >> Of course there is a risk that we break something. ... difficult to > > > > implement. > > > > It is not difficult to implement. I know Surefire like my shoes. > > > > My problem is that I left during Covid and the historical continuity > > > > with > > > > new developers was broken, that's my fault, I am honest now. If this did > > > > not happen then most probably Slawomir would continue straight ahead and > > > > others maybe too. > > > > > > > > Cheers > > > > Tibor > > > > > > > > On Tue, Oct 7, 2025 at 8:22 PM Martin Desruisseaux > > > > <[email protected]> wrote: > > > > > > > > > Hello Tibor > > > > > > > > > > Even if Maven wanted to be independent of JUnit 5, Maven does not need > > > > > to invent its own test API. JUnit5 has a separation between API and > > > > > implementations, so it would be possible to use only the API part if > > > > > necessary (I don't think that it would be necessary, just mentioning > > > > > that we have this possibility). Therefore, a full switch to JUnit 5 > > > > > would not necessarily make us fully prisoner of JUnit 5. > > > > > > > > > > Keeping direct support of JUnit 3 (as opposed to indirect support > > > > > through JUnit 5 API) has a cost. For example, I saw requests on this > > > > > mailing list for testing the different versions of a multi-release JAR > > > > > file. Currently, any classes in META-INF/versions/* will be ignored at > > > > > testing time, because Surefire runs the tests with the `*.class` files > > > > > on the class-path while multi-release works only with JAR files. Last > > > > > Saturday, I wanted to start looking for the possibility of adding an > > > > > option for running the tests many times with different > > > > > `META-INF/versions/*` directories added to the class-path. It was not > > > > > easy to find my way in the Surefire code, and I rely a lot on the > > > > > simplification proposed by Romain before to make any other attempt. > > > > > > > > > > Of course there is a risk that we break something. But the example > > > > > given > > > > > in the previous email (tests as POJO) should be easy to port. The > > > > > alternative (not removing direct JUnit 3 support) is more like > > > > > freezing > > > > > the Surefire plugin: no accidental lost of feature, but also less new > > > > > features (e.g. easier testing of multi-release projects) because they > > > > > would be more difficult to implement. > > > > > > > > > > Martin > > > > > > > > > > > > > > > Le 07/10/2025 à 19:16, Tibor Digana a écrit : > > > > > > Romain, > > > > > > > > > > > > to be honest the JUnit guys always wanted to make their own business > > > > and > > > > > > monopoly over the testing phase in Maven, which means using JUnit5 > > > > only, > > > > > > that's it. > > > > > > > > > > > > I have been a JUnit4 developer since cca 2011 or so, and we know > > > > > > each > > > > > other. > > > > > > If I say business then I really mean business and not only the word. > > > > > > Defining report schema and forking mechanism JVMs in JUnit5, these > > > > > > are > > > > > > still the same competition problems between us, and I say it's about > > > > the > > > > > > monopoly, which means who takes the control over these things takes > > > > full > > > > > > control over Maven testing and then Maven becomes completely > > > > > > dependent > > > > on > > > > > > some test framework which is the risk for Maven. > > > > > > > > > > > > This is the main problem but you wouldn't see this unless you have > > > > spent > > > > > > years and years developing both parties as I have. > > > > > > > > > > > > Making Maven strictly dependent only on JUnit5 is the worst decision > > > > > > ever, I am telling you! > > > > > > Btw, experienced Maven guys must know what a strict dependency means > > > > and > > > > > > the consequences too. > > > > > > We are still in the loop for years with deleting something because > > > > > somebody > > > > > > has got a crazy idea in the morning, found out github/dev-factory > > > > > > or > > > > > > anything else, some tool or technology as many other, those tools > > > > > > which > > > > > > come and go every year. > > > > > > > > > > > > On Mon, Sep 29, 2025 at 9:09 AM Romain Manni-Bucau < > > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > >> Hi all, > > > > > >> > > > > > >> I'd like to start a thread about potentially dropping surefire > > > > totally. > > > > > >> The rational is that surefire (and failsafe) are mainly an > > > > > >> abstraction > > > > > >> layer on top of main test providers. > > > > > >> However, since JUnit5 the platform/engine is itself such an > > > > abstraction > > > > > >> layer and a runner. > > > > > >> > > > > > >> On another side, testng and junit4 are slowly getting abandonned - > > > > even > > > > > EE > > > > > >> TCK started to move. > > > > > >> > > > > > >> In terms of additional features we do have the maven site > > > > > >> integratoin > > > > - > > > > > but > > > > > >> I doubt it is much used outside and to be honest it can be replaced > > > > > with a > > > > > >> github/dev-factory link with more benefit these days. > > > > > >> > > > > > >> So overall I think we can converge by dropping surefire plugin in > > > > favor > > > > > of > > > > > >> a thin wrapper of junit5 console runner ([1]). > > > > > >> > > > > > >> Short terms I'm sure Christian could help us getting something fast > > > > > based > > > > > >> on its implementation ([2] - including a small surefire > > > > > >> compatibility > > > > > mode) > > > > > >> and long term it will reduce the maintenance cost we do have for a > > > > very > > > > > >> poor gain in current world (site and remoting are no more key > > > > > >> features > > > > > >> thanks the CI and doc evolution). > > > > > >> > > > > > >> Wdyt? Is maven 4 the mometum to do it? > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > > > > To unsubscribe, e-mail: [email protected] > > > > > For additional commands, e-mail: [email protected] > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [email protected] > > For additional commands, e-mail: [email protected] > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
