date:20210910

Hi Emond,

This section of code is executed for the forkCount > 1
https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L194-L202
The above part runs if the forkCount is 1.

Do you think that using the session together with the try-with-resource in
the selected section of code makes the trick?
Pls check it your local in your local branch and your project.

T



On Fri, Sep 10, 2021 at 11:33 PM Emond Papegaaij 
wrote:

> Hi Tibor,
>
> That's what I implemented, although I couldn't use the fancy try, because
> of the way the code is structured. The LauncherSession is started by
> LazyLauncher. This will allow registering a listener for opening and
> closing the session, given a place for pre and post fixtures.
>
> Best regards,
> Emond
>
> Op vr 10 sep. 2021 23:00 schreef Tibor Digana :
>
> > Hi Emond,
> >
> > Are you looking for this?
> >
> >
> https://github.com/junit-team/junit5/blob/main/documentation/src/test/java/example/UsingTheLauncherDemo.java#L86-L96
> >
> > On Fri, Sep 10, 2021 at 10:49 PM Emond Papegaaij <
> > emond.papega...@gmail.com>
> > wrote:
> >
> > > On Fri, Sep 10, 2021 at 9:15 PM Emond Papegaaij <
> > emond.papega...@gmail.com
> > > >
> > > wrote:
> > >
> > > > On Fri, Sep 10, 2021 at 8:41 PM Christian Stein 
> > > > wrote:
> > > >
> > > >> On Fri, Sep 10, 2021 at 8:27 PM Emond Papegaaij <
> > > >> emond.papega...@gmail.com>
> > > >> wrote:
> > > >>
> > > >> For now, I think the LauncherSession is the best way to at least
> > provide
> > > >> > some hooks for pre and post fixtures. It shouldn't be too hard to
> > get
> > > >> this
> > > >> > in the current code base with backwards compatibility for JUnit
> > > Platform
> > > >> > 1.7 and older. I'll see if I can create a pull request for this
> > > >> somewhere
> > > >> > next week.
> > > >> >
> > > >>
> > > >> Before investing too much energy into a PR, please start a
> discussion
> > or
> > > >> open a feature request first - chances are that what you seek to
> > achieve
> > > >> is already possible, or somebody else has filed a similar request.
> > > >
> > > >
> > > > You mean at JUnit? I'm pretty sure that it's currently not possible.
> > I've
> > > > been stepping through that code for quite some time now, and the
> whole
> > > > TestPlan construction is very static. There doesn't seem to be an
> open
> > > > request for this at the moment.
> > > >
> > > > The API for a LauncherSession is very simple and I think it's a good
> > idea
> > > > to use it anyway. The hardest part will be to only start the session
> > when
> > > > using 1.8 or higher.
> > > >
> > >
> > > That was easier than I thought. I've already got a working POC at
> > >
> > >
> >
> https://github.com/topicusonderwijs/maven-surefire/commit/b112130170c244846d5e35353a4c90afaf42aff1
> > > . The only problems at the moment are that some tests fail because a
> > > TestPlan is immutable in 1.8, the main process creates a
> LauncherSession
> > > that is never closed and I screwed up the formatting in some places.
> With
> > > this change, a LauncherSession is created when JUnit Platform 1.8 is
> > used.
> > > This session is closed when all tests have been executed. When running
> on
> > > 1.7 or lower, it falls back to the old behavior. I've verified that
> only
> > > one LauncherSession is created per fork when running with forkCount >
> 1.
> > >
> > > Best regards,
> > > Emond
> > >
> >
>

Re: JUnit Platform, forkCount > 1 and the TestPlan

Hi Tibor,

That's what I implemented, although I couldn't use the fancy try, because
of the way the code is structured. The LauncherSession is started by
LazyLauncher. This will allow registering a listener for opening and
closing the session, given a place for pre and post fixtures.

Best regards,
Emond

Op vr 10 sep. 2021 23:00 schreef Tibor Digana :

> Hi Emond,
>
> Are you looking for this?
>
> https://github.com/junit-team/junit5/blob/main/documentation/src/test/java/example/UsingTheLauncherDemo.java#L86-L96
>
> On Fri, Sep 10, 2021 at 10:49 PM Emond Papegaaij <
> emond.papega...@gmail.com>
> wrote:
>
> > On Fri, Sep 10, 2021 at 9:15 PM Emond Papegaaij <
> emond.papega...@gmail.com
> > >
> > wrote:
> >
> > > On Fri, Sep 10, 2021 at 8:41 PM Christian Stein 
> > > wrote:
> > >
> > >> On Fri, Sep 10, 2021 at 8:27 PM Emond Papegaaij <
> > >> emond.papega...@gmail.com>
> > >> wrote:
> > >>
> > >> For now, I think the LauncherSession is the best way to at least
> provide
> > >> > some hooks for pre and post fixtures. It shouldn't be too hard to
> get
> > >> this
> > >> > in the current code base with backwards compatibility for JUnit
> > Platform
> > >> > 1.7 and older. I'll see if I can create a pull request for this
> > >> somewhere
> > >> > next week.
> > >> >
> > >>
> > >> Before investing too much energy into a PR, please start a discussion
> or
> > >> open a feature request first - chances are that what you seek to
> achieve
> > >> is already possible, or somebody else has filed a similar request.
> > >
> > >
> > > You mean at JUnit? I'm pretty sure that it's currently not possible.
> I've
> > > been stepping through that code for quite some time now, and the whole
> > > TestPlan construction is very static. There doesn't seem to be an open
> > > request for this at the moment.
> > >
> > > The API for a LauncherSession is very simple and I think it's a good
> idea
> > > to use it anyway. The hardest part will be to only start the session
> when
> > > using 1.8 or higher.
> > >
> >
> > That was easier than I thought. I've already got a working POC at
> >
> >
> https://github.com/topicusonderwijs/maven-surefire/commit/b112130170c244846d5e35353a4c90afaf42aff1
> > . The only problems at the moment are that some tests fail because a
> > TestPlan is immutable in 1.8, the main process creates a LauncherSession
> > that is never closed and I screwed up the formatting in some places. With
> > this change, a LauncherSession is created when JUnit Platform 1.8 is
> used.
> > This session is closed when all tests have been executed. When running on
> > 1.7 or lower, it falls back to the old behavior. I've verified that only
> > one LauncherSession is created per fork when running with forkCount > 1.
> >
> > Best regards,
> > Emond
> >
>

Re: JUnit Platform, forkCount > 1 and the TestPlan

Hi Emond,

Are you looking for this?
https://github.com/junit-team/junit5/blob/main/documentation/src/test/java/example/UsingTheLauncherDemo.java#L86-L96

On Fri, Sep 10, 2021 at 10:49 PM Emond Papegaaij 
wrote:

> On Fri, Sep 10, 2021 at 9:15 PM Emond Papegaaij  >
> wrote:
>
> > On Fri, Sep 10, 2021 at 8:41 PM Christian Stein 
> > wrote:
> >
> >> On Fri, Sep 10, 2021 at 8:27 PM Emond Papegaaij <
> >> emond.papega...@gmail.com>
> >> wrote:
> >>
> >> For now, I think the LauncherSession is the best way to at least provide
> >> > some hooks for pre and post fixtures. It shouldn't be too hard to get
> >> this
> >> > in the current code base with backwards compatibility for JUnit
> Platform
> >> > 1.7 and older. I'll see if I can create a pull request for this
> >> somewhere
> >> > next week.
> >> >
> >>
> >> Before investing too much energy into a PR, please start a discussion or
> >> open a feature request first - chances are that what you seek to achieve
> >> is already possible, or somebody else has filed a similar request.
> >
> >
> > You mean at JUnit? I'm pretty sure that it's currently not possible. I've
> > been stepping through that code for quite some time now, and the whole
> > TestPlan construction is very static. There doesn't seem to be an open
> > request for this at the moment.
> >
> > The API for a LauncherSession is very simple and I think it's a good idea
> > to use it anyway. The hardest part will be to only start the session when
> > using 1.8 or higher.
> >
>
> That was easier than I thought. I've already got a working POC at
>
> https://github.com/topicusonderwijs/maven-surefire/commit/b112130170c244846d5e35353a4c90afaf42aff1
> . The only problems at the moment are that some tests fail because a
> TestPlan is immutable in 1.8, the main process creates a LauncherSession
> that is never closed and I screwed up the formatting in some places. With
> this change, a LauncherSession is created when JUnit Platform 1.8 is used.
> This session is closed when all tests have been executed. When running on
> 1.7 or lower, it falls back to the old behavior. I've verified that only
> one LauncherSession is created per fork when running with forkCount > 1.
>
> Best regards,
> Emond
>

Re: JUnit Platform, forkCount > 1 and the TestPlan

On Fri, Sep 10, 2021 at 9:15 PM Emond Papegaaij 
wrote:

> On Fri, Sep 10, 2021 at 8:41 PM Christian Stein 
> wrote:
>
>> On Fri, Sep 10, 2021 at 8:27 PM Emond Papegaaij <
>> emond.papega...@gmail.com>
>> wrote:
>>
>> For now, I think the LauncherSession is the best way to at least provide
>> > some hooks for pre and post fixtures. It shouldn't be too hard to get
>> this
>> > in the current code base with backwards compatibility for JUnit Platform
>> > 1.7 and older. I'll see if I can create a pull request for this
>> somewhere
>> > next week.
>> >
>>
>> Before investing too much energy into a PR, please start a discussion or
>> open a feature request first - chances are that what you seek to achieve
>> is already possible, or somebody else has filed a similar request.
>
>
> You mean at JUnit? I'm pretty sure that it's currently not possible. I've
> been stepping through that code for quite some time now, and the whole
> TestPlan construction is very static. There doesn't seem to be an open
> request for this at the moment.
>
> The API for a LauncherSession is very simple and I think it's a good idea
> to use it anyway. The hardest part will be to only start the session when
> using 1.8 or higher.
>

That was easier than I thought. I've already got a working POC at
https://github.com/topicusonderwijs/maven-surefire/commit/b112130170c244846d5e35353a4c90afaf42aff1
. The only problems at the moment are that some tests fail because a
TestPlan is immutable in 1.8, the main process creates a LauncherSession
that is never closed and I screwed up the formatting in some places. With
this change, a LauncherSession is created when JUnit Platform 1.8 is used.
This session is closed when all tests have been executed. When running on
1.7 or lower, it falls back to the old behavior. I've verified that only
one LauncherSession is created per fork when running with forkCount > 1.

Best regards,
Emond

Re: JUnit Platform, forkCount > 1 and the TestPlan

On Fri, Sep 10, 2021 at 8:41 PM Christian Stein  wrote:

> On Fri, Sep 10, 2021 at 8:27 PM Emond Papegaaij  >
> wrote:
>
> For now, I think the LauncherSession is the best way to at least provide
> > some hooks for pre and post fixtures. It shouldn't be too hard to get
> this
> > in the current code base with backwards compatibility for JUnit Platform
> > 1.7 and older. I'll see if I can create a pull request for this somewhere
> > next week.
> >
>
> Before investing too much energy into a PR, please start a discussion or
> open a feature request first - chances are that what you seek to achieve
> is already possible, or somebody else has filed a similar request.

You mean at JUnit? I'm pretty sure that it's currently not possible. I've
been stepping through that code for quite some time now, and the whole
TestPlan construction is very static. There doesn't seem to be an open
request for this at the moment.

The API for a LauncherSession is very simple and I think it's a good idea
to use it anyway. The hardest part will be to only start the session when
using 1.8 or higher.

Best regards,
Emond

Re: JUnit Platform, forkCount > 1 and the TestPlan

Sorry for typos, I sent it from my mobile.
T

On Fri, Sep 10, 2021 at 6:18 PM Tibor Digana  wrote:

> We have to dig into Junit 5. Surefire is a streamer of classes across the
> forks which is our load balancer. Therefore each class is running
> separately, pre and post fixtures. If the Junit 5 used Java Streamer
> including dome kind of control of fixtures then web would have this issue.
>
> Dňa pi 10. 9. 2021, 18:03 Emond Papegaaij 
> napísal(a):
>
>> On Fri, Sep 10, 2021 at 5:06 PM Tibor Digana 
>> wrote:
>>
>> > If you use forkCount > 1, the Surefire loads test classes via load
>> > balancer.
>> > If you use default forkCount = 0, all the classes are run eagerly as a
>> > suite via JUnit5 Launcher in one shot.
>> >
>>
>> Yes, this is what I saw in the JUnitPlatformProvider. The cause of my
>> issues lies in the difference in events triggered by Arquillian due to the
>> classes executed one by one. The demo-project attached to the ticket
>> demonstrates this in just a few lines of code.
>>
>> If you are aiming for Arquillian, testing the applications in the
>> > application server, you should use maven-failsafe-plugin which has
>> another
>> > testing model.
>> > See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin
>> > Documentation (apache.org)
>> > <
>> https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html>
>> >
>> https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html
>> > There are Maven phases:
>> > pre-integration-test
>> > integration-test
>> > post-integration-test
>> > verify
>> >
>> > Therefore you should start the application server in the phase
>> > pre-integration-test.
>> > Accordingly, you should stop it in the phase post-integration-test.
>> >
>> > Then use Failsafe plugin in the phases integration-test and verify.
>> >
>>
>> Unfortunately, this is not how arquillian is setup. You can run tests
>> against running servers (which you can then start in
>> pre-integration-test),
>> but using arquillian cube, these containers can be started as part of the
>> test lifecycle. Also, Arquillian is just a single incarnation of the
>> underlying issue. Every extension that takes more than a few seconds to
>> start and is built to only initialize once for a suite will trigger this
>> same problem (as can be seen by the demo project). As surefire and
>> failsafe
>> use the same JUnitPlatformProvider, the problem will also be the same.
>>
>> What do you think about the solution of using a LauncherSession to bind
>> the
>> multiple executions together? This will require JUnit Platform 1.8 and
>> changes in the JUnitPlatformProvider. However, I'm not sure what the
>> intended audience of LauncherSessionListener is.
>>
>> Best regards,
>> Emond
>>
>> On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij <
>> emond.papega...@gmail.com>
>> > wrote:
>> >
>> > > Hi all,
>> > >
>> > > First of all, sorry for the lengthy post. I decided to add some
>> context
>> > to
>> > > explain things a bit, but it resulted in quite a long e-mail. For the
>> > past
>> > > few weeks I've been trying to come up with a solution for the issue I
>> > > filled under SUREFIRE-1935, but I'm getting stuck and starting to feel
>> > like
>> > > the issue cannot be solved with the current JUnit Platform API. To
>> give
>> > > some perspective into why this issue is important for us, I first
>> have to
>> > > explain a bit about our setup.
>> > >
>> > > We write our tests in the Spock framework and use Arquillian to run
>> the
>> > > tests in the application container. Some of our tests, especially the
>> > > Selenium based tests, require quite some setup and tear down. Prior to
>> > > starting the test, Arquillian cube builds and starts several docker
>> > > containers and the tests are run against these containers. We
>> currently
>> > use
>> > > the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to
>> > > Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For
>> this,
>> > > I've already started working on a Spock extension that integrates
>> Spock 2
>> > > in the Arquillian test life cycle [1]. This extension is inspired by
>> the
>> > > (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global
>> > > registration to keep track of the state managed by Arquilllian. This
>> will
>> > > end up somewhere at the root of the TestPlan (for example see [3]).
>> > >
>> > > Because our tests are quite extensive, with a total run time of 6
>> hours,
>> > we
>> > > run them with a forkCount of 8, greatly reducing the total duration.
>> > > However, this is where SUREFIRE-1935 comes into play. With a
>> forkCount >
>> > 1,
>> > > the entire test life cycle is started over and over again for every
>> test
>> > > class. This happens in JUnitPlatformProvider at line 197 [4]. This
>> > results
>> > > in the entire Arquillian suite being torn down and setup for every
>> class,
>> > > in our case adding several minutes to the execution of every test
>> class

Re: JUnit Platform, forkCount > 1 and the TestPlan

2021-09-10 Thread Christian Stein

On Fri, Sep 10, 2021 at 8:27 PM Emond Papegaaij 
wrote:

> [...]

For now, I think the LauncherSession is the best way to at least provide
> some hooks for pre and post fixtures. It shouldn't be too hard to get this
> in the current code base with backwards compatibility for JUnit Platform
> 1.7 and older. I'll see if I can create a pull request for this somewhere
> next week.
>

Before investing too much energy into a PR, please start a discussion or
open a feature request first - chances are that what you seek to achieve
is already possible, or somebody else has filed a similar request.

Cheers,
Christian

Re: JUnit Platform, forkCount > 1 and the TestPlan

On Fri, Sep 10, 2021 at 6:18 PM Tibor Digana  wrote:

> We have to dig into Junit 5. Surefire is a streamer of classes across the
> forks which is our load balancer. Therefore each class is running
> separately, pre and post fixtures. If the Junit 5 used Java Streamer
> including dome kind of control of fixtures then web would have this issue.
>

I totally agree. To me JUnit 5 looks overly complicated. Everything can be
customized, but the overall result is that even the simple things are hard
or impossible. I've looked at various ways to change the TestPlan, but the
only way I could think of to support streaming test classes is to have some
sort of streaming TestEngine in between. I'll file a feature request at
JUnit for support for this.

For now, I think the LauncherSession is the best way to at least provide
some hooks for pre and post fixtures. It shouldn't be too hard to get this
in the current code base with backwards compatibility for JUnit Platform
1.7 and older. I'll see if I can create a pull request for this somewhere
next week.

Best regards,
Emond

[GitHub] [maven-site] michael-o commented on pull request #257: run.md: change example to use the "compile" phase



michael-o commented on pull request #257:
URL: https://github.com/apache/maven-site/pull/257#issuecomment-917065645


   @waldyrious Muito obrigado!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] [maven-site] michael-o merged pull request #257: run.md: change example to use the "compile" phase



michael-o merged pull request #257:
URL: https://github.com/apache/maven-site/pull/257


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] [maven-site] waldyrious commented on pull request #257: run.md: change example to use the "compile" phase



waldyrious commented on pull request #257:
URL: https://github.com/apache/maven-site/pull/257#issuecomment-917045041


   Ok, updated with the changes discussed above.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: JUnit Platform, forkCount > 1 and the TestPlan

We have to dig into Junit 5. Surefire is a streamer of classes across the
forks which is our load balancer. Therefore each class is running
separately, pre and post fixtures. If the Junit 5 used Java Streamer
including dome kind of control of fixtures then web would have this issue.

Dňa pi 10. 9. 2021, 18:03 Emond Papegaaij 
napísal(a):

> On Fri, Sep 10, 2021 at 5:06 PM Tibor Digana 
> wrote:
>
> > If you use forkCount > 1, the Surefire loads test classes via load
> > balancer.
> > If you use default forkCount = 0, all the classes are run eagerly as a
> > suite via JUnit5 Launcher in one shot.
> >
>
> Yes, this is what I saw in the JUnitPlatformProvider. The cause of my
> issues lies in the difference in events triggered by Arquillian due to the
> classes executed one by one. The demo-project attached to the ticket
> demonstrates this in just a few lines of code.
>
> If you are aiming for Arquillian, testing the applications in the
> > application server, you should use maven-failsafe-plugin which has
> another
> > testing model.
> > See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin
> > Documentation (apache.org)
> > <
> https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html>
> > https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html
> > There are Maven phases:
> > pre-integration-test
> > integration-test
> > post-integration-test
> > verify
> >
> > Therefore you should start the application server in the phase
> > pre-integration-test.
> > Accordingly, you should stop it in the phase post-integration-test.
> >
> > Then use Failsafe plugin in the phases integration-test and verify.
> >
>
> Unfortunately, this is not how arquillian is setup. You can run tests
> against running servers (which you can then start in pre-integration-test),
> but using arquillian cube, these containers can be started as part of the
> test lifecycle. Also, Arquillian is just a single incarnation of the
> underlying issue. Every extension that takes more than a few seconds to
> start and is built to only initialize once for a suite will trigger this
> same problem (as can be seen by the demo project). As surefire and failsafe
> use the same JUnitPlatformProvider, the problem will also be the same.
>
> What do you think about the solution of using a LauncherSession to bind the
> multiple executions together? This will require JUnit Platform 1.8 and
> changes in the JUnitPlatformProvider. However, I'm not sure what the
> intended audience of LauncherSessionListener is.
>
> Best regards,
> Emond
>
> On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij <
> emond.papega...@gmail.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > First of all, sorry for the lengthy post. I decided to add some context
> > to
> > > explain things a bit, but it resulted in quite a long e-mail. For the
> > past
> > > few weeks I've been trying to come up with a solution for the issue I
> > > filled under SUREFIRE-1935, but I'm getting stuck and starting to feel
> > like
> > > the issue cannot be solved with the current JUnit Platform API. To give
> > > some perspective into why this issue is important for us, I first have
> to
> > > explain a bit about our setup.
> > >
> > > We write our tests in the Spock framework and use Arquillian to run the
> > > tests in the application container. Some of our tests, especially the
> > > Selenium based tests, require quite some setup and tear down. Prior to
> > > starting the test, Arquillian cube builds and starts several docker
> > > containers and the tests are run against these containers. We currently
> > use
> > > the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to
> > > Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For
> this,
> > > I've already started working on a Spock extension that integrates
> Spock 2
> > > in the Arquillian test life cycle [1]. This extension is inspired by
> the
> > > (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global
> > > registration to keep track of the state managed by Arquilllian. This
> will
> > > end up somewhere at the root of the TestPlan (for example see [3]).
> > >
> > > Because our tests are quite extensive, with a total run time of 6
> hours,
> > we
> > > run them with a forkCount of 8, greatly reducing the total duration.
> > > However, this is where SUREFIRE-1935 comes into play. With a forkCount
> >
> > 1,
> > > the entire test life cycle is started over and over again for every
> test
> > > class. This happens in JUnitPlatformProvider at line 197 [4]. This
> > results
> > > in the entire Arquillian suite being torn down and setup for every
> class,
> > > in our case adding several minutes to the execution of every test class
> > > because the docker setup is done over and over again.
> > >
> > > To overcome this issue, the state from one test class execution has to
> be
> > > carried to the next. It seems the LauncherSession (introduced in JUnit
> > 5.8)
> > > is m

Re: JUnit Platform, forkCount > 1 and the TestPlan

On Fri, Sep 10, 2021 at 5:06 PM Tibor Digana  wrote:

> If you use forkCount > 1, the Surefire loads test classes via load
> balancer.
> If you use default forkCount = 0, all the classes are run eagerly as a
> suite via JUnit5 Launcher in one shot.
>

Yes, this is what I saw in the JUnitPlatformProvider. The cause of my
issues lies in the difference in events triggered by Arquillian due to the
classes executed one by one. The demo-project attached to the ticket
demonstrates this in just a few lines of code.

If you are aiming for Arquillian, testing the applications in the
> application server, you should use maven-failsafe-plugin which has another
> testing model.
> See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin
> Documentation (apache.org)
> 
> https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html
> There are Maven phases:
> pre-integration-test
> integration-test
> post-integration-test
> verify
>
> Therefore you should start the application server in the phase
> pre-integration-test.
> Accordingly, you should stop it in the phase post-integration-test.
>
> Then use Failsafe plugin in the phases integration-test and verify.
>

Unfortunately, this is not how arquillian is setup. You can run tests
against running servers (which you can then start in pre-integration-test),
but using arquillian cube, these containers can be started as part of the
test lifecycle. Also, Arquillian is just a single incarnation of the
underlying issue. Every extension that takes more than a few seconds to
start and is built to only initialize once for a suite will trigger this
same problem (as can be seen by the demo project). As surefire and failsafe
use the same JUnitPlatformProvider, the problem will also be the same.

What do you think about the solution of using a LauncherSession to bind the
multiple executions together? This will require JUnit Platform 1.8 and
changes in the JUnitPlatformProvider. However, I'm not sure what the
intended audience of LauncherSessionListener is.

Best regards,
Emond

On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij 
> wrote:
>
> > Hi all,
> >
> > First of all, sorry for the lengthy post. I decided to add some context
> to
> > explain things a bit, but it resulted in quite a long e-mail. For the
> past
> > few weeks I've been trying to come up with a solution for the issue I
> > filled under SUREFIRE-1935, but I'm getting stuck and starting to feel
> like
> > the issue cannot be solved with the current JUnit Platform API. To give
> > some perspective into why this issue is important for us, I first have to
> > explain a bit about our setup.
> >
> > We write our tests in the Spock framework and use Arquillian to run the
> > tests in the application container. Some of our tests, especially the
> > Selenium based tests, require quite some setup and tear down. Prior to
> > starting the test, Arquillian cube builds and starts several docker
> > containers and the tests are run against these containers. We currently
> use
> > the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to
> > Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For this,
> > I've already started working on a Spock extension that integrates Spock 2
> > in the Arquillian test life cycle [1]. This extension is inspired by the
> > (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global
> > registration to keep track of the state managed by Arquilllian. This will
> > end up somewhere at the root of the TestPlan (for example see [3]).
> >
> > Because our tests are quite extensive, with a total run time of 6 hours,
> we
> > run them with a forkCount of 8, greatly reducing the total duration.
> > However, this is where SUREFIRE-1935 comes into play. With a forkCount >
> 1,
> > the entire test life cycle is started over and over again for every test
> > class. This happens in JUnitPlatformProvider at line 197 [4]. This
> results
> > in the entire Arquillian suite being torn down and setup for every class,
> > in our case adding several minutes to the execution of every test class
> > because the docker setup is done over and over again.
> >
> > To overcome this issue, the state from one test class execution has to be
> > carried to the next. It seems the LauncherSession (introduced in JUnit
> 5.8)
> > is meant to close this gap. However, this would mean my extension would
> > also need to implement a LauncherSessionListener, and I'm not sure if
> > extensions are supposed to integrate with the launcher as well. Also, for
> > this surefire would need to start a session prior to the tests, and close
> > it when done. I think this is a good idea anyway when running on platform
> > 1.8 or higher.
> >
> > Another solution could be a (sort of) dynamic test that produces the
> tests
> > to be run one by one. However, here my knowledge of JUnit really falls
> > short. I've got no idea

[GitHub] [maven-site] michael-o merged pull request #258: Add Gradle Enterprise extension



michael-o merged pull request #258:
URL: https://github.com/apache/maven-site/pull/258


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] [maven-site] michael-o commented on pull request #257: run.md: change example to use the "compile" phase



michael-o commented on pull request #257:
URL: https://github.com/apache/maven-site/pull/257#issuecomment-917007266


   > 
   > 
   > Well, in [Introduction to the Build Lifecycle § Usual Command Line 
Calls](http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#usual-command-line-calls)
 there's this passage:
   > 
   > > If you are uncertain what you want, the preferred phase to call is
   > > `mvn verify`
   > 
   > So perhaps it would be good to use `verify` in this example as well, for 
consistency?
   
   Agree.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: JUnit Platform, forkCount > 1 and the TestPlan

Please read the documentation
https://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html

On Fri, Sep 10, 2021 at 5:05 PM Tibor Digana  wrote:

> If you use forkCount > 1, the Surefire loads test classes via load
> balancer.
> If you use default forkCount = 0, all the classes are run eagerly as a
> suite via JUnit5 Launcher in one shot.
>
> If you are aiming for Arquillian, testing the applications in the
> application server, you should use maven-failsafe-plugin which has another
> testing model.
> See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin
> Documentation (apache.org)
> 
> https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html
> There are Maven phases:
> pre-integration-test
> integration-test
> post-integration-test
> verify
>
> Therefore you should start the application server in the phase
> pre-integration-test.
> Accordingly, you should stop it in the phase post-integration-test.
>
> Then use Failsafe plugin in the phases integration-test and verify.
>
> Cheers
> Tibor
>
>
>
> On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij <
> emond.papega...@gmail.com> wrote:
>
>> Hi all,
>>
>> First of all, sorry for the lengthy post. I decided to add some context to
>> explain things a bit, but it resulted in quite a long e-mail. For the past
>> few weeks I've been trying to come up with a solution for the issue I
>> filled under SUREFIRE-1935, but I'm getting stuck and starting to feel
>> like
>> the issue cannot be solved with the current JUnit Platform API. To give
>> some perspective into why this issue is important for us, I first have to
>> explain a bit about our setup.
>>
>> We write our tests in the Spock framework and use Arquillian to run the
>> tests in the application container. Some of our tests, especially the
>> Selenium based tests, require quite some setup and tear down. Prior to
>> starting the test, Arquillian cube builds and starts several docker
>> containers and the tests are run against these containers. We currently
>> use
>> the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to
>> Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For this,
>> I've already started working on a Spock extension that integrates Spock 2
>> in the Arquillian test life cycle [1]. This extension is inspired by the
>> (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global
>> registration to keep track of the state managed by Arquilllian. This will
>> end up somewhere at the root of the TestPlan (for example see [3]).
>>
>> Because our tests are quite extensive, with a total run time of 6 hours,
>> we
>> run them with a forkCount of 8, greatly reducing the total duration.
>> However, this is where SUREFIRE-1935 comes into play. With a forkCount >
>> 1,
>> the entire test life cycle is started over and over again for every test
>> class. This happens in JUnitPlatformProvider at line 197 [4]. This results
>> in the entire Arquillian suite being torn down and setup for every class,
>> in our case adding several minutes to the execution of every test class
>> because the docker setup is done over and over again.
>>
>> To overcome this issue, the state from one test class execution has to be
>> carried to the next. It seems the LauncherSession (introduced in JUnit
>> 5.8)
>> is meant to close this gap. However, this would mean my extension would
>> also need to implement a LauncherSessionListener, and I'm not sure if
>> extensions are supposed to integrate with the launcher as well. Also, for
>> this surefire would need to start a session prior to the tests, and close
>> it when done. I think this is a good idea anyway when running on platform
>> 1.8 or higher.
>>
>> Another solution could be a (sort of) dynamic test that produces the tests
>> to be run one by one. However, here my knowledge of JUnit really falls
>> short. I've got no idea of this is even possible.
>>
>> I hope someone can help me out on this one and point me in the right
>> direction, as we like to upgrade our test frameworks and this is blocking
>> us at the moment.
>>
>> Best regards,
>> Emond Papegaaij
>>
>> [1] Arquillian extension for Spock Framework 2:
>>
>> https://github.com/topicusonderwijs/arquillian-testrunner-spock/tree/spock-2.0-junit5
>> [2] Arquillian module for JUnit 5:
>> https://github.com/arquillian/arquillian-core/tree/master/junit5
>> [3] Registration in root store:
>>
>> https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/JUnitJupiterTestClassLifecycleManager.java#L20
>> [4] JUnitPlatformProvider launching the tests:
>>
>> https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L197
>>
>

Re: JUnit Platform, forkCount > 1 and the TestPlan

If you use forkCount > 1, the Surefire loads test classes via load balancer.
If you use default forkCount = 0, all the classes are run eagerly as a
suite via JUnit5 Launcher in one shot.

If you are aiming for Arquillian, testing the applications in the
application server, you should use maven-failsafe-plugin which has another
testing model.
See the plugin goals of Failsafe plugin. Maven Failsafe Plugin – Plugin
Documentation (apache.org)

https://maven.apache.org/surefire/maven-failsafe-plugin/plugin-info.html
There are Maven phases:
pre-integration-test
integration-test
post-integration-test
verify

Therefore you should start the application server in the phase
pre-integration-test.
Accordingly, you should stop it in the phase post-integration-test.

Then use Failsafe plugin in the phases integration-test and verify.

Cheers
Tibor



On Fri, Sep 10, 2021 at 12:42 PM Emond Papegaaij 
wrote:

> Hi all,
>
> First of all, sorry for the lengthy post. I decided to add some context to
> explain things a bit, but it resulted in quite a long e-mail. For the past
> few weeks I've been trying to come up with a solution for the issue I
> filled under SUREFIRE-1935, but I'm getting stuck and starting to feel like
> the issue cannot be solved with the current JUnit Platform API. To give
> some perspective into why this issue is important for us, I first have to
> explain a bit about our setup.
>
> We write our tests in the Spock framework and use Arquillian to run the
> tests in the application container. Some of our tests, especially the
> Selenium based tests, require quite some setup and tear down. Prior to
> starting the test, Arquillian cube builds and starts several docker
> containers and the tests are run against these containers. We currently use
> the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to
> Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For this,
> I've already started working on a Spock extension that integrates Spock 2
> in the Arquillian test life cycle [1]. This extension is inspired by the
> (currently Alpha) JUnit5 module for Arquillian [2]. Both use a global
> registration to keep track of the state managed by Arquilllian. This will
> end up somewhere at the root of the TestPlan (for example see [3]).
>
> Because our tests are quite extensive, with a total run time of 6 hours, we
> run them with a forkCount of 8, greatly reducing the total duration.
> However, this is where SUREFIRE-1935 comes into play. With a forkCount > 1,
> the entire test life cycle is started over and over again for every test
> class. This happens in JUnitPlatformProvider at line 197 [4]. This results
> in the entire Arquillian suite being torn down and setup for every class,
> in our case adding several minutes to the execution of every test class
> because the docker setup is done over and over again.
>
> To overcome this issue, the state from one test class execution has to be
> carried to the next. It seems the LauncherSession (introduced in JUnit 5.8)
> is meant to close this gap. However, this would mean my extension would
> also need to implement a LauncherSessionListener, and I'm not sure if
> extensions are supposed to integrate with the launcher as well. Also, for
> this surefire would need to start a session prior to the tests, and close
> it when done. I think this is a good idea anyway when running on platform
> 1.8 or higher.
>
> Another solution could be a (sort of) dynamic test that produces the tests
> to be run one by one. However, here my knowledge of JUnit really falls
> short. I've got no idea of this is even possible.
>
> I hope someone can help me out on this one and point me in the right
> direction, as we like to upgrade our test frameworks and this is blocking
> us at the moment.
>
> Best regards,
> Emond Papegaaij
>
> [1] Arquillian extension for Spock Framework 2:
>
> https://github.com/topicusonderwijs/arquillian-testrunner-spock/tree/spock-2.0-junit5
> [2] Arquillian module for JUnit 5:
> https://github.com/arquillian/arquillian-core/tree/master/junit5
> [3] Registration in root store:
>
> https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/JUnitJupiterTestClassLifecycleManager.java#L20
> [4] JUnitPlatformProvider launching the tests:
>
> https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L197
>

[GitHub] [maven-site] waldyrious commented on pull request #257: run.md: change example to use the "compile" phase



waldyrious commented on pull request #257:
URL: https://github.com/apache/maven-site/pull/257#issuecomment-916928861


   Well, in [Introduction to the Build Lifecycle § Usual Command Line 
Calls](http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html#usual-command-line-calls)
 there's this passage:
   
   > If you are uncertain what you want, the preferred phase to call is
   >
   > `mvn verify`
   
   So perhaps it would be good to use `verify` in this example as well, for 
consistency?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[GitHub] [maven-site] marcphilipp commented on pull request #258: Add Gradle Enterprise extension



marcphilipp commented on pull request #258:
URL: https://github.com/apache/maven-site/pull/258#issuecomment-916914831


   After a discussion on Slack, I created a subsection for commercial 
extensions.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[ANN] Apache Maven PMD Plugin 3.15.0 Released

2021-09-10 Thread Andreas Dangel

The Maven team is pleased to announce the release of the Apache Maven 
PMD Plugin, version 3.15.0


A Maven plugin for the PMD toolkit, that produces a report on both code 
rule violations and detected copy and paste

fragments, as well as being able to fail the build based on these metrics.

https://maven.apache.org/plugins/maven-pmd-plugin/

You should specify the version in your project's plugin configuration:


  org.apache.maven.plugins
  maven-pmd-plugin
  3.15.0



Release Notes - Maven PMD Plugin - Version 3.15.0

Bug
* [MPMD-320] Error when using toolchain and spaces in repository path
* [MPMD-318] Incorrect aux classpath if 'includeTests' set to true
* [MPMD-317] NoClassDefFoundError for provided classes
* [MPMD-315] Maven PMD Plugin fails on Java 16: Unsupported targetJdk 
value '16'.

* [MPMD-314] PMD report extension not set correctly for custom report class

Improvement
* [MPMD-322] Display when PMD/CPD is skipped
* [MPMD-321] Display PMD version that is being used also for pmd:pmd and 
pmd:cpd

* [MPMD-319] Add GitHub Action to confirm build PR
* [MPMD-313] Improve  parameter description
* [MPMD-311] Improve excludeFromFailureFile docs

Task
* [MPMD-316] Require Java 8
* [MPMD-312] Upgrade to PMD 6.38.0
* [MPMD-308] Set Maven 3.1.0 as minimum version
* [MPMD-283] Create a real aggregate goal

Enjoy,

-The Maven team



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

[RESULT] [VOTE] Release Apache Maven PMD Plugin version 3.15.0

2021-09-10 Thread Andreas Dangel


Hi,

The vote has passed with the following result:

+1 : Olivier Lamy, Tamás Cservenák, Robert Scholte, Hervé Boutemy, 
Arnaud Héritier,

Sylwester Lachiewicz, Andreas Dangel

PMC quorum: reached.

I will promote the artifacts to the central repo.

Regards,
Andreas




-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Initialisation of a vote for Maven Shade Plugin version 3.3.0

2021-09-10 Thread Andrew Bryan


+1

A new build would greatly improve my life.



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

JUnit Platform, forkCount > 1 and the TestPlan

Hi all,

First of all, sorry for the lengthy post. I decided to add some context to
explain things a bit, but it resulted in quite a long e-mail. For the past
few weeks I've been trying to come up with a solution for the issue I
filled under SUREFIRE-1935, but I'm getting stuck and starting to feel like
the issue cannot be solved with the current JUnit Platform API. To give
some perspective into why this issue is important for us, I first have to
explain a bit about our setup.

We write our tests in the Spock framework and use Arquillian to run the
tests in the application container. Some of our tests, especially the
Selenium based tests, require quite some setup and tear down. Prior to
starting the test, Arquillian cube builds and starts several docker
containers and the tests are run against these containers. We currently use
the JUnit 4 based Spock 1.3 with Groovy 2.5, but we like to upgrade to
Spock 2 and Groovy 3, which runs on top of the JUnit Platform. For this,
I've already started working on a Spock extension that integrates Spock 2
in the Arquillian test life cycle [1]. This extension is inspired by the
(currently Alpha) JUnit5 module for Arquillian [2]. Both use a global
registration to keep track of the state managed by Arquilllian. This will
end up somewhere at the root of the TestPlan (for example see [3]).

Because our tests are quite extensive, with a total run time of 6 hours, we
run them with a forkCount of 8, greatly reducing the total duration.
However, this is where SUREFIRE-1935 comes into play. With a forkCount > 1,
the entire test life cycle is started over and over again for every test
class. This happens in JUnitPlatformProvider at line 197 [4]. This results
in the entire Arquillian suite being torn down and setup for every class,
in our case adding several minutes to the execution of every test class
because the docker setup is done over and over again.

To overcome this issue, the state from one test class execution has to be
carried to the next. It seems the LauncherSession (introduced in JUnit 5.8)
is meant to close this gap. However, this would mean my extension would
also need to implement a LauncherSessionListener, and I'm not sure if
extensions are supposed to integrate with the launcher as well. Also, for
this surefire would need to start a session prior to the tests, and close
it when done. I think this is a good idea anyway when running on platform
1.8 or higher.

Another solution could be a (sort of) dynamic test that produces the tests
to be run one by one. However, here my knowledge of JUnit really falls
short. I've got no idea of this is even possible.

I hope someone can help me out on this one and point me in the right
direction, as we like to upgrade our test frameworks and this is blocking
us at the moment.

Best regards,
Emond Papegaaij

[1] Arquillian extension for Spock Framework 2:
https://github.com/topicusonderwijs/arquillian-testrunner-spock/tree/spock-2.0-junit5
[2] Arquillian module for JUnit 5:
https://github.com/arquillian/arquillian-core/tree/master/junit5
[3] Registration in root store:
https://github.com/arquillian/arquillian-core/blob/master/junit5/core/src/main/java/org/jboss/arquillian/junit5/JUnitJupiterTestClassLifecycleManager.java#L20
[4] JUnitPlatformProvider launching the tests:
https://github.com/apache/maven-surefire/blob/master/surefire-providers/surefire-junit-platform/src/main/java/org/apache/maven/surefire/junitplatform/JUnitPlatformProvider.java#L197

[GitHub] [maven-site] michael-o commented on pull request #257: run.md: change example to use the "compile" phase