Re: Take threaddump on hung surefire tests

[Debraj Manna]

Thanks Tibor. It is clear now.

On Sat 5 Oct, 2019, 9:36 PM Tibor Digana,  wrote:

> pleaseStop() can be called only inside Surefire.
> So your solution has to be embedded in Surefire and the feature has to
> configured.
> Unfortunately it's by design of JUnit.
> RunNotifier cannot be exposed because it must be only one instance and we
> need to instantiate it and have it under the control in Surefire.
> Therefore this feature has to be internal and implement a hook between
> RunListener and RunNotifier which already exists in Surefire.
> This is the support I was talking about in one of my previous emails.
>
> T
>
> On Sat, Oct 5, 2019 at 5:57 PM Debraj Manna 
> wrote:
>
> > Enrico apache-bookeeper seems to be using timeout in @Test as I can see
> > from their github repo here
> > <
> >
> https://github.com/apache/bookkeeper/search?q=%40Test%28timeout+%3D_q=%40Test%28timeout+%3D
> > >
> > .
> >
> > Tibor how can I pass an instance of RunNotifier to a class extending from
> > RunListener so that I can call pleaseStop() from the listener when the
> > timeout of a test expires? Is there any example code I can refer to?
> >
> > I am using Junit 4.12 and maven-surefire 2.22.2
> >
> > On Sat, Oct 5, 2019 at 6:27 PM Tibor Digana 
> > wrote:
> >
> > > Some users may have a requirement to stop the exertion when a test
> hangs.
> > > Other users woul maybe prefer to interrupt the test and continue with
> > next
> > > test.
> > > This would lead to configuration with more than one config value =>
> POJO
> > in
> > > new config parameter.
> > >
> > > On Sat, Oct 5, 2019 at 2:17 PM Debraj Manna 
> > > wrote:
> > >
> > > > Thanks again Enrico. I will try to find out from apache-bookmark code
> > or
> > > > check in apache-bookkeper mailing list.
> > > >
> > > > Yes Tibor I am looking for a global timeout without explicitly adding
> > the
> > > > timeout annotation in all of my tests / classes. I am not using
> > > forkCount >
> > > > 1. I get your testStarted part but it is still not clear to me about
> > > > calling 'pleaseStop() on the listener'. Are you suggesting to add
> this
> > in
> > > > each of my test classes? Can you explain this a bit more?
> > > >
> > > > On Fri, Oct 4, 2019 at 11:52 PM Tibor Digana  >
> > > > wrote:
> > > >
> > > > > Hi Debraj,
> > > > >
> > > > > It depends on your requirements.
> > > > >
> > > > > As I initially understood your email, every test method wants to
> have
> > > > > implicit timeout value even without the annotation @Timeout.
> > > > > Then the testStarted is important event even your code will see
> when
> > > the
> > > > > test method has started and timeout can be easily computed.
> > > > > Of course you need to use an extra thread which checks the end
> > events.
> > > > >
> > > > >
> > > >
> > >
> >
> https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html#testStarted(org.junit.runner.Description)
> > > > > Your code can call the method "pleaseStop()" on the listener. I
> think
> > > it
> > > > > would not be reliable algorithm in all use cases if you use
> forkCount
> > > > 1
> > > > > and there you will need to have our support.
> > > > >
> > > > > Is it this what you need, the global timeout with the same value on
> > > every
> > > > > test method?
> > > > >
> > > > > Cheers
> > > > > Tibor
> > > > >
> > > > > On Fri, Oct 4, 2019 at 4:30 PM Debraj Manna <
> > subharaj.ma...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Enrico
> > > > > >
> > > > > > If I get the approach correctly then all my junit4 tests should
> > have
> > > > > > timeout specified (either via @Test or via @Rule) then only I can
> > use
> > > > the
> > > > > > listener. But the problem is we are having more than 2000 tests
> and
> > > > > > specifying a timeout in each of the tests/classes is cumbersome.
> > > > > >
> > > > > > Correct me if I have misunderstood anything.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Fri, Oct 4, 2019 at 3:18 PM Debraj Manna <
> > > subharaj.ma...@gmail.com>
> > > > > > wrote:
> > > > > >
> > > > > > > Yeah sure ... thanks.
> > > > > > >
> > > > > > > On Thu, Oct 3, 2019 at 7:50 PM Tibor Digana <
> > > tibordig...@apache.org>
> > > > > > > wrote:
> > > > > > >
> > > > > > >> Hi Debraj,
> > > > > > >>
> > > > > > >> There is nice technical idea from Enrico.
> > > > > > >> If you apply it and you are convinced that it would work
> > properly
> > > > for
> > > > > > all
> > > > > > >> the Java community, feel free to show it and we can discuss it
> > on
> > > > how
> > > > > we
> > > > > > >> would adopt your solution in Surefire project.
> > > > > > >>
> > > > > > >> Cheers
> > > > > > >> Tibor17
> > > > > > >>
> > > > > > >> On Thu, Oct 3, 2019 at 2:49 PM Debraj Manna <
> > > > subharaj.ma...@gmail.com
> > > > > >
> > > > > > >> wrote:
> > > > > > >>
> > > > > > >> > Sometimes I have maven surefire tests that get hung, due to
> > > either
> > > > > > >> races or
> > > > > > >> > deadlocks.
> > > > > > >> >
> > > > > > >> > When 

Re: Take threaddump on hung surefire tests

[Tibor Digana]

pleaseStop() can be called only inside Surefire.
So your solution has to be embedded in Surefire and the feature has to
configured.
Unfortunately it's by design of JUnit.
RunNotifier cannot be exposed because it must be only one instance and we
need to instantiate it and have it under the control in Surefire.
Therefore this feature has to be internal and implement a hook between
RunListener and RunNotifier which already exists in Surefire.
This is the support I was talking about in one of my previous emails.

T

On Sat, Oct 5, 2019 at 5:57 PM Debraj Manna 
wrote:

> Enrico apache-bookeeper seems to be using timeout in @Test as I can see
> from their github repo here
> <
> https://github.com/apache/bookkeeper/search?q=%40Test%28timeout+%3D_q=%40Test%28timeout+%3D
> >
> .
>
> Tibor how can I pass an instance of RunNotifier to a class extending from
> RunListener so that I can call pleaseStop() from the listener when the
> timeout of a test expires? Is there any example code I can refer to?
>
> I am using Junit 4.12 and maven-surefire 2.22.2
>
> On Sat, Oct 5, 2019 at 6:27 PM Tibor Digana 
> wrote:
>
> > Some users may have a requirement to stop the exertion when a test hangs.
> > Other users woul maybe prefer to interrupt the test and continue with
> next
> > test.
> > This would lead to configuration with more than one config value => POJO
> in
> > new config parameter.
> >
> > On Sat, Oct 5, 2019 at 2:17 PM Debraj Manna 
> > wrote:
> >
> > > Thanks again Enrico. I will try to find out from apache-bookmark code
> or
> > > check in apache-bookkeper mailing list.
> > >
> > > Yes Tibor I am looking for a global timeout without explicitly adding
> the
> > > timeout annotation in all of my tests / classes. I am not using
> > forkCount >
> > > 1. I get your testStarted part but it is still not clear to me about
> > > calling 'pleaseStop() on the listener'. Are you suggesting to add this
> in
> > > each of my test classes? Can you explain this a bit more?
> > >
> > > On Fri, Oct 4, 2019 at 11:52 PM Tibor Digana 
> > > wrote:
> > >
> > > > Hi Debraj,
> > > >
> > > > It depends on your requirements.
> > > >
> > > > As I initially understood your email, every test method wants to have
> > > > implicit timeout value even without the annotation @Timeout.
> > > > Then the testStarted is important event even your code will see when
> > the
> > > > test method has started and timeout can be easily computed.
> > > > Of course you need to use an extra thread which checks the end
> events.
> > > >
> > > >
> > >
> >
> https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html#testStarted(org.junit.runner.Description)
> > > > Your code can call the method "pleaseStop()" on the listener. I think
> > it
> > > > would not be reliable algorithm in all use cases if you use forkCount
> > > 1
> > > > and there you will need to have our support.
> > > >
> > > > Is it this what you need, the global timeout with the same value on
> > every
> > > > test method?
> > > >
> > > > Cheers
> > > > Tibor
> > > >
> > > > On Fri, Oct 4, 2019 at 4:30 PM Debraj Manna <
> subharaj.ma...@gmail.com>
> > > > wrote:
> > > >
> > > > > Enrico
> > > > >
> > > > > If I get the approach correctly then all my junit4 tests should
> have
> > > > > timeout specified (either via @Test or via @Rule) then only I can
> use
> > > the
> > > > > listener. But the problem is we are having more than 2000 tests and
> > > > > specifying a timeout in each of the tests/classes is cumbersome.
> > > > >
> > > > > Correct me if I have misunderstood anything.
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Oct 4, 2019 at 3:18 PM Debraj Manna <
> > subharaj.ma...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Yeah sure ... thanks.
> > > > > >
> > > > > > On Thu, Oct 3, 2019 at 7:50 PM Tibor Digana <
> > tibordig...@apache.org>
> > > > > > wrote:
> > > > > >
> > > > > >> Hi Debraj,
> > > > > >>
> > > > > >> There is nice technical idea from Enrico.
> > > > > >> If you apply it and you are convinced that it would work
> properly
> > > for
> > > > > all
> > > > > >> the Java community, feel free to show it and we can discuss it
> on
> > > how
> > > > we
> > > > > >> would adopt your solution in Surefire project.
> > > > > >>
> > > > > >> Cheers
> > > > > >> Tibor17
> > > > > >>
> > > > > >> On Thu, Oct 3, 2019 at 2:49 PM Debraj Manna <
> > > subharaj.ma...@gmail.com
> > > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > Sometimes I have maven surefire tests that get hung, due to
> > either
> > > > > >> races or
> > > > > >> > deadlocks.
> > > > > >> >
> > > > > >> > When this happens I have to discover what slave is being used,
> > and
> > > > > then
> > > > > >> I
> > > > > >> > have to log on that slave, sudo to jenkins account and execute
> > > > either
> > > > > >> > jstack or kill -3
> > > > > >> >
> > > > > >> > I am looking for a simple solution like doing jstack / kill -3
> > > when
> > > > > >> someone
> > > > > >> > presses abort button on 

Re: Take threaddump on hung surefire tests

[Debraj Manna]

Enrico apache-bookeeper seems to be using timeout in @Test as I can see
from their github repo here

.

Tibor how can I pass an instance of RunNotifier to a class extending from
RunListener so that I can call pleaseStop() from the listener when the
timeout of a test expires? Is there any example code I can refer to?

I am using Junit 4.12 and maven-surefire 2.22.2

On Sat, Oct 5, 2019 at 6:27 PM Tibor Digana  wrote:

> Some users may have a requirement to stop the exertion when a test hangs.
> Other users woul maybe prefer to interrupt the test and continue with next
> test.
> This would lead to configuration with more than one config value => POJO in
> new config parameter.
>
> On Sat, Oct 5, 2019 at 2:17 PM Debraj Manna 
> wrote:
>
> > Thanks again Enrico. I will try to find out from apache-bookmark code or
> > check in apache-bookkeper mailing list.
> >
> > Yes Tibor I am looking for a global timeout without explicitly adding the
> > timeout annotation in all of my tests / classes. I am not using
> forkCount >
> > 1. I get your testStarted part but it is still not clear to me about
> > calling 'pleaseStop() on the listener'. Are you suggesting to add this in
> > each of my test classes? Can you explain this a bit more?
> >
> > On Fri, Oct 4, 2019 at 11:52 PM Tibor Digana 
> > wrote:
> >
> > > Hi Debraj,
> > >
> > > It depends on your requirements.
> > >
> > > As I initially understood your email, every test method wants to have
> > > implicit timeout value even without the annotation @Timeout.
> > > Then the testStarted is important event even your code will see when
> the
> > > test method has started and timeout can be easily computed.
> > > Of course you need to use an extra thread which checks the end events.
> > >
> > >
> >
> https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html#testStarted(org.junit.runner.Description)
> > > Your code can call the method "pleaseStop()" on the listener. I think
> it
> > > would not be reliable algorithm in all use cases if you use forkCount
> > 1
> > > and there you will need to have our support.
> > >
> > > Is it this what you need, the global timeout with the same value on
> every
> > > test method?
> > >
> > > Cheers
> > > Tibor
> > >
> > > On Fri, Oct 4, 2019 at 4:30 PM Debraj Manna 
> > > wrote:
> > >
> > > > Enrico
> > > >
> > > > If I get the approach correctly then all my junit4 tests should have
> > > > timeout specified (either via @Test or via @Rule) then only I can use
> > the
> > > > listener. But the problem is we are having more than 2000 tests and
> > > > specifying a timeout in each of the tests/classes is cumbersome.
> > > >
> > > > Correct me if I have misunderstood anything.
> > > >
> > > >
> > > >
> > > > On Fri, Oct 4, 2019 at 3:18 PM Debraj Manna <
> subharaj.ma...@gmail.com>
> > > > wrote:
> > > >
> > > > > Yeah sure ... thanks.
> > > > >
> > > > > On Thu, Oct 3, 2019 at 7:50 PM Tibor Digana <
> tibordig...@apache.org>
> > > > > wrote:
> > > > >
> > > > >> Hi Debraj,
> > > > >>
> > > > >> There is nice technical idea from Enrico.
> > > > >> If you apply it and you are convinced that it would work properly
> > for
> > > > all
> > > > >> the Java community, feel free to show it and we can discuss it on
> > how
> > > we
> > > > >> would adopt your solution in Surefire project.
> > > > >>
> > > > >> Cheers
> > > > >> Tibor17
> > > > >>
> > > > >> On Thu, Oct 3, 2019 at 2:49 PM Debraj Manna <
> > subharaj.ma...@gmail.com
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Sometimes I have maven surefire tests that get hung, due to
> either
> > > > >> races or
> > > > >> > deadlocks.
> > > > >> >
> > > > >> > When this happens I have to discover what slave is being used,
> and
> > > > then
> > > > >> I
> > > > >> > have to log on that slave, sudo to jenkins account and execute
> > > either
> > > > >> > jstack or kill -3
> > > > >> >
> > > > >> > I am looking for a simple solution like doing jstack / kill -3
> > when
> > > > >> someone
> > > > >> > presses abort button on the jenkins.
> > > > >> >
> > > > >> > Can someone suggest how can I automate this or some better way
> of
> > > > >> handling
> > > > >> > this?
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: Proposal: maven release lifecycle

[Tibor Digana]

Release lifecycle does not exist yet.
There's only "default", "clean", "site".

Therefore i would prefer to have this in CLI, i.e.
$ maven release

>> nowdays needs (release, js build, cloud deployment, gh-pages, etc...)
We cannot be technology specific but we can provide customization of the
release lifecycle.

The outcome of this discussion should be some frame-work as well as list of
items where we can agree and start writing Wiki page.
We do not have to go deeply "how" to do.
We should agree on "what" to do at least.



On Sat, Oct 5, 2019 at 4:08 PM Romain Manni-Bucau 
wrote:

> Personally Id like to keep current release lifecycle, it is the most
> generic one we can get but we need to let users create their own chain of
> mojo very easily in the pom.
>
> It is trivial to do with a custom descriptor and extension but it must be
> built in cause of nowdays needs (release, js build, cloud deployment,
> gh-pages, etc...)
>
> Le sam. 5 oct. 2019 à 15:18, Tibor Digana  a
> écrit :
>
> > I have never seen a documentation for ship-maven-plugin.
> > It could be a good motivation for us.
> >
> > I can say for myself, my requirements are to the new lifecycle and plugin
> > goal release:release:
> > 1. executes phases from clean to deploy
> > 2. installation of "release" artifacts is skipped (snapshots are not
> > skipped)
> > 3. unit and integration tests are executed only once and not two times
> >
> > Tagging and SCM checkout are just fine.
> >
> > Second lifecycle for release without scm checkout and deployment. The
> test
> > would be executed once, tagging and both commit messages are the must.
> >
> > WDYT? How you would like to see the release lifecysle?
> > Notice that we should not say how the project should look like in the
> > customer. The customer may have reasons to use his project structures as
> > they are and the architecture too. So we should NOT define these
> > structures!
> >
> >
> > On Sat, Oct 5, 2019 at 2:47 PM Marco Schulz 
> > wrote:
> >
> > > All this points are in my opinion an indicator that a release is a
> > complex
> > > process and very difficult to handle in just a plugin. To maintain all
> > that
> > > different points a relesaplugin will ver a ueber (god) plugin.
> > >
> > > Docker for example is another big toic we need to think about.
> > >
> > > rwgards and anice weekend to all
> > > . marco
> > >
> > > Sent from Outlook Mobile
> > >
> > > 
> > > From: Romain Manni-Bucau 
> > > Sent: Saturday, October 5, 2019 2:06:27 AM
> > > To: Maven Developers List 
> > > Cc: users 
> > > Subject: Re: Proposal: maven release lifecycle
> > >
> > > I like the words but fail to see the missing pieces (understand actions
> > to
> > > do).
> > >
> > > Typically today when i release at work i use release plugin enriched
> with
> > > github page deployment for the doc using antora + a ftp update of my
> > server
> > > output capture to have a mock backing openapi/swagger ui + docker ilage
> > > deployment on docker hub + an intellij plugin deployment on jetbrains
> > > marketplace etc...adding a flyway migration with a rolling update in a
> > k8s
> > > cluster is trivial (ops need to say yes though ;)).
> > >
> > > So technically it is verbose but doable.
> > >
> > > Custom lifecycle definition is not neat, custom build tasks require a
> > build
> > > hack or using groovy plugin but it works.
> > >
> > > Typically, an extension could enable all that based on convention, just
> > > injecting needed plugins based on the file presence and values of the
> > > distribution urls.
> > >
> > > This is why i ended up thinking we are more on 1. Sugar in release
> plugin
> > > and 2. Maybe on a generic extension cretion (it replaces rigid
> lifecycle
> > > for such cases these days due to their writing easiness)
> > >
> > > Am I missing something?
> > >
> > > Le sam. 5 oct. 2019 à 08:23, Marco Schulz  a
> > > écrit :
> > >
> > > > Hello Romain, hello Tibor
> > > >
> > > > Thanks for your feedback. I had yesterday a very interesting
> > conversation
> > > > with
> > > > Karl Heinz. He gave me some very informative links about deeply maven
> > > > insights.
> > > > Before I saw his talk on youtube I thought I have a good knowledge
> > about
> > > > maven
> > > > ;-)  now I was lerning a lot of new things.
> > > >
> > > > He defined Maven as an plugin execution framework. I like to extend
> > this
> > > > definition: Maven is a process engine framework and define industrial
> > > > defacto
> > > > standards for the software development process. And with this idea in
> > > mind
> > > > I
> > > > think it could be great to define more workflows in a equal manner
> like
> > > the
> > > > build lifecycle. The basic proposal is a first draft and I know some
> > > > points are
> > > > missing. I tried to keep the release process as simple as possible.
> Two
> > > > positions in this definition was impotent for me.
> > > >
> > > > Often companies don't really 

Re: Proposal: maven release lifecycle

[Romain Manni-Bucau]

Personally Id like to keep current release lifecycle, it is the most
generic one we can get but we need to let users create their own chain of
mojo very easily in the pom.

It is trivial to do with a custom descriptor and extension but it must be
built in cause of nowdays needs (release, js build, cloud deployment,
gh-pages, etc...)

Le sam. 5 oct. 2019 à 15:18, Tibor Digana  a écrit :

> I have never seen a documentation for ship-maven-plugin.
> It could be a good motivation for us.
>
> I can say for myself, my requirements are to the new lifecycle and plugin
> goal release:release:
> 1. executes phases from clean to deploy
> 2. installation of "release" artifacts is skipped (snapshots are not
> skipped)
> 3. unit and integration tests are executed only once and not two times
>
> Tagging and SCM checkout are just fine.
>
> Second lifecycle for release without scm checkout and deployment. The test
> would be executed once, tagging and both commit messages are the must.
>
> WDYT? How you would like to see the release lifecysle?
> Notice that we should not say how the project should look like in the
> customer. The customer may have reasons to use his project structures as
> they are and the architecture too. So we should NOT define these
> structures!
>
>
> On Sat, Oct 5, 2019 at 2:47 PM Marco Schulz 
> wrote:
>
> > All this points are in my opinion an indicator that a release is a
> complex
> > process and very difficult to handle in just a plugin. To maintain all
> that
> > different points a relesaplugin will ver a ueber (god) plugin.
> >
> > Docker for example is another big toic we need to think about.
> >
> > rwgards and anice weekend to all
> > . marco
> >
> > Sent from Outlook Mobile
> >
> > 
> > From: Romain Manni-Bucau 
> > Sent: Saturday, October 5, 2019 2:06:27 AM
> > To: Maven Developers List 
> > Cc: users 
> > Subject: Re: Proposal: maven release lifecycle
> >
> > I like the words but fail to see the missing pieces (understand actions
> to
> > do).
> >
> > Typically today when i release at work i use release plugin enriched with
> > github page deployment for the doc using antora + a ftp update of my
> server
> > output capture to have a mock backing openapi/swagger ui + docker ilage
> > deployment on docker hub + an intellij plugin deployment on jetbrains
> > marketplace etc...adding a flyway migration with a rolling update in a
> k8s
> > cluster is trivial (ops need to say yes though ;)).
> >
> > So technically it is verbose but doable.
> >
> > Custom lifecycle definition is not neat, custom build tasks require a
> build
> > hack or using groovy plugin but it works.
> >
> > Typically, an extension could enable all that based on convention, just
> > injecting needed plugins based on the file presence and values of the
> > distribution urls.
> >
> > This is why i ended up thinking we are more on 1. Sugar in release plugin
> > and 2. Maybe on a generic extension cretion (it replaces rigid lifecycle
> > for such cases these days due to their writing easiness)
> >
> > Am I missing something?
> >
> > Le sam. 5 oct. 2019 à 08:23, Marco Schulz  a
> > écrit :
> >
> > > Hello Romain, hello Tibor
> > >
> > > Thanks for your feedback. I had yesterday a very interesting
> conversation
> > > with
> > > Karl Heinz. He gave me some very informative links about deeply maven
> > > insights.
> > > Before I saw his talk on youtube I thought I have a good knowledge
> about
> > > maven
> > > ;-)  now I was lerning a lot of new things.
> > >
> > > He defined Maven as an plugin execution framework. I like to extend
> this
> > > definition: Maven is a process engine framework and define industrial
> > > defacto
> > > standards for the software development process. And with this idea in
> > mind
> > > I
> > > think it could be great to define more workflows in a equal manner like
> > the
> > > build lifecycle. The basic proposal is a first draft and I know some
> > > points are
> > > missing. I tried to keep the release process as simple as possible. Two
> > > positions in this definition was impotent for me.
> > >
> > > Often companies don't really have well structured release processes.
> For
> > > that
> > > reason I would be great to have a workable standard. I have a small
> Java
> > > project
> > > on GitHub, an sometimes I also deploy to maven central. If you do this
> > the
> > > first
> > > time a bit complicated process. And publishing on maven central is
> also a
> > > release process. So I had the intention to define some ordered steps
> like
> > > in the
> > > build lifecycle to prepare and publish a release.
> > >
> > > Let me describe an example scenario for a release: After mvn build was
> > > successful executed all unit tests are passed, the sprint is finished
> and
> > > as
> > > build manager we want to release our artifact. Very important would be
> an
> > > option
> > > to take the results from the build lifecycle and prepare the release.

Re: Proposal: maven release lifecycle

[Tibor Digana]

I have never seen a documentation for ship-maven-plugin.
It could be a good motivation for us.

I can say for myself, my requirements are to the new lifecycle and plugin
goal release:release:
1. executes phases from clean to deploy
2. installation of "release" artifacts is skipped (snapshots are not
skipped)
3. unit and integration tests are executed only once and not two times

Tagging and SCM checkout are just fine.

Second lifecycle for release without scm checkout and deployment. The test
would be executed once, tagging and both commit messages are the must.

WDYT? How you would like to see the release lifecysle?
Notice that we should not say how the project should look like in the
customer. The customer may have reasons to use his project structures as
they are and the architecture too. So we should NOT define these structures!


On Sat, Oct 5, 2019 at 2:47 PM Marco Schulz 
wrote:

> All this points are in my opinion an indicator that a release is a complex
> process and very difficult to handle in just a plugin. To maintain all that
> different points a relesaplugin will ver a ueber (god) plugin.
>
> Docker for example is another big toic we need to think about.
>
> rwgards and anice weekend to all
> . marco
>
> Sent from Outlook Mobile
>
> 
> From: Romain Manni-Bucau 
> Sent: Saturday, October 5, 2019 2:06:27 AM
> To: Maven Developers List 
> Cc: users 
> Subject: Re: Proposal: maven release lifecycle
>
> I like the words but fail to see the missing pieces (understand actions to
> do).
>
> Typically today when i release at work i use release plugin enriched with
> github page deployment for the doc using antora + a ftp update of my server
> output capture to have a mock backing openapi/swagger ui + docker ilage
> deployment on docker hub + an intellij plugin deployment on jetbrains
> marketplace etc...adding a flyway migration with a rolling update in a k8s
> cluster is trivial (ops need to say yes though ;)).
>
> So technically it is verbose but doable.
>
> Custom lifecycle definition is not neat, custom build tasks require a build
> hack or using groovy plugin but it works.
>
> Typically, an extension could enable all that based on convention, just
> injecting needed plugins based on the file presence and values of the
> distribution urls.
>
> This is why i ended up thinking we are more on 1. Sugar in release plugin
> and 2. Maybe on a generic extension cretion (it replaces rigid lifecycle
> for such cases these days due to their writing easiness)
>
> Am I missing something?
>
> Le sam. 5 oct. 2019 à 08:23, Marco Schulz  a
> écrit :
>
> > Hello Romain, hello Tibor
> >
> > Thanks for your feedback. I had yesterday a very interesting conversation
> > with
> > Karl Heinz. He gave me some very informative links about deeply maven
> > insights.
> > Before I saw his talk on youtube I thought I have a good knowledge about
> > maven
> > ;-)  now I was lerning a lot of new things.
> >
> > He defined Maven as an plugin execution framework. I like to extend this
> > definition: Maven is a process engine framework and define industrial
> > defacto
> > standards for the software development process. And with this idea in
> mind
> > I
> > think it could be great to define more workflows in a equal manner like
> the
> > build lifecycle. The basic proposal is a first draft and I know some
> > points are
> > missing. I tried to keep the release process as simple as possible. Two
> > positions in this definition was impotent for me.
> >
> > Often companies don't really have well structured release processes. For
> > that
> > reason I would be great to have a workable standard. I have a small Java
> > project
> > on GitHub, an sometimes I also deploy to maven central. If you do this
> the
> > first
> > time a bit complicated process. And publishing on maven central is also a
> > release process. So I had the intention to define some ordered steps like
> > in the
> > build lifecycle to prepare and publish a release.
> >
> > Let me describe an example scenario for a release: After mvn build was
> > successful executed all unit tests are passed, the sprint is finished and
> > as
> > build manager we want to release our artifact. Very important would be an
> > option
> > to take the results from the build lifecycle and prepare the release. As
> > first
> > we need to see that the planned artifact version is not exist in any
> > configured
> > public repository and the pom contains all mandatory information for
> > publishing
> > on maven central(validate). To secure a reproducible build, in a second
> > step we
> > enforce that no SNAPSHOT artifacts are involved, the correct maven
> version
> > is
> > used etc. (the existing enforcer plugin do a great job). After the
> > preconditions
> > are fine we can execute external acceptance tests like JGiven. In another
> > step
> > the JavaDoc got generated. the pgp plugin sing the artifacts and check
> > that the
> > 

Re: Take threaddump on hung surefire tests

[Tibor Digana]

Some users may have a requirement to stop the exertion when a test hangs.
Other users woul maybe prefer to interrupt the test and continue with next
test.
This would lead to configuration with more than one config value => POJO in
new config parameter.

On Sat, Oct 5, 2019 at 2:17 PM Debraj Manna 
wrote:

> Thanks again Enrico. I will try to find out from apache-bookmark code or
> check in apache-bookkeper mailing list.
>
> Yes Tibor I am looking for a global timeout without explicitly adding the
> timeout annotation in all of my tests / classes. I am not using forkCount >
> 1. I get your testStarted part but it is still not clear to me about
> calling 'pleaseStop() on the listener'. Are you suggesting to add this in
> each of my test classes? Can you explain this a bit more?
>
> On Fri, Oct 4, 2019 at 11:52 PM Tibor Digana 
> wrote:
>
> > Hi Debraj,
> >
> > It depends on your requirements.
> >
> > As I initially understood your email, every test method wants to have
> > implicit timeout value even without the annotation @Timeout.
> > Then the testStarted is important event even your code will see when the
> > test method has started and timeout can be easily computed.
> > Of course you need to use an extra thread which checks the end events.
> >
> >
> https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html#testStarted(org.junit.runner.Description)
> > Your code can call the method "pleaseStop()" on the listener. I think it
> > would not be reliable algorithm in all use cases if you use forkCount > 1
> > and there you will need to have our support.
> >
> > Is it this what you need, the global timeout with the same value on every
> > test method?
> >
> > Cheers
> > Tibor
> >
> > On Fri, Oct 4, 2019 at 4:30 PM Debraj Manna 
> > wrote:
> >
> > > Enrico
> > >
> > > If I get the approach correctly then all my junit4 tests should have
> > > timeout specified (either via @Test or via @Rule) then only I can use
> the
> > > listener. But the problem is we are having more than 2000 tests and
> > > specifying a timeout in each of the tests/classes is cumbersome.
> > >
> > > Correct me if I have misunderstood anything.
> > >
> > >
> > >
> > > On Fri, Oct 4, 2019 at 3:18 PM Debraj Manna 
> > > wrote:
> > >
> > > > Yeah sure ... thanks.
> > > >
> > > > On Thu, Oct 3, 2019 at 7:50 PM Tibor Digana 
> > > > wrote:
> > > >
> > > >> Hi Debraj,
> > > >>
> > > >> There is nice technical idea from Enrico.
> > > >> If you apply it and you are convinced that it would work properly
> for
> > > all
> > > >> the Java community, feel free to show it and we can discuss it on
> how
> > we
> > > >> would adopt your solution in Surefire project.
> > > >>
> > > >> Cheers
> > > >> Tibor17
> > > >>
> > > >> On Thu, Oct 3, 2019 at 2:49 PM Debraj Manna <
> subharaj.ma...@gmail.com
> > >
> > > >> wrote:
> > > >>
> > > >> > Sometimes I have maven surefire tests that get hung, due to either
> > > >> races or
> > > >> > deadlocks.
> > > >> >
> > > >> > When this happens I have to discover what slave is being used, and
> > > then
> > > >> I
> > > >> > have to log on that slave, sudo to jenkins account and execute
> > either
> > > >> > jstack or kill -3
> > > >> >
> > > >> > I am looking for a simple solution like doing jstack / kill -3
> when
> > > >> someone
> > > >> > presses abort button on the jenkins.
> > > >> >
> > > >> > Can someone suggest how can I automate this or some better way of
> > > >> handling
> > > >> > this?
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: Take threaddump on hung surefire tests

[Tibor Digana]

Notice that Surefire instantiates new JUnitCore() and then runs the test
class(es).
The method pleaseStop() stops this execution so that no new test method
would be followed.

On Sat, Oct 5, 2019 at 2:17 PM Debraj Manna 
wrote:

> Thanks again Enrico. I will try to find out from apache-bookmark code or
> check in apache-bookkeper mailing list.
>
> Yes Tibor I am looking for a global timeout without explicitly adding the
> timeout annotation in all of my tests / classes. I am not using forkCount >
> 1. I get your testStarted part but it is still not clear to me about
> calling 'pleaseStop() on the listener'. Are you suggesting to add this in
> each of my test classes? Can you explain this a bit more?
>
> On Fri, Oct 4, 2019 at 11:52 PM Tibor Digana 
> wrote:
>
> > Hi Debraj,
> >
> > It depends on your requirements.
> >
> > As I initially understood your email, every test method wants to have
> > implicit timeout value even without the annotation @Timeout.
> > Then the testStarted is important event even your code will see when the
> > test method has started and timeout can be easily computed.
> > Of course you need to use an extra thread which checks the end events.
> >
> >
> https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html#testStarted(org.junit.runner.Description)
> > Your code can call the method "pleaseStop()" on the listener. I think it
> > would not be reliable algorithm in all use cases if you use forkCount > 1
> > and there you will need to have our support.
> >
> > Is it this what you need, the global timeout with the same value on every
> > test method?
> >
> > Cheers
> > Tibor
> >
> > On Fri, Oct 4, 2019 at 4:30 PM Debraj Manna 
> > wrote:
> >
> > > Enrico
> > >
> > > If I get the approach correctly then all my junit4 tests should have
> > > timeout specified (either via @Test or via @Rule) then only I can use
> the
> > > listener. But the problem is we are having more than 2000 tests and
> > > specifying a timeout in each of the tests/classes is cumbersome.
> > >
> > > Correct me if I have misunderstood anything.
> > >
> > >
> > >
> > > On Fri, Oct 4, 2019 at 3:18 PM Debraj Manna 
> > > wrote:
> > >
> > > > Yeah sure ... thanks.
> > > >
> > > > On Thu, Oct 3, 2019 at 7:50 PM Tibor Digana 
> > > > wrote:
> > > >
> > > >> Hi Debraj,
> > > >>
> > > >> There is nice technical idea from Enrico.
> > > >> If you apply it and you are convinced that it would work properly
> for
> > > all
> > > >> the Java community, feel free to show it and we can discuss it on
> how
> > we
> > > >> would adopt your solution in Surefire project.
> > > >>
> > > >> Cheers
> > > >> Tibor17
> > > >>
> > > >> On Thu, Oct 3, 2019 at 2:49 PM Debraj Manna <
> subharaj.ma...@gmail.com
> > >
> > > >> wrote:
> > > >>
> > > >> > Sometimes I have maven surefire tests that get hung, due to either
> > > >> races or
> > > >> > deadlocks.
> > > >> >
> > > >> > When this happens I have to discover what slave is being used, and
> > > then
> > > >> I
> > > >> > have to log on that slave, sudo to jenkins account and execute
> > either
> > > >> > jstack or kill -3
> > > >> >
> > > >> > I am looking for a simple solution like doing jstack / kill -3
> when
> > > >> someone
> > > >> > presses abort button on the jenkins.
> > > >> >
> > > >> > Can someone suggest how can I automate this or some better way of
> > > >> handling
> > > >> > this?
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: Proposal: maven release lifecycle

[Marco Schulz]

All this points are in my opinion an indicator that a release is a complex 
process and very difficult to handle in just a plugin. To maintain all that 
different points a relesaplugin will ver a ueber (god) plugin.

Docker for example is another big toic we need to think about.

rwgards and anice weekend to all
. marco

Sent from Outlook Mobile


From: Romain Manni-Bucau 
Sent: Saturday, October 5, 2019 2:06:27 AM
To: Maven Developers List 
Cc: users 
Subject: Re: Proposal: maven release lifecycle

I like the words but fail to see the missing pieces (understand actions to
do).

Typically today when i release at work i use release plugin enriched with
github page deployment for the doc using antora + a ftp update of my server
output capture to have a mock backing openapi/swagger ui + docker ilage
deployment on docker hub + an intellij plugin deployment on jetbrains
marketplace etc...adding a flyway migration with a rolling update in a k8s
cluster is trivial (ops need to say yes though ;)).

So technically it is verbose but doable.

Custom lifecycle definition is not neat, custom build tasks require a build
hack or using groovy plugin but it works.

Typically, an extension could enable all that based on convention, just
injecting needed plugins based on the file presence and values of the
distribution urls.

This is why i ended up thinking we are more on 1. Sugar in release plugin
and 2. Maybe on a generic extension cretion (it replaces rigid lifecycle
for such cases these days due to their writing easiness)

Am I missing something?

Le sam. 5 oct. 2019 à 08:23, Marco Schulz  a
écrit :

> Hello Romain, hello Tibor
>
> Thanks for your feedback. I had yesterday a very interesting conversation
> with
> Karl Heinz. He gave me some very informative links about deeply maven
> insights.
> Before I saw his talk on youtube I thought I have a good knowledge about
> maven
> ;-)  now I was lerning a lot of new things.
>
> He defined Maven as an plugin execution framework. I like to extend this
> definition: Maven is a process engine framework and define industrial
> defacto
> standards for the software development process. And with this idea in mind
> I
> think it could be great to define more workflows in a equal manner like the
> build lifecycle. The basic proposal is a first draft and I know some
> points are
> missing. I tried to keep the release process as simple as possible. Two
> positions in this definition was impotent for me.
>
> Often companies don't really have well structured release processes. For
> that
> reason I would be great to have a workable standard. I have a small Java
> project
> on GitHub, an sometimes I also deploy to maven central. If you do this the
> first
> time a bit complicated process. And publishing on maven central is also a
> release process. So I had the intention to define some ordered steps like
> in the
> build lifecycle to prepare and publish a release.
>
> Let me describe an example scenario for a release: After mvn build was
> successful executed all unit tests are passed, the sprint is finished and
> as
> build manager we want to release our artifact. Very important would be an
> option
> to take the results from the build lifecycle and prepare the release. As
> first
> we need to see that the planned artifact version is not exist in any
> configured
> public repository and the pom contains all mandatory information for
> publishing
> on maven central(validate). To secure a reproducible build, in a second
> step we
> enforce that no SNAPSHOT artifacts are involved, the correct maven version
> is
> used etc. (the existing enforcer plugin do a great job). After the
> preconditions
> are fine we can execute external acceptance tests like JGiven. In another
> step
> the JavaDoc got generated. the pgp plugin sing the artifacts and check
> that the
> public key is available. The SCM plugin create a tag for the released
> revision.
> Difficult parts are auto increment the version number and auto check the
> scm if
> the release is produced with the last revision of the code. This actions
> are by
> my experience a bit error prune.
> A release process could have some vocabulary like prepare, perform,
> deploy. This
> are just conventions, like it is used in the build lifecycle. To realize
> this
> idea, many already existing plugins can used, like the release plugin. In
> this
> proposal I was also not mentions external plugins to use like flyway, for
> database versioning and so on. A lot of things ar imaginable.
>
> In future more lifecycles or workflows could be possible. For example a
> deploy
> lifecycle and so on. And then maven transform from a plugin execution
> framework
> to a process management framework, like Jason van Zyl described in his book
> better build with maven.
>
> warm regards to all
> .marco
>
>
>
>
> On Fri, 2019-10-04 at 11:28 +0200, Romain Manni-Bucau wrote:
> > @Tibor: I agree merging both in one 

Re: Take threaddump on hung surefire tests

[Debraj Manna]

Thanks again Enrico. I will try to find out from apache-bookmark code or
check in apache-bookkeper mailing list.

Yes Tibor I am looking for a global timeout without explicitly adding the
timeout annotation in all of my tests / classes. I am not using forkCount >
1. I get your testStarted part but it is still not clear to me about
calling 'pleaseStop() on the listener'. Are you suggesting to add this in
each of my test classes? Can you explain this a bit more?

On Fri, Oct 4, 2019 at 11:52 PM Tibor Digana  wrote:

> Hi Debraj,
>
> It depends on your requirements.
>
> As I initially understood your email, every test method wants to have
> implicit timeout value even without the annotation @Timeout.
> Then the testStarted is important event even your code will see when the
> test method has started and timeout can be easily computed.
> Of course you need to use an extra thread which checks the end events.
>
> https://junit.org/junit4/javadoc/4.12/org/junit/runner/notification/RunListener.html#testStarted(org.junit.runner.Description)
> Your code can call the method "pleaseStop()" on the listener. I think it
> would not be reliable algorithm in all use cases if you use forkCount > 1
> and there you will need to have our support.
>
> Is it this what you need, the global timeout with the same value on every
> test method?
>
> Cheers
> Tibor
>
> On Fri, Oct 4, 2019 at 4:30 PM Debraj Manna 
> wrote:
>
> > Enrico
> >
> > If I get the approach correctly then all my junit4 tests should have
> > timeout specified (either via @Test or via @Rule) then only I can use the
> > listener. But the problem is we are having more than 2000 tests and
> > specifying a timeout in each of the tests/classes is cumbersome.
> >
> > Correct me if I have misunderstood anything.
> >
> >
> >
> > On Fri, Oct 4, 2019 at 3:18 PM Debraj Manna 
> > wrote:
> >
> > > Yeah sure ... thanks.
> > >
> > > On Thu, Oct 3, 2019 at 7:50 PM Tibor Digana 
> > > wrote:
> > >
> > >> Hi Debraj,
> > >>
> > >> There is nice technical idea from Enrico.
> > >> If you apply it and you are convinced that it would work properly for
> > all
> > >> the Java community, feel free to show it and we can discuss it on how
> we
> > >> would adopt your solution in Surefire project.
> > >>
> > >> Cheers
> > >> Tibor17
> > >>
> > >> On Thu, Oct 3, 2019 at 2:49 PM Debraj Manna  >
> > >> wrote:
> > >>
> > >> > Sometimes I have maven surefire tests that get hung, due to either
> > >> races or
> > >> > deadlocks.
> > >> >
> > >> > When this happens I have to discover what slave is being used, and
> > then
> > >> I
> > >> > have to log on that slave, sudo to jenkins account and execute
> either
> > >> > jstack or kill -3
> > >> >
> > >> > I am looking for a simple solution like doing jstack / kill -3 when
> > >> someone
> > >> > presses abort button on the jenkins.
> > >> >
> > >> > Can someone suggest how can I automate this or some better way of
> > >> handling
> > >> > this?
> > >> >
> > >>
> > >
> >
>

Re: Proposal: maven release lifecycle

[Romain Manni-Bucau]

I like the words but fail to see the missing pieces (understand actions to
do).

Typically today when i release at work i use release plugin enriched with
github page deployment for the doc using antora + a ftp update of my server
output capture to have a mock backing openapi/swagger ui + docker ilage
deployment on docker hub + an intellij plugin deployment on jetbrains
marketplace etc...adding a flyway migration with a rolling update in a k8s
cluster is trivial (ops need to say yes though ;)).

So technically it is verbose but doable.

Custom lifecycle definition is not neat, custom build tasks require a build
hack or using groovy plugin but it works.

Typically, an extension could enable all that based on convention, just
injecting needed plugins based on the file presence and values of the
distribution urls.

This is why i ended up thinking we are more on 1. Sugar in release plugin
and 2. Maybe on a generic extension cretion (it replaces rigid lifecycle
for such cases these days due to their writing easiness)

Am I missing something?

Le sam. 5 oct. 2019 à 08:23, Marco Schulz  a
écrit :

> Hello Romain, hello Tibor
>
> Thanks for your feedback. I had yesterday a very interesting conversation
> with
> Karl Heinz. He gave me some very informative links about deeply maven
> insights.
> Before I saw his talk on youtube I thought I have a good knowledge about
> maven
> ;-)  now I was lerning a lot of new things.
>
> He defined Maven as an plugin execution framework. I like to extend this
> definition: Maven is a process engine framework and define industrial
> defacto
> standards for the software development process. And with this idea in mind
> I
> think it could be great to define more workflows in a equal manner like the
> build lifecycle. The basic proposal is a first draft and I know some
> points are
> missing. I tried to keep the release process as simple as possible. Two
> positions in this definition was impotent for me.
>
> Often companies don't really have well structured release processes. For
> that
> reason I would be great to have a workable standard. I have a small Java
> project
> on GitHub, an sometimes I also deploy to maven central. If you do this the
> first
> time a bit complicated process. And publishing on maven central is also a
> release process. So I had the intention to define some ordered steps like
> in the
> build lifecycle to prepare and publish a release.
>
> Let me describe an example scenario for a release: After mvn build was
> successful executed all unit tests are passed, the sprint is finished and
> as
> build manager we want to release our artifact. Very important would be an
> option
> to take the results from the build lifecycle and prepare the release. As
> first
> we need to see that the planned artifact version is not exist in any
> configured
> public repository and the pom contains all mandatory information for
> publishing
> on maven central(validate). To secure a reproducible build, in a second
> step we
> enforce that no SNAPSHOT artifacts are involved, the correct maven version
> is
> used etc. (the existing enforcer plugin do a great job). After the
> preconditions
> are fine we can execute external acceptance tests like JGiven. In another
> step
> the JavaDoc got generated. the pgp plugin sing the artifacts and check
> that the
> public key is available. The SCM plugin create a tag for the released
> revision.
> Difficult parts are auto increment the version number and auto check the
> scm if
> the release is produced with the last revision of the code. This actions
> are by
> my experience a bit error prune.
> A release process could have some vocabulary like prepare, perform,
> deploy. This
> are just conventions, like it is used in the build lifecycle. To realize
> this
> idea, many already existing plugins can used, like the release plugin. In
> this
> proposal I was also not mentions external plugins to use like flyway, for
> database versioning and so on. A lot of things ar imaginable.
>
> In future more lifecycles or workflows could be possible. For example a
> deploy
> lifecycle and so on. And then maven transform from a plugin execution
> framework
> to a process management framework, like Jason van Zyl described in his book
> better build with maven.
>
> warm regards to all
> .marco
>
>
>
>
> On Fri, 2019-10-04 at 11:28 +0200, Romain Manni-Bucau wrote:
> > @Tibor: I agree merging both in one "super" command can be neat (I always
> > run both at once typically) but I disagree with last parts "skip the
> test"
> > - maven is also there to enforce tests as a good practise, if you don't
> > automatically test it you can configure maven to skip tests for the
> release
> > but it is at your own risk, can be fine or not - and "skip the deploy" -
> > here again you can configure maven to do it if you need but maven must
> > respect the build attached artifact.
> >
> > Romain Manni-Bucau
> > @rmannibucau  |  Blog
> >