Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

@Michael-O
The issue(two errors in ForkModeIT)  with commit
e0bcffd05bb04001f97bf752de56bca7137da3e2 is caused by duplicate properties
but it is not cause of that commit.
I have found the root cause (reuseForks duplicates):
-DforkMode=perthread *-DreuseForks=false -DreuseForks=true* -DthreadCount=1
I have the same tests failed as you had:

Failed tests:
testForkModeOncePerThreadSingleThread(org.apache.maven.surefire.its.ForkModeIT):
pid 1 didn't match pid 2 expected:<[19220]@SK-ZA-04278 testVal...> but
was:<[23028]@SK-ZA-04278 testVal...>

testForkModeOncePerThreadTwoThreads(org.apache.maven.surefire.its.ForkModeIT):
number of different pids is not as expected expected:<2> but was:<3>

We already prevented from this issue in the branch
https://github.com/apache/maven-surefire/commits/SUREFIRE_SYSPROP_DUPLICATES
but of course the old commits, like e0bcffd05, do not have this fix and
therefore this issue come up.

I will remove duplicates with system property "testNgVersion" in 17 POMs of
our tests in the branch.


On Thu, Feb 23, 2017 at 7:20 PM, Michael Osipov  wrote:

> So guys,
>
> I took a different approach to provide Tibor more information about the
> failures. First of all, I added another platform for the test which is
> exotic: HP-UX 11.31. The VM comes directly from HPE which is a repackaged
> Oracle JVM.
>
> I started Git bisect and discovered some broken commits, e.g.,
> 9dd4074e83c0edae5e2050f66e9cadfdba40fe01 where less tests previously
> fail, but doing a git revert of the offending commit does not resolve the
> issue. After that I started a manual bisect going binary through commits
> where I found that last known good commit 
> 502d18442113b4c6c72630dca5842e1eb287b8b0
> has a lower number of failures while the next commit
> e0bcffd05bb04001f97bf752de56bca7137da3e2 introduced a regression from my
> point of view, Tibor has a different opinion.
> Going deeper I have compared the same debug logs between FreeBSD and HP-UX
> and noticed that the consumer from the plugin is too greedy and declares
> the forked VM as dead. I consider this as race conditions as well as stdio
> issues not solvable w/o native code (I might be wrong).
>
> For those who would like to take a closer look at the results, see again
> here:
>
> http://home.apache.org/~michaelo/
>
>> maven-surefire-2.19.2-experimental.tar.gz
>> maven-surefire-308d941c9b6ebb695aba9630f81fc5b400f21322-all-ITs.tar.gz
>> maven-surefire-502d18442113b4c6c72630dca5842e1eb287b8b0.tar.gz
>> maven-surefire-502d18442113b4c6c72630dca5842e1eb287b8b0-all-ITs.tar.gz
>> maven-surefire-502d18442113b4c6c72630dca5842e1eb287b8b0-all-
>> ITs-hp-ux.tar.gz
>> maven-surefire-e0bcffd05bb04001f97bf752de56bca7137da3e2.tar.gz
>> maven-surefire-e0bcffd05bb04001f97bf752de56bca7137da3e2-all-ITs.tar.gz
>> maven-surefire-e0bcffd05bb04001f97bf752de56bca7137da3e2-all-
>> ITs-hp-ux.tar.gz
>>
>
> I consider several commits between Surefire 2.19.1 and master
> broken/regressions.
>
> Tibor, feel free to add your comments and findings for everyone. Your
> analysis is better than mine since your know the code better than I do.
>
> Michael
>
> PS: All build done with Maven master. I will switch to 3.5.0-alpha-1.
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>


-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/23/17 um 19:20 schrieb Michael Osipov:
> So guys,
> 
> I took a different approach to provide Tibor more information about the 
> failures. First of all, I added another platform for the test which is 
> exotic: HP-UX 11.31. The VM comes directly from HPE which is a 
> repackaged Oracle JVM.

Good to have some System V in the mix.


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

So guys,

I took a different approach to provide Tibor more information about the 
failures. First of all, I added another platform for the test which is 
exotic: HP-UX 11.31. The VM comes directly from HPE which is a 
repackaged Oracle JVM.


I started Git bisect and discovered some broken commits, e.g., 
9dd4074e83c0edae5e2050f66e9cadfdba40fe01 where less tests previously 
fail, but doing a git revert of the offending commit does not resolve 
the issue. After that I started a manual bisect going binary through 
commits where I found that last known good commit 
502d18442113b4c6c72630dca5842e1eb287b8b0 has a lower number of failures 
while the next commit e0bcffd05bb04001f97bf752de56bca7137da3e2 
introduced a regression from my point of view, Tibor has a different 
opinion.
Going deeper I have compared the same debug logs between FreeBSD and 
HP-UX and noticed that the consumer from the plugin is too greedy and 
declares the forked VM as dead. I consider this as race conditions as 
well as stdio issues not solvable w/o native code (I might be wrong).


For those who would like to take a closer look at the results, see again 
here:


http://home.apache.org/~michaelo/

maven-surefire-2.19.2-experimental.tar.gz
maven-surefire-308d941c9b6ebb695aba9630f81fc5b400f21322-all-ITs.tar.gz
maven-surefire-502d18442113b4c6c72630dca5842e1eb287b8b0.tar.gz
maven-surefire-502d18442113b4c6c72630dca5842e1eb287b8b0-all-ITs.tar.gz
maven-surefire-502d18442113b4c6c72630dca5842e1eb287b8b0-all-ITs-hp-ux.tar.gz
maven-surefire-e0bcffd05bb04001f97bf752de56bca7137da3e2.tar.gz
maven-surefire-e0bcffd05bb04001f97bf752de56bca7137da3e2-all-ITs.tar.gz
maven-surefire-e0bcffd05bb04001f97bf752de56bca7137da3e2-all-ITs-hp-ux.tar.gz


I consider several commits between Surefire 2.19.1 and master 
broken/regressions.


Tibor, feel free to add your comments and findings for everyone. Your 
analysis is better than mine since your know the code better than I do.


Michael

PS: All build done with Maven master. I will switch to 3.5.0-alpha-1.

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/20/17 um 01:10 schrieb Tibor Digana:
> Can somebody explore what is different between these two surefire builds
> (failed and successful) and why?:
> https://builds.apache.org/view/Maven%20Master%20Release%20Status/

This one is run with current Maven master using JDK 7.

> https://builds.apache.org/job/maven-surefire/

This one is run with Maven 3.2.1 using JDK 8.

You should be able to log into Jenkins using your apache id. If this
does not work for you, ping someone to setup the correct permissions.

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/20/17 um 01:10 schrieb Tibor Digana:
> I meant this build which is in good conditions
> https://builds.apache.org/view/Maven%20Master%20Release%20Status/job/maven-master-release-status-test-surefire-linux/30/console
> 
> Can somebody explore what is different between these two surefire builds
> (failed and successful) and why?:
> https://builds.apache.org/view/Maven%20Master%20Release%20Status/
> https://builds.apache.org/job/maven-surefire/

Please see the attached patch. System.out and System.err do buffering
internally in Java. System.exit seems to not flush/close those streams
internally. I really think we just need to close System.in/System.err
manually everywhere before exiting to get the buffers flushed correctly.

Regards,
-- 
Christian

diff --git a/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java b/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java
index b76df2f..7b9ec7d 100644
--- a/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java
+++ b/surefire-booter/src/main/java/org/apache/maven/surefire/booter/ForkedBooter.java
@@ -243,6 +243,8 @@ private static void exit( int returnCode, Shutdown shutdownType, CommandReader r
 reader.stop();
 }
 launchLastDitchDaemonShutdownThread( returnCode );
+System.out.close();
+System.err.close();
 System.exit( returnCode );
 case DEFAULT:
 // refers to shutdown=testset, but not used now, keeping reader open


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/20/17 um 01:10 schrieb Tibor Digana:
> I meant this build which is in good conditions
> https://builds.apache.org/view/Maven%20Master%20Release%20Status/job/maven-master-release-status-test-surefire-linux/30/console

This points to a console with IT failures. That build never succeeded on
Jenkins. There had been issues with the setup of those jobs. That should
be solved. It does not show the issues we are seeing on the BSDs. That's
what I am referring to with "is only reproducible on *BSD." Here is a
build job showing the issue exists on those others OSes as well.



>From the maven-assembly-plugin:

[INFO] Building:
projects\descriptor-refs\jar-with-dependencies\deps-unpacked-to-root-dir\pom.xml
[INFO] ..FAILED (7.6 s)

You need to log into Jenkins to see the build.log from the workspace.
That build.log contained that "Did not properly say good bye." message
from surefire. I just restarted that job due to this, so that build.log
will have disappeared already. It should run successfully. There are no
plugin ITs failing on windows. Reading the Surefire FAQ on this issue,
it states that this may happen when something runs out of resources etc.
This led to pull request
. Although real
resource leaks, I doubt it's the OS running out of resources producing
that message.

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

@Michael
I have pushed new experimental changes with *FileDescriptor.out* in the
branch 2.19.2-experimental.
Can you pls trigger the build on FreeBSD?
I use only one IT and the build passed successfully - ForkedModeIT.

Running org.apache.maven.surefire.its.ForkModeIT
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 108.522 sec

Results :

Tests run: 12, Failures: 0, Errors: 0, Skipped: 0



On Mon, Feb 20, 2017 at 1:10 AM, Tibor Digana 
wrote:

> I meant this build which is in good conditions
> https://builds.apache.org/view/Maven%20Master%20Release%
> 20Status/job/maven-master-release-status-test-surefire-linux/30/console
>
> Can somebody explore what is different between these two surefire builds
> (failed and successful) and why?:
> https://builds.apache.org/view/Maven%20Master%20Release%20Status/
> https://builds.apache.org/job/maven-surefire/
>
>
> On Mon, Feb 20, 2017 at 12:43 AM, Christian Schulte  wrote:
>
>> Am 02/19/17 um 23:58 schrieb Tibor Digana:
>> > and they are getting really nervous. So we have to make a release and
>> later
>> > we will finish fix for FreeBSD. I am still facing native problem with
>> > reliably streaming data over standard output streams with subprocess.
>> > We with Kristian Roselvold were thinking of introducing extensions for
>> > Surefire in Version 3.0. It seems we will do it in 2.x but not in this
>> > version because we are already running one year without release.
>>
>> No objections to cutting releases or to anything else. There is one
>> thing to note here. @Michael: Please correct me if I am wrong. This is
>> not a FreeBSD or OpenBSD only issue. It is happening on Windows and
>> Linux as well. It's fully reproducible on OpenBSD and - as it seems - on
>> FreeBSD as well. Those BSDs happen to be using the same JDK port from
>> here:
>>
>> 
>>
>> I doubt this port is to blame, because this issue exists with the
>> Jenkins build jobs as well (Ubuntu and Windows). It's good to have a
>> setup around where things like this are reproducible.
>>
>> Regards,
>> --
>> Christian
>>
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
>> For additional commands, e-mail: dev-h...@maven.apache.org
>>
>>
>
>
> --
> Cheers
> Tibor
>



-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

I meant this build which is in good conditions
https://builds.apache.org/view/Maven%20Master%20Release%20Status/job/maven-master-release-status-test-surefire-linux/30/console

Can somebody explore what is different between these two surefire builds
(failed and successful) and why?:
https://builds.apache.org/view/Maven%20Master%20Release%20Status/
https://builds.apache.org/job/maven-surefire/


On Mon, Feb 20, 2017 at 12:43 AM, Christian Schulte  wrote:

> Am 02/19/17 um 23:58 schrieb Tibor Digana:
> > and they are getting really nervous. So we have to make a release and
> later
> > we will finish fix for FreeBSD. I am still facing native problem with
> > reliably streaming data over standard output streams with subprocess.
> > We with Kristian Roselvold were thinking of introducing extensions for
> > Surefire in Version 3.0. It seems we will do it in 2.x but not in this
> > version because we are already running one year without release.
>
> No objections to cutting releases or to anything else. There is one
> thing to note here. @Michael: Please correct me if I am wrong. This is
> not a FreeBSD or OpenBSD only issue. It is happening on Windows and
> Linux as well. It's fully reproducible on OpenBSD and - as it seems - on
> FreeBSD as well. Those BSDs happen to be using the same JDK port from here:
>
> 
>
> I doubt this port is to blame, because this issue exists with the
> Jenkins build jobs as well (Ubuntu and Windows). It's good to have a
> setup around where things like this are reproducible.
>
> Regards,
> --
> Christian
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>


-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/19/17 um 23:58 schrieb Tibor Digana:
> and they are getting really nervous. So we have to make a release and later
> we will finish fix for FreeBSD. I am still facing native problem with
> reliably streaming data over standard output streams with subprocess.
> We with Kristian Roselvold were thinking of introducing extensions for
> Surefire in Version 3.0. It seems we will do it in 2.x but not in this
> version because we are already running one year without release.

No objections to cutting releases or to anything else. There is one
thing to note here. @Michael: Please correct me if I am wrong. This is
not a FreeBSD or OpenBSD only issue. It is happening on Windows and
Linux as well. It's fully reproducible on OpenBSD and - as it seems - on
FreeBSD as well. Those BSDs happen to be using the same JDK port from here:



I doubt this port is to blame, because this issue exists with the
Jenkins build jobs as well (Ubuntu and Windows). It's good to have a
setup around where things like this are reproducible.

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

Currently there is another issue.
Events from plugin don't arrive to forked jvm.
I have placed more logs. We will run the ITs again with Michael tomorrow. I
open to check logs in the morning and in the evening.
The Version 2.19.1 has these issues as well. So it is not a regression.
We have to continue with release of surefire because the Ubuntu tests pass
same as it was with previous versions. The other build process based on
Jenkinsfile is different story where only some text is not found in logs. I
will have a look, but still this does not mean that we have to stop cut
release new version of Surefire. The users are sending me emails one by one
and they are getting really nervous. So we have to make a release and later
we will finish fix for FreeBSD. I am still facing native problem with
reliably streaming data over standard output streams with subprocess.
We with Kristian Roselvold were thinking of introducing extensions for
Surefire in Version 3.0. It seems we will do it in 2.x but not in this
version because we are already running one year without release.


On Sun, Feb 19, 2017 at 10:23 PM, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> Especially as we know the forked money should *exit before* the main
> process, so we should be able to flush the ACK...
>
> The timeout would just be a fallback to prevent zombies
>
> On Sun 19 Feb 2017 at 21:09, Michael Osipov  wrote:
>
> > I made this proposal to Tibor too. The forked VM sends the Z, waits for
> > an ACK and then exits. This should be idiot proof.
> >
> > Am 2017-02-19 um 22:04 schrieb Stephen Connolly:
> > > Can we invert the sequence?
> > >
> > > The forked process sends a Z
> > >
> > > Then the controller sends back an ACK of some sort. If the forked
> process
> > > hasn't received the ACK say 5s after sending the Z is just says screw
> it
> > > and does an exit?
> > > On Sun 19 Feb 2017 at 20:34, Christian Schulte  wrote:
> > >
> > >> Am 19.02.2017 um 21:10 schrieb Michael Osipov:
> > >>> Am 2017-02-19 um 19:54 schrieb Christian Schulte:
> >  Am 02/19/17 um 19:52 schrieb Christian Schulte:
> > > Am 02/19/17 um 19:47 schrieb Christian Schulte:
> > >> Am 02/19/17 um 13:06 schrieb Michael Osipov:
> > >>> Socket give you, of course, a lot of control, but they also
> impose
> > an
> > >>> overhead compares to named pipes (mkfifo, CreateNamePipe).
> > >>>
> > >>> I think our biggest problem is that stdout is subject to
> buffering
> > >>> issues and in case if System#exit() status Z is the buffer, never
> > >>> written out.
> > >>> Surefire thinks that the crashed.
> > >>
> > >> Not sure I understand the issue at hand completely. If the code
> > >> calling
> > >> "exit" is in Surefire and the code consuming the stdout/stderr
> > streams
> > >> also is in Surefire, there is a solution to this. Can we just
> close
> > >> the
> > >> stdio streams before calling exit? Would that be possible? Just
> add
> > >> System.out.close(), System.err.close() and System.in.close()
> before
> > >> the
> > >> call to System.exit(). This could solve it. Will look up posix
> > >> documentation on this now. We maybe are just running into an issue
> > >> where
> > >> the BSDs get it right and others are not strictly posix compliant.
> > >> Getting someone to test this on OSX would be interesting (kernel
> or
> > >> userland issue). This could even be a bug in the VM (not platform
> > >> independent).
> > >>
> > >> Regards,
> > >>
> > >
> > > Quoting from here:
> > >
> > > 
> > >
> > > "A file is disassociated from a stream by “closing” it. Output
> > streams
> > > are flushed (any unwritten buffer contents are transferred to the
> > host
> > > environment) before the stream is disassociated from the file."
> > >
> > > BUGs section has a hint about buffering as well.
> > >
> > > Regards,
> > >
> > 
> >  Well. It's mentioned there explicitly:
> > 
> >  "If the main function returns to its original caller, or the exit(3)
> >  function is called, all open files are closed (hence all output
> > streams
> >  are flushed) before program termination. Other methods of program
> >  termination may not close files properly and hence buffered output
> may
> >  be lost. In particular, _exit(2) does not flush stdio files. Neither
> >  does an exit due to a signal. Buffers are flushed by abort(3), as
> >  required by POSIX, although in previous implementations they were
> > not."
> > >>>
> > >>> This is the same content as for FreeBSD and I think this is likely
> the
> > >>> case here. BSDs strictly adhere to the standards, other
> implementations
> > >>> like GNU/Linux might be sloppier.
> > >>>
> > >>> Using stderr could be a solution. At best, we would have
> setbuf(STDOUT,
> > >>> NULL) 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

Especially as we know the forked money should *exit before* the main
process, so we should be able to flush the ACK...

The timeout would just be a fallback to prevent zombies

On Sun 19 Feb 2017 at 21:09, Michael Osipov  wrote:

> I made this proposal to Tibor too. The forked VM sends the Z, waits for
> an ACK and then exits. This should be idiot proof.
>
> Am 2017-02-19 um 22:04 schrieb Stephen Connolly:
> > Can we invert the sequence?
> >
> > The forked process sends a Z
> >
> > Then the controller sends back an ACK of some sort. If the forked process
> > hasn't received the ACK say 5s after sending the Z is just says screw it
> > and does an exit?
> > On Sun 19 Feb 2017 at 20:34, Christian Schulte  wrote:
> >
> >> Am 19.02.2017 um 21:10 schrieb Michael Osipov:
> >>> Am 2017-02-19 um 19:54 schrieb Christian Schulte:
>  Am 02/19/17 um 19:52 schrieb Christian Schulte:
> > Am 02/19/17 um 19:47 schrieb Christian Schulte:
> >> Am 02/19/17 um 13:06 schrieb Michael Osipov:
> >>> Socket give you, of course, a lot of control, but they also impose
> an
> >>> overhead compares to named pipes (mkfifo, CreateNamePipe).
> >>>
> >>> I think our biggest problem is that stdout is subject to buffering
> >>> issues and in case if System#exit() status Z is the buffer, never
> >>> written out.
> >>> Surefire thinks that the crashed.
> >>
> >> Not sure I understand the issue at hand completely. If the code
> >> calling
> >> "exit" is in Surefire and the code consuming the stdout/stderr
> streams
> >> also is in Surefire, there is a solution to this. Can we just close
> >> the
> >> stdio streams before calling exit? Would that be possible? Just add
> >> System.out.close(), System.err.close() and System.in.close() before
> >> the
> >> call to System.exit(). This could solve it. Will look up posix
> >> documentation on this now. We maybe are just running into an issue
> >> where
> >> the BSDs get it right and others are not strictly posix compliant.
> >> Getting someone to test this on OSX would be interesting (kernel or
> >> userland issue). This could even be a bug in the VM (not platform
> >> independent).
> >>
> >> Regards,
> >>
> >
> > Quoting from here:
> >
> > 
> >
> > "A file is disassociated from a stream by “closing” it. Output
> streams
> > are flushed (any unwritten buffer contents are transferred to the
> host
> > environment) before the stream is disassociated from the file."
> >
> > BUGs section has a hint about buffering as well.
> >
> > Regards,
> >
> 
>  Well. It's mentioned there explicitly:
> 
>  "If the main function returns to its original caller, or the exit(3)
>  function is called, all open files are closed (hence all output
> streams
>  are flushed) before program termination. Other methods of program
>  termination may not close files properly and hence buffered output may
>  be lost. In particular, _exit(2) does not flush stdio files. Neither
>  does an exit due to a signal. Buffers are flushed by abort(3), as
>  required by POSIX, although in previous implementations they were
> not."
> >>>
> >>> This is the same content as for FreeBSD and I think this is likely the
> >>> case here. BSDs strictly adhere to the standards, other implementations
> >>> like GNU/Linux might be sloppier.
> >>>
> >>> Using stderr could be a solution. At best, we would have setbuf(STDOUT,
> >>> NULL) from without Java, but this requires native code :-(
> >>
> >> The buffering of System.in/out/err is controlled solely by the OS and
> >> the VM does not do anything to make this behave consistently across
> >> OSes? The VM isn't platform independent in that area then. If you can
> >> present a native solution on how to make the VM more platform
> >> independent in that area, time to file a bug against the JDK
> >> , IMHO. Does not solve our issue now,
> >> though. Means we need to find a solution for Surefire which is neither a
> >> hack nor a workaround which will hit us again in the future. Maybe
> >> System.in/out/err cannot be used reliably for what we are using it for.
> >>
> >> Regards,
> >> --
> >> Christian
> >>
> >>
> >> -
> >> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> >> For additional commands, e-mail: dev-h...@maven.apache.org
> >>
> >> --
> > Sent from my phone
> >
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
> --
Sent from my phone

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

I made this proposal to Tibor too. The forked VM sends the Z, waits for 
an ACK and then exits. This should be idiot proof.


Am 2017-02-19 um 22:04 schrieb Stephen Connolly:

Can we invert the sequence?

The forked process sends a Z

Then the controller sends back an ACK of some sort. If the forked process
hasn't received the ACK say 5s after sending the Z is just says screw it
and does an exit?
On Sun 19 Feb 2017 at 20:34, Christian Schulte  wrote:


Am 19.02.2017 um 21:10 schrieb Michael Osipov:

Am 2017-02-19 um 19:54 schrieb Christian Schulte:

Am 02/19/17 um 19:52 schrieb Christian Schulte:

Am 02/19/17 um 19:47 schrieb Christian Schulte:

Am 02/19/17 um 13:06 schrieb Michael Osipov:

Socket give you, of course, a lot of control, but they also impose an
overhead compares to named pipes (mkfifo, CreateNamePipe).

I think our biggest problem is that stdout is subject to buffering
issues and in case if System#exit() status Z is the buffer, never
written out.
Surefire thinks that the crashed.


Not sure I understand the issue at hand completely. If the code

calling

"exit" is in Surefire and the code consuming the stdout/stderr streams
also is in Surefire, there is a solution to this. Can we just close

the

stdio streams before calling exit? Would that be possible? Just add
System.out.close(), System.err.close() and System.in.close() before

the

call to System.exit(). This could solve it. Will look up posix
documentation on this now. We maybe are just running into an issue

where

the BSDs get it right and others are not strictly posix compliant.
Getting someone to test this on OSX would be interesting (kernel or
userland issue). This could even be a bug in the VM (not platform
independent).

Regards,



Quoting from here:



"A file is disassociated from a stream by “closing” it. Output streams
are flushed (any unwritten buffer contents are transferred to the host
environment) before the stream is disassociated from the file."

BUGs section has a hint about buffering as well.

Regards,



Well. It's mentioned there explicitly:

"If the main function returns to its original caller, or the exit(3)
function is called, all open files are closed (hence all output streams
are flushed) before program termination. Other methods of program
termination may not close files properly and hence buffered output may
be lost. In particular, _exit(2) does not flush stdio files. Neither
does an exit due to a signal. Buffers are flushed by abort(3), as
required by POSIX, although in previous implementations they were not."


This is the same content as for FreeBSD and I think this is likely the
case here. BSDs strictly adhere to the standards, other implementations
like GNU/Linux might be sloppier.

Using stderr could be a solution. At best, we would have setbuf(STDOUT,
NULL) from without Java, but this requires native code :-(


The buffering of System.in/out/err is controlled solely by the OS and
the VM does not do anything to make this behave consistently across
OSes? The VM isn't platform independent in that area then. If you can
present a native solution on how to make the VM more platform
independent in that area, time to file a bug against the JDK
, IMHO. Does not solve our issue now,
though. Means we need to find a solution for Surefire which is neither a
hack nor a workaround which will hit us again in the future. Maybe
System.in/out/err cannot be used reliably for what we are using it for.

Regards,
--
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

--

Sent from my phone





-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

Can we invert the sequence?

The forked process sends a Z

Then the controller sends back an ACK of some sort. If the forked process
hasn't received the ACK say 5s after sending the Z is just says screw it
and does an exit?
On Sun 19 Feb 2017 at 20:34, Christian Schulte  wrote:

> Am 19.02.2017 um 21:10 schrieb Michael Osipov:
> > Am 2017-02-19 um 19:54 schrieb Christian Schulte:
> >> Am 02/19/17 um 19:52 schrieb Christian Schulte:
> >>> Am 02/19/17 um 19:47 schrieb Christian Schulte:
>  Am 02/19/17 um 13:06 schrieb Michael Osipov:
> > Socket give you, of course, a lot of control, but they also impose an
> > overhead compares to named pipes (mkfifo, CreateNamePipe).
> >
> > I think our biggest problem is that stdout is subject to buffering
> > issues and in case if System#exit() status Z is the buffer, never
> > written out.
> > Surefire thinks that the crashed.
> 
>  Not sure I understand the issue at hand completely. If the code
> calling
>  "exit" is in Surefire and the code consuming the stdout/stderr streams
>  also is in Surefire, there is a solution to this. Can we just close
> the
>  stdio streams before calling exit? Would that be possible? Just add
>  System.out.close(), System.err.close() and System.in.close() before
> the
>  call to System.exit(). This could solve it. Will look up posix
>  documentation on this now. We maybe are just running into an issue
> where
>  the BSDs get it right and others are not strictly posix compliant.
>  Getting someone to test this on OSX would be interesting (kernel or
>  userland issue). This could even be a bug in the VM (not platform
>  independent).
> 
>  Regards,
> 
> >>>
> >>> Quoting from here:
> >>>
> >>> 
> >>>
> >>> "A file is disassociated from a stream by “closing” it. Output streams
> >>> are flushed (any unwritten buffer contents are transferred to the host
> >>> environment) before the stream is disassociated from the file."
> >>>
> >>> BUGs section has a hint about buffering as well.
> >>>
> >>> Regards,
> >>>
> >>
> >> Well. It's mentioned there explicitly:
> >>
> >> "If the main function returns to its original caller, or the exit(3)
> >> function is called, all open files are closed (hence all output streams
> >> are flushed) before program termination. Other methods of program
> >> termination may not close files properly and hence buffered output may
> >> be lost. In particular, _exit(2) does not flush stdio files. Neither
> >> does an exit due to a signal. Buffers are flushed by abort(3), as
> >> required by POSIX, although in previous implementations they were not."
> >
> > This is the same content as for FreeBSD and I think this is likely the
> > case here. BSDs strictly adhere to the standards, other implementations
> > like GNU/Linux might be sloppier.
> >
> > Using stderr could be a solution. At best, we would have setbuf(STDOUT,
> > NULL) from without Java, but this requires native code :-(
>
> The buffering of System.in/out/err is controlled solely by the OS and
> the VM does not do anything to make this behave consistently across
> OSes? The VM isn't platform independent in that area then. If you can
> present a native solution on how to make the VM more platform
> independent in that area, time to file a bug against the JDK
> , IMHO. Does not solve our issue now,
> though. Means we need to find a solution for Surefire which is neither a
> hack nor a workaround which will hit us again in the future. Maybe
> System.in/out/err cannot be used reliably for what we are using it for.
>
> Regards,
> --
> Christian
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
> --
Sent from my phone

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 19.02.2017 um 21:10 schrieb Michael Osipov:
> Am 2017-02-19 um 19:54 schrieb Christian Schulte:
>> Am 02/19/17 um 19:52 schrieb Christian Schulte:
>>> Am 02/19/17 um 19:47 schrieb Christian Schulte:
 Am 02/19/17 um 13:06 schrieb Michael Osipov:
> Socket give you, of course, a lot of control, but they also impose an
> overhead compares to named pipes (mkfifo, CreateNamePipe).
>
> I think our biggest problem is that stdout is subject to buffering
> issues and in case if System#exit() status Z is the buffer, never
> written out.
> Surefire thinks that the crashed.

 Not sure I understand the issue at hand completely. If the code calling
 "exit" is in Surefire and the code consuming the stdout/stderr streams
 also is in Surefire, there is a solution to this. Can we just close the
 stdio streams before calling exit? Would that be possible? Just add
 System.out.close(), System.err.close() and System.in.close() before the
 call to System.exit(). This could solve it. Will look up posix
 documentation on this now. We maybe are just running into an issue where
 the BSDs get it right and others are not strictly posix compliant.
 Getting someone to test this on OSX would be interesting (kernel or
 userland issue). This could even be a bug in the VM (not platform
 independent).

 Regards,

>>>
>>> Quoting from here:
>>>
>>> 
>>>
>>> "A file is disassociated from a stream by “closing” it. Output streams
>>> are flushed (any unwritten buffer contents are transferred to the host
>>> environment) before the stream is disassociated from the file."
>>>
>>> BUGs section has a hint about buffering as well.
>>>
>>> Regards,
>>>
>>
>> Well. It's mentioned there explicitly:
>>
>> "If the main function returns to its original caller, or the exit(3)
>> function is called, all open files are closed (hence all output streams
>> are flushed) before program termination. Other methods of program
>> termination may not close files properly and hence buffered output may
>> be lost. In particular, _exit(2) does not flush stdio files. Neither
>> does an exit due to a signal. Buffers are flushed by abort(3), as
>> required by POSIX, although in previous implementations they were not."
> 
> This is the same content as for FreeBSD and I think this is likely the 
> case here. BSDs strictly adhere to the standards, other implementations 
> like GNU/Linux might be sloppier.
> 
> Using stderr could be a solution. At best, we would have setbuf(STDOUT, 
> NULL) from without Java, but this requires native code :-(

The buffering of System.in/out/err is controlled solely by the OS and
the VM does not do anything to make this behave consistently across
OSes? The VM isn't platform independent in that area then. If you can
present a native solution on how to make the VM more platform
independent in that area, time to file a bug against the JDK
, IMHO. Does not solve our issue now,
though. Means we need to find a solution for Surefire which is neither a
hack nor a workaround which will hit us again in the future. Maybe
System.in/out/err cannot be used reliably for what we are using it for.

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

Am 2017-02-19 um 19:54 schrieb Christian Schulte:

Am 02/19/17 um 19:52 schrieb Christian Schulte:

Am 02/19/17 um 19:47 schrieb Christian Schulte:

Am 02/19/17 um 13:06 schrieb Michael Osipov:

Socket give you, of course, a lot of control, but they also impose an
overhead compares to named pipes (mkfifo, CreateNamePipe).

I think our biggest problem is that stdout is subject to buffering
issues and in case if System#exit() status Z is the buffer, never
written out.
Surefire thinks that the crashed.


Not sure I understand the issue at hand completely. If the code calling
"exit" is in Surefire and the code consuming the stdout/stderr streams
also is in Surefire, there is a solution to this. Can we just close the
stdio streams before calling exit? Would that be possible? Just add
System.out.close(), System.err.close() and System.in.close() before the
call to System.exit(). This could solve it. Will look up posix
documentation on this now. We maybe are just running into an issue where
the BSDs get it right and others are not strictly posix compliant.
Getting someone to test this on OSX would be interesting (kernel or
userland issue). This could even be a bug in the VM (not platform
independent).

Regards,



Quoting from here:



"A file is disassociated from a stream by “closing” it. Output streams
are flushed (any unwritten buffer contents are transferred to the host
environment) before the stream is disassociated from the file."

BUGs section has a hint about buffering as well.

Regards,



Well. It's mentioned there explicitly:

"If the main function returns to its original caller, or the exit(3)
function is called, all open files are closed (hence all output streams
are flushed) before program termination. Other methods of program
termination may not close files properly and hence buffered output may
be lost. In particular, _exit(2) does not flush stdio files. Neither
does an exit due to a signal. Buffers are flushed by abort(3), as
required by POSIX, although in previous implementations they were not."


This is the same content as for FreeBSD and I think this is likely the 
case here. BSDs strictly adhere to the standards, other implementations 
like GNU/Linux might be sloppier.


Using stderr could be a solution. At best, we would have setbuf(STDOUT, 
NULL) from without Java, but this requires native code :-(




-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

"Initially, the standard error stream is unbuffered; "

Maybe just use System.err instead of System.out to avoid any buffering,
if there is no other solution.


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/19/17 um 19:52 schrieb Christian Schulte:
> Am 02/19/17 um 19:47 schrieb Christian Schulte:
>> Am 02/19/17 um 13:06 schrieb Michael Osipov:
>>> Socket give you, of course, a lot of control, but they also impose an 
>>> overhead compares to named pipes (mkfifo, CreateNamePipe).
>>>
>>> I think our biggest problem is that stdout is subject to buffering 
>>> issues and in case if System#exit() status Z is the buffer, never 
>>> written out.
>>> Surefire thinks that the crashed.
>>
>> Not sure I understand the issue at hand completely. If the code calling
>> "exit" is in Surefire and the code consuming the stdout/stderr streams
>> also is in Surefire, there is a solution to this. Can we just close the
>> stdio streams before calling exit? Would that be possible? Just add
>> System.out.close(), System.err.close() and System.in.close() before the
>> call to System.exit(). This could solve it. Will look up posix
>> documentation on this now. We maybe are just running into an issue where
>> the BSDs get it right and others are not strictly posix compliant.
>> Getting someone to test this on OSX would be interesting (kernel or
>> userland issue). This could even be a bug in the VM (not platform
>> independent).
>>
>> Regards,
>>
> 
> Quoting from here:
> 
> 
> 
> "A file is disassociated from a stream by “closing” it. Output streams
> are flushed (any unwritten buffer contents are transferred to the host
> environment) before the stream is disassociated from the file."
> 
> BUGs section has a hint about buffering as well.
> 
> Regards,
> 

Well. It's mentioned there explicitly:

"If the main function returns to its original caller, or the exit(3)
function is called, all open files are closed (hence all output streams
are flushed) before program termination. Other methods of program
termination may not close files properly and hence buffered output may
be lost. In particular, _exit(2) does not flush stdio files. Neither
does an exit due to a signal. Buffers are flushed by abort(3), as
required by POSIX, although in previous implementations they were not."



-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/19/17 um 19:47 schrieb Christian Schulte:
> Am 02/19/17 um 13:06 schrieb Michael Osipov:
>> Socket give you, of course, a lot of control, but they also impose an 
>> overhead compares to named pipes (mkfifo, CreateNamePipe).
>>
>> I think our biggest problem is that stdout is subject to buffering 
>> issues and in case if System#exit() status Z is the buffer, never 
>> written out.
>> Surefire thinks that the crashed.
> 
> Not sure I understand the issue at hand completely. If the code calling
> "exit" is in Surefire and the code consuming the stdout/stderr streams
> also is in Surefire, there is a solution to this. Can we just close the
> stdio streams before calling exit? Would that be possible? Just add
> System.out.close(), System.err.close() and System.in.close() before the
> call to System.exit(). This could solve it. Will look up posix
> documentation on this now. We maybe are just running into an issue where
> the BSDs get it right and others are not strictly posix compliant.
> Getting someone to test this on OSX would be interesting (kernel or
> userland issue). This could even be a bug in the VM (not platform
> independent).
> 
> Regards,
> 

Quoting from here:



"A file is disassociated from a stream by “closing” it. Output streams
are flushed (any unwritten buffer contents are transferred to the host
environment) before the stream is disassociated from the file."

BUGs section has a hint about buffering as well.

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/19/17 um 13:06 schrieb Michael Osipov:
> Socket give you, of course, a lot of control, but they also impose an 
> overhead compares to named pipes (mkfifo, CreateNamePipe).
> 
> I think our biggest problem is that stdout is subject to buffering 
> issues and in case if System#exit() status Z is the buffer, never 
> written out.
> Surefire thinks that the crashed.

Not sure I understand the issue at hand completely. If the code calling
"exit" is in Surefire and the code consuming the stdout/stderr streams
also is in Surefire, there is a solution to this. Can we just close the
stdio streams before calling exit? Would that be possible? Just add
System.out.close(), System.err.close() and System.in.close() before the
call to System.exit(). This could solve it. Will look up posix
documentation on this now. We maybe are just running into an issue where
the BSDs get it right and others are not strictly posix compliant.
Getting someone to test this on OSX would be interesting (kernel or
userland issue). This could even be a bug in the VM (not platform
independent).

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

On Sun 19 Feb 2017 at 12:06, Michael Osipov  wrote:

> Socket give you, of course, a lot of control, but they also impose an
> overhead compares to named pipes (mkfifo, CreateNamePipe).


Yep but windows restricts choice of technology. Docker for Windows (as in
where Windows is the guest (and consequently host) OS) doubly so.

Sockets allow forking to be remote (although SSH as a remoting engine would
give back stdio it can also give port forwarding)

>
>
> I think our biggest problem is that stdout is subject to buffering
> issues and in case if System#exit() status Z is the buffer, never
> written out.
> Surefire thinks that the crashed.
>
> Am 2017-02-19 um 12:32 schrieb Stephen Connolly:
> > What about using a local host bound socket rather than studio?
> >
> > In fact a socket could open up other options like forking the tests
> inside
> > docker, or running the surefire forks on remote JVMs.
> >
> > Plus we gain more control (modulo TCP retry/timeouts)
> >
> > On Sun 19 Feb 2017 at 11:16, Tibor Digana 
> > wrote:
> >
> >> Currently I placed sleep(1sec) before child process exit(0). If the
> reader
> >> process ThreadedStreamConsumer would read buffer entirely within 1 sec
> and
> >> the build pass then I have to add a confirmation mechanism where the
> exit
> >> event must be confirmed by plugin until short period of time.
> >> Later we should not use these pipes but standard file system, but we
> have
> >> to take care of SecurityException thrown by JUnit3.
> >>
> >> On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov 
> >> wrote:
> >>
> >>> Am 2017-02-18 um 23:19 schrieb Christian Schulte:
> >>>
>  Am 02/17/17 um 21:48 schrieb Michael Osipov:
> 
> > Hi folks,
> >
> > Christian Schulte asked me a couple of days ago wether I am able to
> > build Surefire master with Maven master. It was constantly failing
> for
> > him on his OpenBSD machines. Since I have several real servers with
> > FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able
> >> to
> > reproduce it. Our entire test infrastructure wasn't unfortunately!
> >
> > @Tibor: correct me if something is wrong or missing!
> >
> > After several days of heavy testing, thread dumps and log file
> analysis
> > with Tibor Digana and various Maven combinations (3.3.9, master,
> > MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are
> >> several
> > serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and
> >> likely
> > Maven Clean Plugin 3.0.0.
> >
> > Since crucial parts of Surefire rely on native code in the JVM
> (forks,
> > streams), our code was not robust enough.
> >
> > As of today we have found:
> >
> > * Missing flushes in streams caused forked VM to be apparently
> > non-responsive
> > * TestNG tests mostly failed due to duplicate contradicting
> properties
> > passed to forked VMs
> > * Uninitialized/too early attributes made daemon threads to kill
> forked
> > VMs
> > * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated
> > failures of a handful ITs
> > @Karl Heinz: were you able to figure out something here?
> >
> > Issues in JIRA are pending...
> >
> > Everyone's invited to take a look at the log output as well as the
> > target directory of surefire-integration-tests and contribute:
> > http://home.apache.org/~michaelo/maven/surefire/. The filenames
> should
> > be pretty much self-explanatory.
> >
> > My big question is: how can we improve our test infrastructure? Can
> we
> > raise with INFRA to get at least one FreeBSD and Solaris node for
> > Jenkins? I consider coverage on Windows and Ubuntu way to small, we
> do
> > not even have a CentOS node. Surefire ITs and Maven ITs are paramount
> > for us, we should treat them as such!
> >
> > Michael
> >
> 
>  Does any of your findings solve the following already? This is what
>  makes the Jenkins build jobs appear unreliable. It's sending out an
>  email about a failed job and all you see is something like the
> following
>  sporadically. I am having the same issue locally sporadically.
> 
>  [ERROR] Failed to execute goal
>  org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>  (default-test) on project child2: Execution default-test of goal
>  org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
>  forked VM terminated without saying properly goodbye. VM crash or
>  System.exit called ? -> [Help 1]
>  org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>  execute goal
> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>  (default-test) on project child2: Execution default-test of goal
>  org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
>  

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

On Sun 19 Feb 2017 at 11:51, Tibor Digana  wrote:

> Too much work for socket, and port can be used by other process.


Random assignment of port and pass the "secret" on launch.

Any unexpected socket connection you just reject.

Yes it is more work but how cool to have surefire extensions that delegate
test execution to remote JVMs.

>
> But We have good news, see IRC
> [12:43]  ThreadedStreamConsumer#23504796 run() :: items :: Z,0,BYE!
> [12:43]  we received Z
> [12:44]  now I have to find why ForkStarter is faster to check the
> Z event
> [12:45]  because ForkStarter does not see this event for some
> reason, it is stored in volatile variable, so maybe timing issue
>
>
> On Sun, Feb 19, 2017 at 12:32 PM, stephenconnolly [via Maven] <
> ml-node+s40175n5899224...@n5.nabble.com> wrote:
>
> > What about using a local host bound socket rather than studio?
> >
> > In fact a socket could open up other options like forking the tests
> inside
> > docker, or running the surefire forks on remote JVMs.
> >
> > Plus we gain more control (modulo TCP retry/timeouts)
> >
> > On Sun 19 Feb 2017 at 11:16, Tibor Digana <[hidden email]
> > >
> > wrote:
> >
> > > Currently I placed sleep(1sec) before child process exit(0). If the
> > reader
> > > process ThreadedStreamConsumer would read buffer entirely within 1 sec
> > and
> > > the build pass then I have to add a confirmation mechanism where the
> > exit
> > > event must be confirmed by plugin until short period of time.
> > > Later we should not use these pipes but standard file system, but we
> > have
> > > to take care of SecurityException thrown by JUnit3.
> > >
> > > On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov <[hidden email]
> > >
> > > wrote:
> > >
> > > > Am 2017-02-18 um 23:19 schrieb Christian Schulte:
> > > >
> > > >> Am 02/17/17 um 21:48 schrieb Michael Osipov:
> > > >>
> > > >>> Hi folks,
> > > >>>
> > > >>> Christian Schulte asked me a couple of days ago wether I am able to
> > > >>> build Surefire master with Maven master. It was constantly failing
> > for
> > > >>> him on his OpenBSD machines. Since I have several real servers with
> > > >>> FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was
> > able
> > > to
> > > >>> reproduce it. Our entire test infrastructure wasn't unfortunately!
> > > >>>
> > > >>> @Tibor: correct me if something is wrong or missing!
> > > >>>
> > > >>> After several days of heavy testing, thread dumps and log file
> > analysis
> > > >>> with Tibor Digana and various Maven combinations (3.3.9, master,
> > > >>> MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are
> > > several
> > > >>> serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and
> > > likely
> > > >>> Maven Clean Plugin 3.0.0.
> > > >>>
> > > >>> Since crucial parts of Surefire rely on native code in the JVM
> > (forks,
> > > >>> streams), our code was not robust enough.
> > > >>>
> > > >>> As of today we have found:
> > > >>>
> > > >>> * Missing flushes in streams caused forked VM to be apparently
> > > >>> non-responsive
> > > >>> * TestNG tests mostly failed due to duplicate contradicting
> > properties
> > > >>> passed to forked VMs
> > > >>> * Uninitialized/too early attributes made daemon threads to kill
> > forked
> > > >>> VMs
> > > >>> * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause
> > repeated
> > > >>> failures of a handful ITs
> > > >>> @Karl Heinz: were you able to figure out something here?
> > > >>>
> > > >>> Issues in JIRA are pending...
> > > >>>
> > > >>> Everyone's invited to take a look at the log output as well as the
> > > >>> target directory of surefire-integration-tests and contribute:
> > > >>> http://home.apache.org/~michaelo/maven/surefire/. The filenames
> > should
> > > >>> be pretty much self-explanatory.
> > > >>>
> > > >>> My big question is: how can we improve our test infrastructure? Can
> > we
> > > >>> raise with INFRA to get at least one FreeBSD and Solaris node for
> > > >>> Jenkins? I consider coverage on Windows and Ubuntu way to small, we
> > do
> > > >>> not even have a CentOS node. Surefire ITs and Maven ITs are
> > paramount
> > > >>> for us, we should treat them as such!
> > > >>>
> > > >>> Michael
> > > >>>
> > > >>
> > > >> Does any of your findings solve the following already? This is what
> > > >> makes the Jenkins build jobs appear unreliable. It's sending out an
> > > >> email about a failed job and all you see is something like the
> > following
> > > >> sporadically. I am having the same issue locally sporadically.
> > > >>
> > > >> [ERROR] Failed to execute goal
> > > >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
> > > >> (default-test) on project child2: Execution default-test of goal
> > > >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed:
> > The
> > > >> forked VM terminated without saying properly 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

Michael, pls try to build now. It might be multiple issues we are facing.
The threading issue workaround was pushed to 2.19.2-experimental.

On Sun, Feb 19, 2017 at 1:07 PM, Michael Osipov-2 [via Maven] <
ml-node+s40175n5899236...@n5.nabble.com> wrote:

> Socket give you, of course, a lot of control, but they also impose an
> overhead compares to named pipes (mkfifo, CreateNamePipe).
>
> I think our biggest problem is that stdout is subject to buffering
> issues and in case if System#exit() status Z is the buffer, never
> written out.
> Surefire thinks that the crashed.
>
> Am 2017-02-19 um 12:32 schrieb Stephen Connolly:
>
> > What about using a local host bound socket rather than studio?
> >
> > In fact a socket could open up other options like forking the tests
> inside
> > docker, or running the surefire forks on remote JVMs.
> >
> > Plus we gain more control (modulo TCP retry/timeouts)
> >
> > On Sun 19 Feb 2017 at 11:16, Tibor Digana <[hidden email]
> >
> > wrote:
> >
> >> Currently I placed sleep(1sec) before child process exit(0). If the
> reader
> >> process ThreadedStreamConsumer would read buffer entirely within 1 sec
> and
> >> the build pass then I have to add a confirmation mechanism where the
> exit
> >> event must be confirmed by plugin until short period of time.
> >> Later we should not use these pipes but standard file system, but we
> have
> >> to take care of SecurityException thrown by JUnit3.
> >>
> >> On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov <[hidden email]
> >
> >> wrote:
> >>
> >>> Am 2017-02-18 um 23:19 schrieb Christian Schulte:
> >>>
>  Am 02/17/17 um 21:48 schrieb Michael Osipov:
> 
> > Hi folks,
> >
> > Christian Schulte asked me a couple of days ago wether I am able to
> > build Surefire master with Maven master. It was constantly failing
> for
> > him on his OpenBSD machines. Since I have several real servers with
> > FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was
> able
> >> to
> > reproduce it. Our entire test infrastructure wasn't unfortunately!
> >
> > @Tibor: correct me if something is wrong or missing!
> >
> > After several days of heavy testing, thread dumps and log file
> analysis
> > with Tibor Digana and various Maven combinations (3.3.9, master,
> > MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are
> >> several
> > serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and
> >> likely
> > Maven Clean Plugin 3.0.0.
> >
> > Since crucial parts of Surefire rely on native code in the JVM
> (forks,
> > streams), our code was not robust enough.
> >
> > As of today we have found:
> >
> > * Missing flushes in streams caused forked VM to be apparently
> > non-responsive
> > * TestNG tests mostly failed due to duplicate contradicting
> properties
> > passed to forked VMs
> > * Uninitialized/too early attributes made daemon threads to kill
> forked
> > VMs
> > * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause
> repeated
> > failures of a handful ITs
> > @Karl Heinz: were you able to figure out something here?
> >
> > Issues in JIRA are pending...
> >
> > Everyone's invited to take a look at the log output as well as the
> > target directory of surefire-integration-tests and contribute:
> > http://home.apache.org/~michaelo/maven/surefire/. The filenames
> should
> > be pretty much self-explanatory.
> >
> > My big question is: how can we improve our test infrastructure? Can
> we
> > raise with INFRA to get at least one FreeBSD and Solaris node for
> > Jenkins? I consider coverage on Windows and Ubuntu way to small, we
> do
> > not even have a CentOS node. Surefire ITs and Maven ITs are
> paramount
> > for us, we should treat them as such!
> >
> > Michael
> >
> 
>  Does any of your findings solve the following already? This is what
>  makes the Jenkins build jobs appear unreliable. It's sending out an
>  email about a failed job and all you see is something like the
> following
>  sporadically. I am having the same issue locally sporadically.
> 
>  [ERROR] Failed to execute goal
>  org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>  (default-test) on project child2: Execution default-test of goal
>  org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed:
> The
>  forked VM terminated without saying properly goodbye. VM crash or
>  System.exit called ? -> [Help 1]
>  org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>  execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>
>  (default-test) on project child2: Execution default-test of goal
>  org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed:
> The
> 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

Socket give you, of course, a lot of control, but they also impose an 
overhead compares to named pipes (mkfifo, CreateNamePipe).


I think our biggest problem is that stdout is subject to buffering 
issues and in case if System#exit() status Z is the buffer, never 
written out.

Surefire thinks that the crashed.

Am 2017-02-19 um 12:32 schrieb Stephen Connolly:

What about using a local host bound socket rather than studio?

In fact a socket could open up other options like forking the tests inside
docker, or running the surefire forks on remote JVMs.

Plus we gain more control (modulo TCP retry/timeouts)

On Sun 19 Feb 2017 at 11:16, Tibor Digana 
wrote:


Currently I placed sleep(1sec) before child process exit(0). If the reader
process ThreadedStreamConsumer would read buffer entirely within 1 sec and
the build pass then I have to add a confirmation mechanism where the exit
event must be confirmed by plugin until short period of time.
Later we should not use these pipes but standard file system, but we have
to take care of SecurityException thrown by JUnit3.

On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov 
wrote:


Am 2017-02-18 um 23:19 schrieb Christian Schulte:


Am 02/17/17 um 21:48 schrieb Michael Osipov:


Hi folks,

Christian Schulte asked me a couple of days ago wether I am able to
build Surefire master with Maven master. It was constantly failing for
him on his OpenBSD machines. Since I have several real servers with
FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able

to

reproduce it. Our entire test infrastructure wasn't unfortunately!

@Tibor: correct me if something is wrong or missing!

After several days of heavy testing, thread dumps and log file analysis
with Tibor Digana and various Maven combinations (3.3.9, master,
MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are

several

serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and

likely

Maven Clean Plugin 3.0.0.

Since crucial parts of Surefire rely on native code in the JVM (forks,
streams), our code was not robust enough.

As of today we have found:

* Missing flushes in streams caused forked VM to be apparently
non-responsive
* TestNG tests mostly failed due to duplicate contradicting properties
passed to forked VMs
* Uninitialized/too early attributes made daemon threads to kill forked
VMs
* Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated
failures of a handful ITs
@Karl Heinz: were you able to figure out something here?

Issues in JIRA are pending...

Everyone's invited to take a look at the log output as well as the
target directory of surefire-integration-tests and contribute:
http://home.apache.org/~michaelo/maven/surefire/. The filenames should
be pretty much self-explanatory.

My big question is: how can we improve our test infrastructure? Can we
raise with INFRA to get at least one FreeBSD and Solaris node for
Jenkins? I consider coverage on Windows and Ubuntu way to small, we do
not even have a CentOS node. Surefire ITs and Maven ITs are paramount
for us, we should treat them as such!

Michael



Does any of your findings solve the following already? This is what
makes the Jenkins build jobs appear unreliable. It's sending out an
email about a failed job and all you see is something like the following
sporadically. I am having the same issue locally sporadically.

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ? -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ?



We were able to solve only a few cases so far: 10%. The rest is still
failing.

I made some digging: it seems to be a very common problem in NodeJS with
async and forking:
https://github.com/nodejs/node/issues/2972
https://github.com/nodejs/node/issues/6456
http://stackoverflow.com/q/11138355/696632
http://stackoverflow.com/q/1716296/696632
https://github.com/nodejs/node-v0.x-archive/issues/8329
https://github.com/nodejs/node-v0.x-archive/issues/1669
https://github.com/nodejs/node-v0.x-archive/issues/3737


Especially this answer is our case:


That's not a bug - process.exit() is an imperative that says "exit

now!".


When stdout/stderr is a pipe (as is the case with child processes) and
said pipe is full, pending data gets lost. Waiting until the pipe drains
is not an option, that could hang the process indefinitely.



I think if we don't gain more control over 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

Too much work for socket, and port can be used by other process.
But We have good news, see IRC
[12:43]  ThreadedStreamConsumer#23504796 run() :: items :: Z,0,BYE!
[12:43]  we received Z
[12:44]  now I have to find why ForkStarter is faster to check the
Z event
[12:45]  because ForkStarter does not see this event for some
reason, it is stored in volatile variable, so maybe timing issue


On Sun, Feb 19, 2017 at 12:32 PM, stephenconnolly [via Maven] <
ml-node+s40175n5899224...@n5.nabble.com> wrote:

> What about using a local host bound socket rather than studio?
>
> In fact a socket could open up other options like forking the tests inside
> docker, or running the surefire forks on remote JVMs.
>
> Plus we gain more control (modulo TCP retry/timeouts)
>
> On Sun 19 Feb 2017 at 11:16, Tibor Digana <[hidden email]
> >
> wrote:
>
> > Currently I placed sleep(1sec) before child process exit(0). If the
> reader
> > process ThreadedStreamConsumer would read buffer entirely within 1 sec
> and
> > the build pass then I have to add a confirmation mechanism where the
> exit
> > event must be confirmed by plugin until short period of time.
> > Later we should not use these pipes but standard file system, but we
> have
> > to take care of SecurityException thrown by JUnit3.
> >
> > On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov <[hidden email]
> >
> > wrote:
> >
> > > Am 2017-02-18 um 23:19 schrieb Christian Schulte:
> > >
> > >> Am 02/17/17 um 21:48 schrieb Michael Osipov:
> > >>
> > >>> Hi folks,
> > >>>
> > >>> Christian Schulte asked me a couple of days ago wether I am able to
> > >>> build Surefire master with Maven master. It was constantly failing
> for
> > >>> him on his OpenBSD machines. Since I have several real servers with
> > >>> FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was
> able
> > to
> > >>> reproduce it. Our entire test infrastructure wasn't unfortunately!
> > >>>
> > >>> @Tibor: correct me if something is wrong or missing!
> > >>>
> > >>> After several days of heavy testing, thread dumps and log file
> analysis
> > >>> with Tibor Digana and various Maven combinations (3.3.9, master,
> > >>> MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are
> > several
> > >>> serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and
> > likely
> > >>> Maven Clean Plugin 3.0.0.
> > >>>
> > >>> Since crucial parts of Surefire rely on native code in the JVM
> (forks,
> > >>> streams), our code was not robust enough.
> > >>>
> > >>> As of today we have found:
> > >>>
> > >>> * Missing flushes in streams caused forked VM to be apparently
> > >>> non-responsive
> > >>> * TestNG tests mostly failed due to duplicate contradicting
> properties
> > >>> passed to forked VMs
> > >>> * Uninitialized/too early attributes made daemon threads to kill
> forked
> > >>> VMs
> > >>> * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause
> repeated
> > >>> failures of a handful ITs
> > >>> @Karl Heinz: were you able to figure out something here?
> > >>>
> > >>> Issues in JIRA are pending...
> > >>>
> > >>> Everyone's invited to take a look at the log output as well as the
> > >>> target directory of surefire-integration-tests and contribute:
> > >>> http://home.apache.org/~michaelo/maven/surefire/. The filenames
> should
> > >>> be pretty much self-explanatory.
> > >>>
> > >>> My big question is: how can we improve our test infrastructure? Can
> we
> > >>> raise with INFRA to get at least one FreeBSD and Solaris node for
> > >>> Jenkins? I consider coverage on Windows and Ubuntu way to small, we
> do
> > >>> not even have a CentOS node. Surefire ITs and Maven ITs are
> paramount
> > >>> for us, we should treat them as such!
> > >>>
> > >>> Michael
> > >>>
> > >>
> > >> Does any of your findings solve the following already? This is what
> > >> makes the Jenkins build jobs appear unreliable. It's sending out an
> > >> email about a failed job and all you see is something like the
> following
> > >> sporadically. I am having the same issue locally sporadically.
> > >>
> > >> [ERROR] Failed to execute goal
> > >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
> > >> (default-test) on project child2: Execution default-test of goal
> > >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed:
> The
> > >> forked VM terminated without saying properly goodbye. VM crash or
> > >> System.exit called ? -> [Help 1]
> > >> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> > >> execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>
> > >> (default-test) on project child2: Execution default-test of goal
> > >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed:
> The
> > >> forked VM terminated without saying properly goodbye. VM crash or
> > >> System.exit called ?
> > >>
> > >
> > > We were able to solve only a few cases so far: 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

What about using a local host bound socket rather than studio?

In fact a socket could open up other options like forking the tests inside
docker, or running the surefire forks on remote JVMs.

Plus we gain more control (modulo TCP retry/timeouts)

On Sun 19 Feb 2017 at 11:16, Tibor Digana 
wrote:

> Currently I placed sleep(1sec) before child process exit(0). If the reader
> process ThreadedStreamConsumer would read buffer entirely within 1 sec and
> the build pass then I have to add a confirmation mechanism where the exit
> event must be confirmed by plugin until short period of time.
> Later we should not use these pipes but standard file system, but we have
> to take care of SecurityException thrown by JUnit3.
>
> On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov 
> wrote:
>
> > Am 2017-02-18 um 23:19 schrieb Christian Schulte:
> >
> >> Am 02/17/17 um 21:48 schrieb Michael Osipov:
> >>
> >>> Hi folks,
> >>>
> >>> Christian Schulte asked me a couple of days ago wether I am able to
> >>> build Surefire master with Maven master. It was constantly failing for
> >>> him on his OpenBSD machines. Since I have several real servers with
> >>> FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able
> to
> >>> reproduce it. Our entire test infrastructure wasn't unfortunately!
> >>>
> >>> @Tibor: correct me if something is wrong or missing!
> >>>
> >>> After several days of heavy testing, thread dumps and log file analysis
> >>> with Tibor Digana and various Maven combinations (3.3.9, master,
> >>> MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are
> several
> >>> serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and
> likely
> >>> Maven Clean Plugin 3.0.0.
> >>>
> >>> Since crucial parts of Surefire rely on native code in the JVM (forks,
> >>> streams), our code was not robust enough.
> >>>
> >>> As of today we have found:
> >>>
> >>> * Missing flushes in streams caused forked VM to be apparently
> >>> non-responsive
> >>> * TestNG tests mostly failed due to duplicate contradicting properties
> >>> passed to forked VMs
> >>> * Uninitialized/too early attributes made daemon threads to kill forked
> >>> VMs
> >>> * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated
> >>> failures of a handful ITs
> >>> @Karl Heinz: were you able to figure out something here?
> >>>
> >>> Issues in JIRA are pending...
> >>>
> >>> Everyone's invited to take a look at the log output as well as the
> >>> target directory of surefire-integration-tests and contribute:
> >>> http://home.apache.org/~michaelo/maven/surefire/. The filenames should
> >>> be pretty much self-explanatory.
> >>>
> >>> My big question is: how can we improve our test infrastructure? Can we
> >>> raise with INFRA to get at least one FreeBSD and Solaris node for
> >>> Jenkins? I consider coverage on Windows and Ubuntu way to small, we do
> >>> not even have a CentOS node. Surefire ITs and Maven ITs are paramount
> >>> for us, we should treat them as such!
> >>>
> >>> Michael
> >>>
> >>
> >> Does any of your findings solve the following already? This is what
> >> makes the Jenkins build jobs appear unreliable. It's sending out an
> >> email about a failed job and all you see is something like the following
> >> sporadically. I am having the same issue locally sporadically.
> >>
> >> [ERROR] Failed to execute goal
> >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
> >> (default-test) on project child2: Execution default-test of goal
> >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
> >> forked VM terminated without saying properly goodbye. VM crash or
> >> System.exit called ? -> [Help 1]
> >> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
> >> execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
> >> (default-test) on project child2: Execution default-test of goal
> >> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
> >> forked VM terminated without saying properly goodbye. VM crash or
> >> System.exit called ?
> >>
> >
> > We were able to solve only a few cases so far: 10%. The rest is still
> > failing.
> >
> > I made some digging: it seems to be a very common problem in NodeJS with
> > async and forking:
> > https://github.com/nodejs/node/issues/2972
> > https://github.com/nodejs/node/issues/6456
> > http://stackoverflow.com/q/11138355/696632
> > http://stackoverflow.com/q/1716296/696632
> > https://github.com/nodejs/node-v0.x-archive/issues/8329
> > https://github.com/nodejs/node-v0.x-archive/issues/1669
> > https://github.com/nodejs/node-v0.x-archive/issues/3737
> >
> >
> > Especially this answer is our case:
> >
> >> That's not a bug - process.exit() is an imperative that says "exit
> now!".
> >>
> >> When stdout/stderr is a pipe (as is the case with child processes) and
> >> said pipe is full, pending data gets lost. Waiting until the pipe drains
> >> is not 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

Currently I placed sleep(1sec) before child process exit(0). If the reader
process ThreadedStreamConsumer would read buffer entirely within 1 sec and
the build pass then I have to add a confirmation mechanism where the exit
event must be confirmed by plugin until short period of time.
Later we should not use these pipes but standard file system, but we have
to take care of SecurityException thrown by JUnit3.

On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov 
wrote:

> Am 2017-02-18 um 23:19 schrieb Christian Schulte:
>
>> Am 02/17/17 um 21:48 schrieb Michael Osipov:
>>
>>> Hi folks,
>>>
>>> Christian Schulte asked me a couple of days ago wether I am able to
>>> build Surefire master with Maven master. It was constantly failing for
>>> him on his OpenBSD machines. Since I have several real servers with
>>> FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able to
>>> reproduce it. Our entire test infrastructure wasn't unfortunately!
>>>
>>> @Tibor: correct me if something is wrong or missing!
>>>
>>> After several days of heavy testing, thread dumps and log file analysis
>>> with Tibor Digana and various Maven combinations (3.3.9, master,
>>> MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are several
>>> serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and likely
>>> Maven Clean Plugin 3.0.0.
>>>
>>> Since crucial parts of Surefire rely on native code in the JVM (forks,
>>> streams), our code was not robust enough.
>>>
>>> As of today we have found:
>>>
>>> * Missing flushes in streams caused forked VM to be apparently
>>> non-responsive
>>> * TestNG tests mostly failed due to duplicate contradicting properties
>>> passed to forked VMs
>>> * Uninitialized/too early attributes made daemon threads to kill forked
>>> VMs
>>> * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated
>>> failures of a handful ITs
>>> @Karl Heinz: were you able to figure out something here?
>>>
>>> Issues in JIRA are pending...
>>>
>>> Everyone's invited to take a look at the log output as well as the
>>> target directory of surefire-integration-tests and contribute:
>>> http://home.apache.org/~michaelo/maven/surefire/. The filenames should
>>> be pretty much self-explanatory.
>>>
>>> My big question is: how can we improve our test infrastructure? Can we
>>> raise with INFRA to get at least one FreeBSD and Solaris node for
>>> Jenkins? I consider coverage on Windows and Ubuntu way to small, we do
>>> not even have a CentOS node. Surefire ITs and Maven ITs are paramount
>>> for us, we should treat them as such!
>>>
>>> Michael
>>>
>>
>> Does any of your findings solve the following already? This is what
>> makes the Jenkins build jobs appear unreliable. It's sending out an
>> email about a failed job and all you see is something like the following
>> sporadically. I am having the same issue locally sporadically.
>>
>> [ERROR] Failed to execute goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>> (default-test) on project child2: Execution default-test of goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
>> forked VM terminated without saying properly goodbye. VM crash or
>> System.exit called ? -> [Help 1]
>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>> execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
>> (default-test) on project child2: Execution default-test of goal
>> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
>> forked VM terminated without saying properly goodbye. VM crash or
>> System.exit called ?
>>
>
> We were able to solve only a few cases so far: 10%. The rest is still
> failing.
>
> I made some digging: it seems to be a very common problem in NodeJS with
> async and forking:
> https://github.com/nodejs/node/issues/2972
> https://github.com/nodejs/node/issues/6456
> http://stackoverflow.com/q/11138355/696632
> http://stackoverflow.com/q/1716296/696632
> https://github.com/nodejs/node-v0.x-archive/issues/8329
> https://github.com/nodejs/node-v0.x-archive/issues/1669
> https://github.com/nodejs/node-v0.x-archive/issues/3737
>
>
> Especially this answer is our case:
>
>> That's not a bug - process.exit() is an imperative that says "exit now!".
>>
>> When stdout/stderr is a pipe (as is the case with child processes) and
>> said pipe is full, pending data gets lost. Waiting until the pipe drains
>> is not an option, that could hang the process indefinitely.
>>
>
> I think if we don't gain more control over stdio, we are lost...
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>


-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

Am 2017-02-18 um 23:19 schrieb Christian Schulte:

Am 02/17/17 um 21:48 schrieb Michael Osipov:

Hi folks,

Christian Schulte asked me a couple of days ago wether I am able to
build Surefire master with Maven master. It was constantly failing for
him on his OpenBSD machines. Since I have several real servers with
FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able to
reproduce it. Our entire test infrastructure wasn't unfortunately!

@Tibor: correct me if something is wrong or missing!

After several days of heavy testing, thread dumps and log file analysis
with Tibor Digana and various Maven combinations (3.3.9, master,
MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are several
serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and likely
Maven Clean Plugin 3.0.0.

Since crucial parts of Surefire rely on native code in the JVM (forks,
streams), our code was not robust enough.

As of today we have found:

* Missing flushes in streams caused forked VM to be apparently
non-responsive
* TestNG tests mostly failed due to duplicate contradicting properties
passed to forked VMs
* Uninitialized/too early attributes made daemon threads to kill forked VMs
* Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated
failures of a handful ITs
@Karl Heinz: were you able to figure out something here?

Issues in JIRA are pending...

Everyone's invited to take a look at the log output as well as the
target directory of surefire-integration-tests and contribute:
http://home.apache.org/~michaelo/maven/surefire/. The filenames should
be pretty much self-explanatory.

My big question is: how can we improve our test infrastructure? Can we
raise with INFRA to get at least one FreeBSD and Solaris node for
Jenkins? I consider coverage on Windows and Ubuntu way to small, we do
not even have a CentOS node. Surefire ITs and Maven ITs are paramount
for us, we should treat them as such!

Michael


Does any of your findings solve the following already? This is what
makes the Jenkins build jobs appear unreliable. It's sending out an
email about a failed job and all you see is something like the following
sporadically. I am having the same issue locally sporadically.

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ? -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ?


We were able to solve only a few cases so far: 10%. The rest is still 
failing.


I made some digging: it seems to be a very common problem in NodeJS with 
async and forking:

https://github.com/nodejs/node/issues/2972
https://github.com/nodejs/node/issues/6456
http://stackoverflow.com/q/11138355/696632
http://stackoverflow.com/q/1716296/696632
https://github.com/nodejs/node-v0.x-archive/issues/8329
https://github.com/nodejs/node-v0.x-archive/issues/1669
https://github.com/nodejs/node-v0.x-archive/issues/3737


Especially this answer is our case:

That's not a bug - process.exit() is an imperative that says "exit now!".

When stdout/stderr is a pipe (as is the case with child processes) and
said pipe is full, pending data gets lost. Waiting until the pipe drains
is not an option, that could hang the process indefinitely.


I think if we don't gain more control over stdio, we are lost...


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Christian Schulte]

Am 02/17/17 um 21:48 schrieb Michael Osipov:
> Hi folks,
> 
> Christian Schulte asked me a couple of days ago wether I am able to 
> build Surefire master with Maven master. It was constantly failing for 
> him on his OpenBSD machines. Since I have several real servers with 
> FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able to 
> reproduce it. Our entire test infrastructure wasn't unfortunately!
> 
> @Tibor: correct me if something is wrong or missing!
> 
> After several days of heavy testing, thread dumps and log file analysis 
> with Tibor Digana and various Maven combinations (3.3.9, master, 
> MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are several 
> serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and likely 
> Maven Clean Plugin 3.0.0.
> 
> Since crucial parts of Surefire rely on native code in the JVM (forks, 
> streams), our code was not robust enough.
> 
> As of today we have found:
> 
> * Missing flushes in streams caused forked VM to be apparently 
> non-responsive
> * TestNG tests mostly failed due to duplicate contradicting properties 
> passed to forked VMs
> * Uninitialized/too early attributes made daemon threads to kill forked VMs
> * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated 
> failures of a handful ITs
> @Karl Heinz: were you able to figure out something here?
> 
> Issues in JIRA are pending...
> 
> Everyone's invited to take a look at the log output as well as the 
> target directory of surefire-integration-tests and contribute:
> http://home.apache.org/~michaelo/maven/surefire/. The filenames should 
> be pretty much self-explanatory.
> 
> My big question is: how can we improve our test infrastructure? Can we 
> raise with INFRA to get at least one FreeBSD and Solaris node for 
> Jenkins? I consider coverage on Windows and Ubuntu way to small, we do 
> not even have a CentOS node. Surefire ITs and Maven ITs are paramount 
> for us, we should treat them as such!
> 
> Michael

Does any of your findings solve the following already? This is what
makes the Jenkins build jobs appear unreliable. It's sending out an
email about a failed job and all you see is something like the following
sporadically. I am having the same issue locally sporadically.

[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ? -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ?

Regards,
-- 
Christian


-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

>>MNG-6169 + local revert of clean gives:
Not sure about your change (MNG-6169 + local revert of clean), but I know
that
The Clean Plugin 3.0.0 caused folder target/surefire-reports to be suddenly
deleted.
No report files found.

>> Yes but is this failing because of a regression in core or surefire?
Regarding
shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT)
No regression or Maven and Surefire.
Strange isn't it?
The principle of this bug is that the POM in surefire-integration-tests
module has property
5.7
So the IT runs Java process with this property, but the problem is that
certain TestNG tests add the same property in CLI with version 5.10. The
MavenLauncher has ArrayList of these CLI goals. Duplicates are
allowed in ArrayList. So the branch SUREFIRE_SYSPROP_DUPLICATES is hotfix
and overrides old system property instead of duplicating them.
Best would be to remove the aforementioned property in POM and amend 58
ITs. The hotfix was faster to do.

Long explanation, but I would not judge Maven or Surefire regression.



On Sat, Feb 18, 2017 at 6:38 PM, stephenconnolly [via Maven] <
ml-node+s40175n5899153...@n5.nabble.com> wrote:

> MNG-6169 + local revert of clean gives:
>
> Failed tests:
> shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT):
>
> log pattern does not match nTimes(..)
>
>
> testSkippedSurefireReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772BothReportsIT):
>
> Expecting failsafe report file
>
>
> testReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772BothReportsIT):
>
> Expecting failsafe report file
>
>
> testOptionalSurefireReportGeneration(org.apache.maven.surefire.its.jiras.
> Surefire772NoSurefireReportsIT):
> Expecting failsafe report file
>
>
> testSkippedSurefireReportGeneration(org.apache.maven.surefire.its.jiras.
> Surefire772NoSurefireReportsIT):
> Expecting failsafe report file
>
>
> testSkipOptionalSurefireReportGeneration(org.apache.maven.
> surefire.its.jiras.Surefire772NoSurefireReportsIT):
> Expecting failsafe report file
>
>
> testReportGeneration(org.apache.maven.surefire.its.jiras.
> Surefire772NoSurefireReportsIT):
> Expecting failsafe report file
>
>
> Tests run: 714, Failures: 7, Errors: 0, Skipped: 133
>
> On 18 February 2017 at 17:36, Stephen Connolly <
> [hidden email] >
> wrote:
>
> > Yes but is this failing because of a regression in core or surefire?
> >
> > On 18 February 2017 at 17:32, Tibor Digana <[hidden email]
> > wrote:
> >
> >> Hi Stephen,
> >>
> >> Both branches should be merged together.
> >> SUREFIRE_SYSPROP_DUPLICATES
> >> SUREFIRE_STDOUT_FLUSH
> >> All tests should pass then.
> >> Cheers
> >> Tibor
> >>
> >> On Sat, Feb 18, 2017 at 6:21 PM, Tibor Digana-2 [via Maven] <
> >> [hidden email] >
> wrote:
> >>
> >> > We had this problem and the branch SUREFIRE_SYSPROP_DUPLICATES should
> >> fix
> >> > it.
> >> >
> >> > On Sat, Feb 18, 2017 at 6:12 PM, Stephen Connolly <
> >> > [hidden email] >
>
> >> > wrote:
> >> >
> >> > > So Maven 3.5.0-SNAPSHOT just has one failure:
> >> > >
> >> > >
> >> > > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed:
> 14.835
> >> > sec
> >> > > <<< FAILURE!
> >> > >
> >> > > shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNg
> >> ReportTestIT)
> >> >
> >> > > Time elapsed: 0 sec  <<< FAILURE!
> >> > >
> >> > > java.lang.AssertionError: log pattern does not match nTimes
> >> > >
> >> > > Expected: is <0>
> >> > >
> >> > >  but: was <1>
> >> > >
> >> > > at org.hamcrest.MatcherAssert.ass
> >> ertThat(MatcherAssert.java:20)
> >> > >
> >> > > at
> >> > > org.apache.maven.surefire.its.fixture.OutputValidator.assert
> >> ThatLogLine(
> >> >
> >> > > OutputValidator.java:104)
> >> > >
> >> > > at
> >> > > org.apache.maven.surefire.its.CheckTestNgReportTestIT.should
> >> NotBeVerbose(
> >> >
> >> > > CheckTestNgReportTestIT.java:55)
> >> > >
> >> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> >> Method)
> >> > >
> >> > > at
> >> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
> >> ssorImpl.java:
> >> >
> >> > > 62)
> >> > >
> >> > > at
> >> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> >> > > DelegatingMethodAccessorImpl.java:43)
> >> > >
> >> > > at java.lang.reflect.Method.invoke(Method.java:498)
> >> > >
> >> > > at
> >> > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> >> > > FrameworkMethod.java:50)
> >> > >
> >> > > at
> >> > > org.junit.internal.runners.model.ReflectiveCallable.run(
> >> > > ReflectiveCallable.java:12)
> >> > >
> >> > > at
> >> > > org.junit.runners.model.FrameworkMethod.invokeExplosively(
> >> > > FrameworkMethod.java:47)
> >> > >
> >> > > at
> >> > > 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

MNG-6169 + local revert of clean gives:

Failed tests:
shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT):
log pattern does not match nTimes(..)


testSkippedSurefireReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772BothReportsIT):
Expecting failsafe report file


testReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772BothReportsIT):
Expecting failsafe report file


testOptionalSurefireReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772NoSurefireReportsIT):
Expecting failsafe report file


testSkippedSurefireReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772NoSurefireReportsIT):
Expecting failsafe report file


testSkipOptionalSurefireReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772NoSurefireReportsIT):
Expecting failsafe report file


testReportGeneration(org.apache.maven.surefire.its.jiras.Surefire772NoSurefireReportsIT):
Expecting failsafe report file


Tests run: 714, Failures: 7, Errors: 0, Skipped: 133

On 18 February 2017 at 17:36, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> Yes but is this failing because of a regression in core or surefire?
>
> On 18 February 2017 at 17:32, Tibor Digana  wrote:
>
>> Hi Stephen,
>>
>> Both branches should be merged together.
>> SUREFIRE_SYSPROP_DUPLICATES
>> SUREFIRE_STDOUT_FLUSH
>> All tests should pass then.
>> Cheers
>> Tibor
>>
>> On Sat, Feb 18, 2017 at 6:21 PM, Tibor Digana-2 [via Maven] <
>> ml-node+s40175n5899136...@n5.nabble.com> wrote:
>>
>> > We had this problem and the branch SUREFIRE_SYSPROP_DUPLICATES should
>> fix
>> > it.
>> >
>> > On Sat, Feb 18, 2017 at 6:12 PM, Stephen Connolly <
>> > [hidden email] >
>> > wrote:
>> >
>> > > So Maven 3.5.0-SNAPSHOT just has one failure:
>> > >
>> > >
>> > > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.835
>> > sec
>> > > <<< FAILURE!
>> > >
>> > > shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNg
>> ReportTestIT)
>> >
>> > > Time elapsed: 0 sec  <<< FAILURE!
>> > >
>> > > java.lang.AssertionError: log pattern does not match nTimes
>> > >
>> > > Expected: is <0>
>> > >
>> > >  but: was <1>
>> > >
>> > > at org.hamcrest.MatcherAssert.ass
>> ertThat(MatcherAssert.java:20)
>> > >
>> > > at
>> > > org.apache.maven.surefire.its.fixture.OutputValidator.assert
>> ThatLogLine(
>> >
>> > > OutputValidator.java:104)
>> > >
>> > > at
>> > > org.apache.maven.surefire.its.CheckTestNgReportTestIT.should
>> NotBeVerbose(
>> >
>> > > CheckTestNgReportTestIT.java:55)
>> > >
>> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>> Method)
>> > >
>> > > at
>> > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
>> ssorImpl.java:
>> >
>> > > 62)
>> > >
>> > > at
>> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> > > DelegatingMethodAccessorImpl.java:43)
>> > >
>> > > at java.lang.reflect.Method.invoke(Method.java:498)
>> > >
>> > > at
>> > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
>> > > FrameworkMethod.java:50)
>> > >
>> > > at
>> > > org.junit.internal.runners.model.ReflectiveCallable.run(
>> > > ReflectiveCallable.java:12)
>> > >
>> > > at
>> > > org.junit.runners.model.FrameworkMethod.invokeExplosively(
>> > > FrameworkMethod.java:47)
>> > >
>> > > at
>> > > org.junit.internal.runners.statements.InvokeMethod.
>> > > evaluate(InvokeMethod.java:17)
>> > >
>> > > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:
>> 325)
>> >
>> > >
>> > > at
>> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(
>> > > BlockJUnit4ClassRunner.java:78)
>> > >
>> > > at
>> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(
>> > > BlockJUnit4ClassRunner.java:57)
>> > >
>> > > at org.junit.runners.ParentRunner
>> $3.run(ParentRunner.java:290)
>> > >
>> > > at org.junit.runners.ParentRunner
>> $1.schedule(ParentRunner.java:71)
>> >
>> > >
>> > > at org.junit.runners.ParentRunner.runChildren(
>> > > ParentRunner.java:288)
>> > >
>> > > at org.junit.runners.ParentRunner
>> .access$000(ParentRunner.java:58)
>> >
>> > >
>> > > at org.junit.runners.ParentRunner$2.evaluate(
>> > > ParentRunner.java:268)
>> > >
>> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>> > >
>> > > at org.junit.runners.Suite.runChild(Suite.java:128)
>> > >
>> > > at org.junit.runners.Suite.runChild(Suite.java:27)
>> > >
>> > > at org.junit.runners.ParentRunner
>> $3.run(ParentRunner.java:290)
>> > >
>> > > at org.junit.runners.ParentRunner
>> $1.schedule(ParentRunner.java:71)
>> >
>> > >
>> > > at org.junit.runners.ParentRunner.runChildren(
>> > > ParentRunner.java:288)
>> > >
>> > > at org.junit.runners.ParentRunner
>> .access$000(ParentRunner.java:58)
>> >
>> > >
>> > > at 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

Yes but is this failing because of a regression in core or surefire?

On 18 February 2017 at 17:32, Tibor Digana  wrote:

> Hi Stephen,
>
> Both branches should be merged together.
> SUREFIRE_SYSPROP_DUPLICATES
> SUREFIRE_STDOUT_FLUSH
> All tests should pass then.
> Cheers
> Tibor
>
> On Sat, Feb 18, 2017 at 6:21 PM, Tibor Digana-2 [via Maven] <
> ml-node+s40175n5899136...@n5.nabble.com> wrote:
>
> > We had this problem and the branch SUREFIRE_SYSPROP_DUPLICATES should fix
> > it.
> >
> > On Sat, Feb 18, 2017 at 6:12 PM, Stephen Connolly <
> > [hidden email] >
> > wrote:
> >
> > > So Maven 3.5.0-SNAPSHOT just has one failure:
> > >
> > >
> > > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.835
> > sec
> > > <<< FAILURE!
> > >
> > > shouldNotBeVerbose(org.apache.maven.surefire.its.
> CheckTestNgReportTestIT)
> >
> > > Time elapsed: 0 sec  <<< FAILURE!
> > >
> > > java.lang.AssertionError: log pattern does not match nTimes
> > >
> > > Expected: is <0>
> > >
> > >  but: was <1>
> > >
> > > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:
> 20)
> > >
> > > at
> > > org.apache.maven.surefire.its.fixture.OutputValidator.
> assertThatLogLine(
> >
> > > OutputValidator.java:104)
> > >
> > > at
> > > org.apache.maven.surefire.its.CheckTestNgReportTestIT.
> shouldNotBeVerbose(
> >
> > > CheckTestNgReportTestIT.java:55)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:
> >
> > > 62)
> > >
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > > DelegatingMethodAccessorImpl.java:43)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > >
> > > at
> > > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> > > FrameworkMethod.java:50)
> > >
> > > at
> > > org.junit.internal.runners.model.ReflectiveCallable.run(
> > > ReflectiveCallable.java:12)
> > >
> > > at
> > > org.junit.runners.model.FrameworkMethod.invokeExplosively(
> > > FrameworkMethod.java:47)
> > >
> > > at
> > > org.junit.internal.runners.statements.InvokeMethod.
> > > evaluate(InvokeMethod.java:17)
> > >
> > > at org.junit.runners.ParentRunner.runLeaf(
> ParentRunner.java:325)
> >
> > >
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> > > BlockJUnit4ClassRunner.java:78)
> > >
> > > at
> > > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> > > BlockJUnit4ClassRunner.java:57)
> > >
> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > >
> > > at org.junit.runners.ParentRunner$1.schedule(
> ParentRunner.java:71)
> >
> > >
> > > at org.junit.runners.ParentRunner.runChildren(
> > > ParentRunner.java:288)
> > >
> > > at org.junit.runners.ParentRunner.access$000(
> ParentRunner.java:58)
> >
> > >
> > > at org.junit.runners.ParentRunner$2.evaluate(
> > > ParentRunner.java:268)
> > >
> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > >
> > > at org.junit.runners.Suite.runChild(Suite.java:128)
> > >
> > > at org.junit.runners.Suite.runChild(Suite.java:27)
> > >
> > > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> > >
> > > at org.junit.runners.ParentRunner$1.schedule(
> ParentRunner.java:71)
> >
> > >
> > > at org.junit.runners.ParentRunner.runChildren(
> > > ParentRunner.java:288)
> > >
> > > at org.junit.runners.ParentRunner.access$000(
> ParentRunner.java:58)
> >
> > >
> > > at org.junit.runners.ParentRunner$2.evaluate(
> > > ParentRunner.java:268)
> > >
> > > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> > >
> > > at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> > >
> > > at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
> > >
> > > at
> > > org.apache.maven.surefire.junitcore.JUnitCoreWrapper.
> > > execute(JUnitCoreWrapper.java:62)
> > >
> > > at
> > > org.apache.maven.surefire.junitcore.JUnitCoreProvider.
> > > invoke(JUnitCoreProvider.java:139)
> > >
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > >
> > > at
> > > sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:
> >
> > > 62)
> > >
> > > at
> > > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > > DelegatingMethodAccessorImpl.java:43)
> > >
> > > at java.lang.reflect.Method.invoke(Method.java:498)
> > >
> > > at
> > > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(
> > > ReflectionUtils.java:189)
> > >
> > > at
> > > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> > > ProviderFactory.java:165)
> > >
> > > at
> > > 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

Hi Stephen,

Both branches should be merged together.
SUREFIRE_SYSPROP_DUPLICATES
SUREFIRE_STDOUT_FLUSH
All tests should pass then.
Cheers
Tibor

On Sat, Feb 18, 2017 at 6:21 PM, Tibor Digana-2 [via Maven] <
ml-node+s40175n5899136...@n5.nabble.com> wrote:

> We had this problem and the branch SUREFIRE_SYSPROP_DUPLICATES should fix
> it.
>
> On Sat, Feb 18, 2017 at 6:12 PM, Stephen Connolly <
> [hidden email] >
> wrote:
>
> > So Maven 3.5.0-SNAPSHOT just has one failure:
> >
> >
> > Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.835
> sec
> > <<< FAILURE!
> >
> > shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT)
>
> > Time elapsed: 0 sec  <<< FAILURE!
> >
> > java.lang.AssertionError: log pattern does not match nTimes
> >
> > Expected: is <0>
> >
> >  but: was <1>
> >
> > at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
> >
> > at
> > org.apache.maven.surefire.its.fixture.OutputValidator.assertThatLogLine(
>
> > OutputValidator.java:104)
> >
> > at
> > org.apache.maven.surefire.its.CheckTestNgReportTestIT.shouldNotBeVerbose(
>
> > CheckTestNgReportTestIT.java:55)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
>
> > 62)
> >
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:498)
> >
> > at
> > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> > FrameworkMethod.java:50)
> >
> > at
> > org.junit.internal.runners.model.ReflectiveCallable.run(
> > ReflectiveCallable.java:12)
> >
> > at
> > org.junit.runners.model.FrameworkMethod.invokeExplosively(
> > FrameworkMethod.java:47)
> >
> > at
> > org.junit.internal.runners.statements.InvokeMethod.
> > evaluate(InvokeMethod.java:17)
> >
> > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>
> >
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> > BlockJUnit4ClassRunner.java:78)
> >
> > at
> > org.junit.runners.BlockJUnit4ClassRunner.runChild(
> > BlockJUnit4ClassRunner.java:57)
> >
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> >
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> >
> > at org.junit.runners.ParentRunner.runChildren(
> > ParentRunner.java:288)
> >
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> >
> > at org.junit.runners.ParentRunner$2.evaluate(
> > ParentRunner.java:268)
> >
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> >
> > at org.junit.runners.Suite.runChild(Suite.java:128)
> >
> > at org.junit.runners.Suite.runChild(Suite.java:27)
> >
> > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
> >
> > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> >
> > at org.junit.runners.ParentRunner.runChildren(
> > ParentRunner.java:288)
> >
> > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> >
> > at org.junit.runners.ParentRunner$2.evaluate(
> > ParentRunner.java:268)
> >
> > at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
> >
> > at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
> >
> > at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
> >
> > at
> > org.apache.maven.surefire.junitcore.JUnitCoreWrapper.
> > execute(JUnitCoreWrapper.java:62)
> >
> > at
> > org.apache.maven.surefire.junitcore.JUnitCoreProvider.
> > invoke(JUnitCoreProvider.java:139)
> >
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >
> > at
> > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
>
> > 62)
> >
> > at
> > sun.reflect.DelegatingMethodAccessorImpl.invoke(
> > DelegatingMethodAccessorImpl.java:43)
> >
> > at java.lang.reflect.Method.invoke(Method.java:498)
> >
> > at
> > org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(
> > ReflectionUtils.java:189)
> >
> > at
> > org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> > ProviderFactory.java:165)
> >
> > at
> > org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(
> > ProviderFactory.java:85)
> >
> > at
> > org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.
> > runSuitesInProcess(InPluginVMSurefireStarter.java:80)
> >
> > at
> > org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(
> > AbstractSurefireMojo.java:724)
> >
> > at
> > org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAllProviders(
>
> > AbstractSurefireMojo.java:682)
> >
> > 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

We had this problem and the branch SUREFIRE_SYSPROP_DUPLICATES should fix
it.

On Sat, Feb 18, 2017 at 6:12 PM, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> So Maven 3.5.0-SNAPSHOT just has one failure:
>
>
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.835 sec
> <<< FAILURE!
>
> shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT)
> Time elapsed: 0 sec  <<< FAILURE!
>
> java.lang.AssertionError: log pattern does not match nTimes
>
> Expected: is <0>
>
>  but: was <1>
>
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>
> at
> org.apache.maven.surefire.its.fixture.OutputValidator.assertThatLogLine(
> OutputValidator.java:104)
>
> at
> org.apache.maven.surefire.its.CheckTestNgReportTestIT.shouldNotBeVerbose(
> CheckTestNgReportTestIT.java:55)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 62)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:50)
>
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(
> ReflectiveCallable.java:12)
>
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(
> FrameworkMethod.java:47)
>
> at
> org.junit.internal.runners.statements.InvokeMethod.
> evaluate(InvokeMethod.java:17)
>
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:78)
>
> at
> org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:57)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> at org.junit.runners.ParentRunner.runChildren(
> ParentRunner.java:288)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> at org.junit.runners.ParentRunner$2.evaluate(
> ParentRunner.java:268)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>
> at org.junit.runners.Suite.runChild(Suite.java:128)
>
> at org.junit.runners.Suite.runChild(Suite.java:27)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> at org.junit.runners.ParentRunner.runChildren(
> ParentRunner.java:288)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> at org.junit.runners.ParentRunner$2.evaluate(
> ParentRunner.java:268)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
>
> at
> org.apache.maven.surefire.junitcore.JUnitCoreWrapper.
> execute(JUnitCoreWrapper.java:62)
>
> at
> org.apache.maven.surefire.junitcore.JUnitCoreProvider.
> invoke(JUnitCoreProvider.java:139)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:
> 62)
>
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at
> org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(
> ReflectionUtils.java:189)
>
> at
> org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(
> ProviderFactory.java:165)
>
> at
> org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(
> ProviderFactory.java:85)
>
> at
> org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.
> runSuitesInProcess(InPluginVMSurefireStarter.java:80)
>
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(
> AbstractSurefireMojo.java:724)
>
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAllProviders(
> AbstractSurefireMojo.java:682)
>
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.
> executeAfterPreconditionsChecked(AbstractSurefireMojo.java:648)
>
> at
> org.apache.maven.plugin.surefire.AbstractSurefireMojo.
> execute(AbstractSurefireMojo.java:586)
>
> at
> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:134)
>
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:207)
>
> at
> org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:153)
>
> at
> 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

Same for 3.5.0-SNAPSHOT + MNG-6169 on OS-X

Results :


Failed tests:
shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT):
log pattern does not match nTimes(..)


Tests run: 714, Failures: 1, Errors: 0, Skipped: 133



On 18 February 2017 at 17:12, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> So Maven 3.5.0-SNAPSHOT just has one failure:
>
>
> Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.835 sec
> <<< FAILURE!
>
> shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT)
> Time elapsed: 0 sec  <<< FAILURE!
>
> java.lang.AssertionError: log pattern does not match nTimes
>
> Expected: is <0>
>
>  but: was <1>
>
> at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
>
> at org.apache.maven.surefire.its.fixture.OutputValidator.
> assertThatLogLine(OutputValidator.java:104)
>
> at org.apache.maven.surefire.its.CheckTestNgReportTestIT.
> shouldNotBeVerbose(CheckTestNgReportTestIT.java:55)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
> FrameworkMethod.java:50)
>
> at org.junit.internal.runners.model.ReflectiveCallable.run(
> ReflectiveCallable.java:12)
>
> at org.junit.runners.model.FrameworkMethod.invokeExplosively(
> FrameworkMethod.java:47)
>
> at org.junit.internal.runners.statements.InvokeMethod.
> evaluate(InvokeMethod.java:17)
>
> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:78)
>
> at org.junit.runners.BlockJUnit4ClassRunner.runChild(
> BlockJUnit4ClassRunner.java:57)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> at org.junit.runners.ParentRunner.runChildren(
> ParentRunner.java:288)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> at org.junit.runners.ParentRunner$2.evaluate(
> ParentRunner.java:268)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>
> at org.junit.runners.Suite.runChild(Suite.java:128)
>
> at org.junit.runners.Suite.runChild(Suite.java:27)
>
> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
>
> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
>
> at org.junit.runners.ParentRunner.runChildren(
> ParentRunner.java:288)
>
> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
>
> at org.junit.runners.ParentRunner$2.evaluate(
> ParentRunner.java:268)
>
> at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
>
> at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
>
> at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.
> execute(JUnitCoreWrapper.java:62)
>
> at org.apache.maven.surefire.junitcore.JUnitCoreProvider.
> invoke(JUnitCoreProvider.java:139)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
>
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>
> at java.lang.reflect.Method.invoke(Method.java:498)
>
> at org.apache.maven.surefire.util.ReflectionUtils.
> invokeMethodWithArray(ReflectionUtils.java:189)
>
> at org.apache.maven.surefire.booter.ProviderFactory$
> ProviderProxy.invoke(ProviderFactory.java:165)
>
> at org.apache.maven.surefire.booter.ProviderFactory.
> invokeProvider(ProviderFactory.java:85)
>
> at org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.
> runSuitesInProcess(InPluginVMSurefireStarter.java:80)
>
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.
> executeProvider(AbstractSurefireMojo.java:724)
>
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.
> executeAllProviders(AbstractSurefireMojo.java:682)
>
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.
> executeAfterPreconditionsChecked(AbstractSurefireMojo.java:648)
>
> at org.apache.maven.plugin.surefire.AbstractSurefireMojo.
> execute(AbstractSurefireMojo.java:586)
>
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(
> DefaultBuildPluginManager.java:134)
>
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute(
> MojoExecutor.java:207)
>
> at 

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

So Maven 3.5.0-SNAPSHOT just has one failure:


Tests run: 3, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.835 sec
<<< FAILURE!

shouldNotBeVerbose(org.apache.maven.surefire.its.CheckTestNgReportTestIT)
Time elapsed: 0 sec  <<< FAILURE!

java.lang.AssertionError: log pattern does not match nTimes

Expected: is <0>

 but: was <1>

at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)

at
org.apache.maven.surefire.its.fixture.OutputValidator.assertThatLogLine(OutputValidator.java:104)

at
org.apache.maven.surefire.its.CheckTestNgReportTestIT.shouldNotBeVerbose(CheckTestNgReportTestIT.java:55)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)

at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)

at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)

at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)

at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)

at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)

at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)

at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)

at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)

at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)

at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)

at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)

at org.junit.runners.ParentRunner.run(ParentRunner.java:363)

at org.junit.runners.Suite.runChild(Suite.java:128)

at org.junit.runners.Suite.runChild(Suite.java:27)

at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)

at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)

at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)

at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)

at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)

at org.junit.runners.ParentRunner.run(ParentRunner.java:363)

at org.junit.runner.JUnitCore.run(JUnitCore.java:137)

at org.junit.runner.JUnitCore.run(JUnitCore.java:115)

at
org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:62)

at
org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:139)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:498)

at
org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)

at
org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)

at
org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)

at
org.apache.maven.plugin.surefire.InPluginVMSurefireStarter.runSuitesInProcess(InPluginVMSurefireStarter.java:80)

at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeProvider(AbstractSurefireMojo.java:724)

at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAllProviders(AbstractSurefireMojo.java:682)

at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.executeAfterPreconditionsChecked(AbstractSurefireMojo.java:648)

at
org.apache.maven.plugin.surefire.AbstractSurefireMojo.execute(AbstractSurefireMojo.java:586)

at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)

at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)

at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

at

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

I have downloaded Michaels log for Maven 3.3.9 & Surefire 2.19.2-SNAPSHOT
http://home.apache.org/~michaelo/maven/surefire/maven-surefire-extended-patch-maven-3.3.9.tar.gz
and 27 tests failed.


On Sat, Feb 18, 2017 at 4:52 PM, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> So results so far: surefire master branch builds with 3.3.9 but fails with
> 3.5.0-SNAPSHOT when running on macOS Sierra with Oracle Java 1.8.0_121
>
> That is certainly looking like a regression in Maven core
>
> On 18 February 2017 at 10:55, Michael Osipov  wrote:
>
> > For those who would like to reproduce the issues:
> >
> > 1. Build Maven distribution from 3.3.9, master, MNG-6169 and MNG-6169
> with
> > locally modified maven-core/src/main/resources/
> META-INF/plexus/components.xml
> > maven-clean-plugin 2.6.1 => 3.0.0.
> > You should have four distros now.
> > 2. Clone Surefire
> > 3. Run all four versions of Maven against Surefire master:
> > > ~/apache-maven-$version/bin/mvn -B -V clean install
> > -Dinvoker.mergeSettings | tee ~/maven-surefire.log
> > Leave mergeSettings out if you have direct Internet connection
> > 4. Save maven-surefire.log and surefire-integration-tests/target for
> > later analysis
> > 5. Branch off Surefire master and apply branches:
> > origin/SUREFIRE_STDOUT_FLUSH
> > origin/SUREFIRE_SYSPROP_DUPLICATES
> > 6. Repeat step 3
> > 7. Checkout maven-shared-utils-0.9.x from Subversion, install and update
> > maven-surefire/pom.xml with version 0.9.1-SNAPSHOT
> > 8. Repeat 6
> > 7. Compare results of 3, 5, 6, 8 as well as with my maven-surefire.log
> > files
> >
> > note: my logs are based on an already patched version (from mentioned
> > branches) of Surefire master.
> >
> >
> > Michael
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
>



-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

>>That is certainly looking like a regression in Maven core
But the principle of the bug on FreeBSD should fail the build with Maven
3.3.9 as well. This I do not understand why it is ok with 3.3.9.

On Sat, Feb 18, 2017 at 4:55 PM, Tibor Digana 
wrote:

> ok, let's try with two surefire branches merged together.
>
> On Sat, Feb 18, 2017 at 4:52 PM, Stephen Connolly <
> stephen.alan.conno...@gmail.com> wrote:
>
>> So results so far: surefire master branch builds with 3.3.9 but fails with
>> 3.5.0-SNAPSHOT when running on macOS Sierra with Oracle Java 1.8.0_121
>>
>> That is certainly looking like a regression in Maven core
>>
>> On 18 February 2017 at 10:55, Michael Osipov  wrote:
>>
>> > For those who would like to reproduce the issues:
>> >
>> > 1. Build Maven distribution from 3.3.9, master, MNG-6169 and MNG-6169
>> with
>> > locally modified maven-core/src/main/resources/
>> META-INF/plexus/components.xml
>> > maven-clean-plugin 2.6.1 => 3.0.0.
>> > You should have four distros now.
>> > 2. Clone Surefire
>> > 3. Run all four versions of Maven against Surefire master:
>> > > ~/apache-maven-$version/bin/mvn -B -V clean install
>> > -Dinvoker.mergeSettings | tee ~/maven-surefire.log
>> > Leave mergeSettings out if you have direct Internet connection
>> > 4. Save maven-surefire.log and surefire-integration-tests/target for
>> > later analysis
>> > 5. Branch off Surefire master and apply branches:
>> > origin/SUREFIRE_STDOUT_FLUSH
>> > origin/SUREFIRE_SYSPROP_DUPLICATES
>> > 6. Repeat step 3
>> > 7. Checkout maven-shared-utils-0.9.x from Subversion, install and update
>> > maven-surefire/pom.xml with version 0.9.1-SNAPSHOT
>> > 8. Repeat 6
>> > 7. Compare results of 3, 5, 6, 8 as well as with my maven-surefire.log
>> > files
>> >
>> > note: my logs are based on an already patched version (from mentioned
>> > branches) of Surefire master.
>> >
>> >
>> > Michael
>> >
>> > -
>> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
>> > For additional commands, e-mail: dev-h...@maven.apache.org
>> >
>> >
>>
>
>
>
> --
> Cheers
> Tibor
>



-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Tibor Digana]

ok, let's try with two surefire branches merged together.

On Sat, Feb 18, 2017 at 4:52 PM, Stephen Connolly <
stephen.alan.conno...@gmail.com> wrote:

> So results so far: surefire master branch builds with 3.3.9 but fails with
> 3.5.0-SNAPSHOT when running on macOS Sierra with Oracle Java 1.8.0_121
>
> That is certainly looking like a regression in Maven core
>
> On 18 February 2017 at 10:55, Michael Osipov  wrote:
>
> > For those who would like to reproduce the issues:
> >
> > 1. Build Maven distribution from 3.3.9, master, MNG-6169 and MNG-6169
> with
> > locally modified maven-core/src/main/resources/
> META-INF/plexus/components.xml
> > maven-clean-plugin 2.6.1 => 3.0.0.
> > You should have four distros now.
> > 2. Clone Surefire
> > 3. Run all four versions of Maven against Surefire master:
> > > ~/apache-maven-$version/bin/mvn -B -V clean install
> > -Dinvoker.mergeSettings | tee ~/maven-surefire.log
> > Leave mergeSettings out if you have direct Internet connection
> > 4. Save maven-surefire.log and surefire-integration-tests/target for
> > later analysis
> > 5. Branch off Surefire master and apply branches:
> > origin/SUREFIRE_STDOUT_FLUSH
> > origin/SUREFIRE_SYSPROP_DUPLICATES
> > 6. Repeat step 3
> > 7. Checkout maven-shared-utils-0.9.x from Subversion, install and update
> > maven-surefire/pom.xml with version 0.9.1-SNAPSHOT
> > 8. Repeat 6
> > 7. Compare results of 3, 5, 6, 8 as well as with my maven-surefire.log
> > files
> >
> > note: my logs are based on an already patched version (from mentioned
> > branches) of Surefire master.
> >
> >
> > Michael
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> > For additional commands, e-mail: dev-h...@maven.apache.org
> >
> >
>



-- 
Cheers
Tibor

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Stephen Connolly]

So results so far: surefire master branch builds with 3.3.9 but fails with
3.5.0-SNAPSHOT when running on macOS Sierra with Oracle Java 1.8.0_121

That is certainly looking like a regression in Maven core

On 18 February 2017 at 10:55, Michael Osipov  wrote:

> For those who would like to reproduce the issues:
>
> 1. Build Maven distribution from 3.3.9, master, MNG-6169 and MNG-6169 with
> locally modified maven-core/src/main/resources/META-INF/plexus/components.xml
> maven-clean-plugin 2.6.1 => 3.0.0.
> You should have four distros now.
> 2. Clone Surefire
> 3. Run all four versions of Maven against Surefire master:
> > ~/apache-maven-$version/bin/mvn -B -V clean install
> -Dinvoker.mergeSettings | tee ~/maven-surefire.log
> Leave mergeSettings out if you have direct Internet connection
> 4. Save maven-surefire.log and surefire-integration-tests/target for
> later analysis
> 5. Branch off Surefire master and apply branches:
> origin/SUREFIRE_STDOUT_FLUSH
> origin/SUREFIRE_SYSPROP_DUPLICATES
> 6. Repeat step 3
> 7. Checkout maven-shared-utils-0.9.x from Subversion, install and update
> maven-surefire/pom.xml with version 0.9.1-SNAPSHOT
> 8. Repeat 6
> 7. Compare results of 3, 5, 6, 8 as well as with my maven-surefire.log
> files
>
> note: my logs are based on an already patched version (from mentioned
> branches) of Surefire master.
>
>
> Michael
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
> For additional commands, e-mail: dev-h...@maven.apache.org
>
>

Re: Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

For those who would like to reproduce the issues:

1. Build Maven distribution from 3.3.9, master, MNG-6169 and MNG-6169 
with locally modified 
maven-core/src/main/resources/META-INF/plexus/components.xml 
maven-clean-plugin 2.6.1 => 3.0.0.

You should have four distros now.
2. Clone Surefire
3. Run all four versions of Maven against Surefire master:
> ~/apache-maven-$version/bin/mvn -B -V clean install 
-Dinvoker.mergeSettings | tee ~/maven-surefire.log

Leave mergeSettings out if you have direct Internet connection
4. Save maven-surefire.log and surefire-integration-tests/target for 
later analysis

5. Branch off Surefire master and apply branches:
origin/SUREFIRE_STDOUT_FLUSH
origin/SUREFIRE_SYSPROP_DUPLICATES
6. Repeat step 3
7. Checkout maven-shared-utils-0.9.x from Subversion, install and update 
maven-surefire/pom.xml with version 0.9.1-SNAPSHOT

8. Repeat 6
7. Compare results of 3, 5, 6, 8 as well as with my maven-surefire.log files

note: my logs are based on an already patched version (from mentioned 
branches) of Surefire master.


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org

Recent issues found in Surefire master/Maven Clean Plugin 3.0.0 with Maven master

[Michael Osipov]

Hi folks,

Christian Schulte asked me a couple of days ago wether I am able to 
build Surefire master with Maven master. It was constantly failing for 
him on his OpenBSD machines. Since I have several real servers with 
FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able to 
reproduce it. Our entire test infrastructure wasn't unfortunately!


@Tibor: correct me if something is wrong or missing!

After several days of heavy testing, thread dumps and log file analysis 
with Tibor Digana and various Maven combinations (3.3.9, master, 
MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are several 
serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and likely 
Maven Clean Plugin 3.0.0.


Since crucial parts of Surefire rely on native code in the JVM (forks, 
streams), our code was not robust enough.


As of today we have found:

* Missing flushes in streams caused forked VM to be apparently 
non-responsive
* TestNG tests mostly failed due to duplicate contradicting properties 
passed to forked VMs

* Uninitialized/too early attributes made daemon threads to kill forked VMs
* Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated 
failures of a handful ITs

@Karl Heinz: were you able to figure out something here?

Issues in JIRA are pending...

Everyone's invited to take a look at the log output as well as the 
target directory of surefire-integration-tests and contribute:
http://home.apache.org/~michaelo/maven/surefire/. The filenames should 
be pretty much self-explanatory.


My big question is: how can we improve our test infrastructure? Can we 
raise with INFRA to get at least one FreeBSD and Solaris node for 
Jenkins? I consider coverage on Windows and Ubuntu way to small, we do 
not even have a CentOS node. Surefire ITs and Maven ITs are paramount 
for us, we should treat them as such!


Michael

-
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org