Am 2017-02-18 um 23:19 schrieb Christian Schulte:
Am 02/17/17 um 21:48 schrieb Michael Osipov:
Hi folks,
Christian Schulte asked me a couple of days ago wether I am able to
build Surefire master with Maven master. It was constantly failing for
him on his OpenBSD machines. Since I have several real servers with
FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able to
reproduce it. Our entire test infrastructure wasn't unfortunately!
@Tibor: correct me if something is wrong or missing!
After several days of heavy testing, thread dumps and log file analysis
with Tibor Digana and various Maven combinations (3.3.9, master,
MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are several
serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and likely
Maven Clean Plugin 3.0.0.
Since crucial parts of Surefire rely on native code in the JVM (forks,
streams), our code was not robust enough.
As of today we have found:
* Missing flushes in streams caused forked VM to be apparently
non-responsive
* TestNG tests mostly failed due to duplicate contradicting properties
passed to forked VMs
* Uninitialized/too early attributes made daemon threads to kill forked VMs
* Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated
failures of a handful ITs
@Karl Heinz: were you able to figure out something here?
Issues in JIRA are pending...
Everyone's invited to take a look at the log output as well as the
target directory of surefire-integration-tests and contribute:
http://home.apache.org/~michaelo/maven/surefire/. The filenames should
be pretty much self-explanatory.
My big question is: how can we improve our test infrastructure? Can we
raise with INFRA to get at least one FreeBSD and Solaris node for
Jenkins? I consider coverage on Windows and Ubuntu way to small, we do
not even have a CentOS node. Surefire ITs and Maven ITs are paramount
for us, we should treat them as such!
Michael
Does any of your findings solve the following already? This is what
makes the Jenkins build jobs appear unreliable. It's sending out an
email about a failed job and all you see is something like the following
sporadically. I am having the same issue locally sporadically.
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ? -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
execute goal org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test
(default-test) on project child2: Execution default-test of goal
org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The
forked VM terminated without saying properly goodbye. VM crash or
System.exit called ?
We were able to solve only a few cases so far: 10%. The rest is still
failing.
I made some digging: it seems to be a very common problem in NodeJS with
async and forking:
https://github.com/nodejs/node/issues/2972
https://github.com/nodejs/node/issues/6456
http://stackoverflow.com/q/11138355/696632
http://stackoverflow.com/q/1716296/696632
https://github.com/nodejs/node-v0.x-archive/issues/8329
https://github.com/nodejs/node-v0.x-archive/issues/1669
https://github.com/nodejs/node-v0.x-archive/issues/3737
Especially this answer is our case:
That's not a bug - process.exit() is an imperative that says "exit now!".
When stdout/stderr is a pipe (as is the case with child processes) and
said pipe is full, pending data gets lost. Waiting until the pipe drains
is not an option, that could hang the process indefinitely.
I think if we don't gain more control over stdio, we are lost...
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org
For additional commands, e-mail: dev-h...@maven.apache.org
