On Sun 19 Feb 2017 at 12:06, Michael Osipov <micha...@apache.org> wrote:
> Socket give you, of course, a lot of control, but they also impose an > overhead compares to named pipes (mkfifo, CreateNamePipe). Yep but windows restricts choice of technology. Docker for Windows (as in where Windows is the guest (and consequently host) OS) doubly so. Sockets allow forking to be remote (although SSH as a remoting engine would give back stdio it can also give port forwarding) > > > I think our biggest problem is that stdout is subject to buffering > issues and in case if System#exit() status Z is the buffer, never > written out. > Surefire thinks that the crashed. > > Am 2017-02-19 um 12:32 schrieb Stephen Connolly: > > What about using a local host bound socket rather than studio? > > > > In fact a socket could open up other options like forking the tests > inside > > docker, or running the surefire forks on remote JVMs. > > > > Plus we gain more control (modulo TCP retry/timeouts) > > > > On Sun 19 Feb 2017 at 11:16, Tibor Digana <tibor.dig...@googlemail.com> > > wrote: > > > >> Currently I placed sleep(1sec) before child process exit(0). If the > reader > >> process ThreadedStreamConsumer would read buffer entirely within 1 sec > and > >> the build pass then I have to add a confirmation mechanism where the > exit > >> event must be confirmed by plugin until short period of time. > >> Later we should not use these pipes but standard file system, but we > have > >> to take care of SecurityException thrown by JUnit3. > >> > >> On Sun, Feb 19, 2017 at 11:37 AM, Michael Osipov <micha...@apache.org> > >> wrote: > >> > >>> Am 2017-02-18 um 23:19 schrieb Christian Schulte: > >>> > >>>> Am 02/17/17 um 21:48 schrieb Michael Osipov: > >>>> > >>>>> Hi folks, > >>>>> > >>>>> Christian Schulte asked me a couple of days ago wether I am able to > >>>>> build Surefire master with Maven master. It was constantly failing > for > >>>>> him on his OpenBSD machines. Since I have several real servers with > >>>>> FreeBSD 10.3-STABLE at hand I did run all Surefire ITs and I was able > >> to > >>>>> reproduce it. Our entire test infrastructure wasn't unfortunately! > >>>>> > >>>>> @Tibor: correct me if something is wrong or missing! > >>>>> > >>>>> After several days of heavy testing, thread dumps and log file > analysis > >>>>> with Tibor Digana and various Maven combinations (3.3.9, master, > >>>>> MNG-6169, MNG-6169 + MCLEAN 3.0.0) we figured out that there are > >> several > >>>>> serious bugs in Surefire master, Maven Shared Utils 0.9/3.1.0 and > >> likely > >>>>> Maven Clean Plugin 3.0.0. > >>>>> > >>>>> Since crucial parts of Surefire rely on native code in the JVM > (forks, > >>>>> streams), our code was not robust enough. > >>>>> > >>>>> As of today we have found: > >>>>> > >>>>> * Missing flushes in streams caused forked VM to be apparently > >>>>> non-responsive > >>>>> * TestNG tests mostly failed due to duplicate contradicting > properties > >>>>> passed to forked VMs > >>>>> * Uninitialized/too early attributes made daemon threads to kill > forked > >>>>> VMs > >>>>> * Code or dependency change from MCLEAN 2.6.1 to 3.0.0 cause repeated > >>>>> failures of a handful ITs > >>>>> @Karl Heinz: were you able to figure out something here? > >>>>> > >>>>> Issues in JIRA are pending... > >>>>> > >>>>> Everyone's invited to take a look at the log output as well as the > >>>>> target directory of surefire-integration-tests and contribute: > >>>>> http://home.apache.org/~michaelo/maven/surefire/. The filenames > should > >>>>> be pretty much self-explanatory. > >>>>> > >>>>> My big question is: how can we improve our test infrastructure? Can > we > >>>>> raise with INFRA to get at least one FreeBSD and Solaris node for > >>>>> Jenkins? I consider coverage on Windows and Ubuntu way to small, we > do > >>>>> not even have a CentOS node. Surefire ITs and Maven ITs are paramount > >>>>> for us, we should treat them as such! > >>>>> > >>>>> Michael > >>>>> > >>>> > >>>> Does any of your findings solve the following already? This is what > >>>> makes the Jenkins build jobs appear unreliable. It's sending out an > >>>> email about a failed job and all you see is something like the > following > >>>> sporadically. I am having the same issue locally sporadically. > >>>> > >>>> [ERROR] Failed to execute goal > >>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test > >>>> (default-test) on project child2: Execution default-test of goal > >>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The > >>>> forked VM terminated without saying properly goodbye. VM crash or > >>>> System.exit called ? -> [Help 1] > >>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to > >>>> execute goal > org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test > >>>> (default-test) on project child2: Execution default-test of goal > >>>> org.apache.maven.plugins:maven-surefire-plugin:2.12.4:test failed: The > >>>> forked VM terminated without saying properly goodbye. VM crash or > >>>> System.exit called ? > >>>> > >>> > >>> We were able to solve only a few cases so far: 10%. The rest is still > >>> failing. > >>> > >>> I made some digging: it seems to be a very common problem in NodeJS > with > >>> async and forking: > >>> https://github.com/nodejs/node/issues/2972 > >>> https://github.com/nodejs/node/issues/6456 > >>> http://stackoverflow.com/q/11138355/696632 > >>> http://stackoverflow.com/q/1716296/696632 > >>> https://github.com/nodejs/node-v0.x-archive/issues/8329 > >>> https://github.com/nodejs/node-v0.x-archive/issues/1669 > >>> https://github.com/nodejs/node-v0.x-archive/issues/3737 > >>> > >>> > >>> Especially this answer is our case: > >>> > >>>> That's not a bug - process.exit() is an imperative that says "exit > >> now!". > >>>> > >>>> When stdout/stderr is a pipe (as is the case with child processes) and > >>>> said pipe is full, pending data gets lost. Waiting until the pipe > drains > >>>> is not an option, that could hang the process indefinitely. > >>>> > >>> > >>> I think if we don't gain more control over stdio, we are lost... > >>> > >>> > >>> > >>> --------------------------------------------------------------------- > >>> To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > >>> For additional commands, e-mail: dev-h...@maven.apache.org > >>> > >>> > >> > >> > >> -- > >> Cheers > >> Tibor > >> > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@maven.apache.org > For additional commands, e-mail: dev-h...@maven.apache.org > > -- Sent from my phone
