[ https://issues.apache.org/jira/browse/GEODE-2386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15850537#comment-15850537 ]
Dan Smith edited comment on GEODE-2386 at 2/2/17 9:28 PM: ---------------------------------------------------------- My previous theory about why the classpath was set to the gradle wrapper was wrong. I was able to reproduce this failure by applying the attached reproduce.patch and running this: {code} ./gradlew geode-lucene:distributedTest --tests '*MySuiteDUnitTest*' {code} This suite reproduces the order in which the nightly builds are running the lucene tests. What I found is that LuceneClusterConfigurationDUnitTest ends up invoking the DistributedRestoreSystemProperties rule twice, once because it extends CliCommandTestBase and once because it uses LocatorServerStartupRule. DistributedRestoreSystemProperties has bug where it if it is invoked twice, it ends up calling System.setProperties(null) because it nulls out the value of originalProperties in the first call to after. if you call System.setProperties(null), the JVM calls the native initProperties and sets the system properties to their original values generated by the JVM. This loses the new value of java.class.path that is set gradle's BootstrapSecurityManager. was (Author: upthewaterspout): My previous theory about why the classpath was set to the gradle wrapper was wrong. I was able to reproduce this failure by applying the attached reproduce.patch and running this: ./gradlew geode-lucene:distributedTest --tests '*MySuiteDUnitTest*' This suite reproduces the order in which the nightly builds are running the lucene tests. What I found is that LuceneClusterConfigurationDUnitTest ends up invoking the DistributedRestoreSystemProperties rule twice, once because it extends CliCommandTestBase and once because it uses LocatorServerStartupRule. DistributedRestoreSystemProperties has bug where it if it is invoked twice, it ends up calling System.setProperties(null) because it nulls out the value of originalProperties in the first call to after. if you call System.setProperties(null), the JVM calls the native initProperties and sets the system properties to their original values generated by the JVM. This loses the new value of java.class.path that is set gradle's BootstrapSecurityManager. > Unable to launch dunit VMs in nightly builds > -------------------------------------------- > > Key: GEODE-2386 > URL: https://issues.apache.org/jira/browse/GEODE-2386 > Project: Geode > Issue Type: Bug > Components: build > Reporter: Dan Smith > Fix For: 1.1.0 > > Attachments: reproduce.patch > > > The recent apache nightly builds for the release branch and develop are > seeing lucene tests fail with "java.lang.RuntimeException: Unable to launch > dunit VMs". In the logs we see this error message: > "[locator] Error: Could not find or load main class > org.apache.geode.test.dunit.standalone.ChildVM" > We need to figure out what's going on. > https://builds.apache.org/job/Geode-release/40/#showFailuresLink > https://builds.apache.org/job/Geode-nightly/731/ > {noformat} > Stacktrace > java.lang.RuntimeException: Unable to launch dunit VMs > at > org.apache.geode.test.dunit.standalone.DUnitLauncher.launchIfNeeded(DUnitLauncher.java:144) > at > org.apache.geode.test.dunit.internal.JUnit4DistributedTestCase.initializeDistributedTestCase(JUnit4DistributedTestCase.java:131) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) > at > org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) > at > org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) > at > org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.runTestClass(JUnitTestClassExecuter.java:114) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecuter.execute(JUnitTestClassExecuter.java:57) > at > org.gradle.api.internal.tasks.testing.junit.JUnitTestClassProcessor.processTestClass(JUnitTestClassProcessor.java:66) > at > org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51) > at sun.reflect.GeneratedMethodAccessor15.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32) > at > org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93) > at com.sun.proxy.$Proxy2.processTestClass(Unknown Source) > at > org.gradle.api.internal.tasks.testing.worker.TestWorker.processTestClass(TestWorker.java:109) > at sun.reflect.GeneratedMethodAccessor14.invoke(Unknown Source) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35) > at > org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24) > at > org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:377) > at > org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54) > at > org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.RuntimeException: VMs did not start up within 120 seconds > at > org.apache.geode.test.dunit.standalone.DUnitLauncher.launch(DUnitLauncher.java:220) > at > org.apache.geode.test.dunit.standalone.DUnitLauncher.launchIfNeeded(DUnitLauncher.java:142) > ... 34 more > Standard Output > Executing [/usr/local/asfpackages/java/jdk1.8.0_102/jre/bin/java, -classpath, > /home/jenkins/jenkins-slave/workspace/Geode-release/caches/2.14.1/workerMain/gradle-worker.jar, > -Dgemfire.DUnitLauncher.RMI_PORT=22832, -Dgemfire.DUnitLauncher.VM_NUM=-2, > -Dgemfire.DUnitLauncher.VM_VERSION=000, > -DWORKSPACE_DIR=/home/jenkins/jenkins-slave/workspace/Geode-release/geode-lucene/build/distributedTest/., > -Dgemfire.enable-network-partition-detection=false, -Dlog-level=info, > -Djava.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib, > -Xrunjdwp:transport=dt_socket,server=y,suspend=n, > -XX:+HeapDumpOnOutOfMemoryError, -Xmx512m, > -Dgemfire.DEFAULT_MAX_OPLOG_SIZE=10, -Dgemfire.disallowMcastDefaults=true, > -Dgemfire.use-ephemeral-ports=true, -ea, -XX:MetaspaceSize=512m, > -XX:+PrintGC, -XX:+PrintGCDetails, -XX:+PrintGCTimeStamps, -DdummyArg=true, > org.apache.geode.test.dunit.standalone.ChildVM] > [locator] Listening for transport dt_socket at address: 43952 > [locator] Heap > [locator] PSYoungGen total 153088K, used 7895K [0x00000000f5580000, > 0x0000000100000000, 0x0000000100000000) > [locator] eden space 131584K, 6% used > [0x00000000f5580000,0x00000000f5d35c68,0x00000000fd600000) > [locator] from space 21504K, 0% used > [0x00000000feb00000,0x00000000feb00000,0x0000000100000000) > [locator] to space 21504K, 0% used > [0x00000000fd600000,0x00000000fd600000,0x00000000feb00000) > [locator] ParOldGen total 349696K, used 0K [0x00000000e0000000, > 0x00000000f5580000, 0x00000000f5580000) > [locator] object space 349696K, 0% used > [0x00000000e0000000,0x00000000e0000000,0x00000000f5580000) > [locator] Metaspace used 2703K, capacity 4480K, committed 4480K, > reserved 1056768K > [locator] class space used 300K, capacity 384K, committed 384K, reserved > 1048576K > Standard Error > [locator] Error: Could not find or load main class > org.apache.geode.test.dunit.standalone.ChildVM > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)