PS: the test is using DistributedRestoreSystemProperties as a JUnit Rule, so it's saving and restoring all System properties before and after every test. This gives each test a clean slate for System properties.
On Fri, Feb 3, 2017 at 1:41 PM, Kirk Lund <kl...@pivotal.io> wrote: > JMXMBeanDUnitTest has tests using SSL, not using SSL, configuring SSL with > the old legacy config properties, configuring SSL with the new config > properties. > > One test is currently marked as a FlakyTest: testJMXOverLegacySSL > > But it isn't a FlakyTest. By itself in fresh JVMs, testJMXOverLegacySSL > passes 100% but if I run this test with the other tests > in JMXMBeanDUnitTest it fails 100%. It always runs last (coincidence) and > it apparently fails because a previous test has configured SSL in some way. > > I've tried adding "disconnectAllFromDS()" to tearDown but it still fails. > I believe there's some static either in Geode code or JDK code that gets a > value stuck in it from a previous test so that this test cannot pass. > > I tried adding "invokeInEveryVM(() -> SocketCreatorFactory.close());" to > tearDown but it fails as well. > > I filed GEODE-2427 to better describe the problem since it's not really a > Flaky test. JMXMBeanDUnitTest is an example of a test that we have marked > as Flaky because when we run it via flakyTest, it gets fresh unused dunit > JVMs. This really isn't the purpose of "FlakyTest" and this means we have > some static somewhere that's getting stuck and I simply cannot find that > static (or maybe it's a JDK bug or maybe it's something entirely different). > > Any idea what's preventing this test from passing. Below is the full stack > of the failure when I run it with other tests: > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.test.dunit.NamedRunnable.run in VM 1 running on Host > 10.118.33.232 with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:377) > at org.apache.geode.test.dunit.VM.invoke(VM.java:347) > at org.apache.geode.test.dunit.VM.invoke(VM.java:280) > at org.apache.geode.management.JMXMBeanDUnitTest.testJMXOverLegacySSL( > JMXMBeanDUnitTest.java:147) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( > FrameworkMethod.java:50) > at org.junit.internal.runners.model.ReflectiveCallable.run( > ReflectiveCallable.java:12) > at org.junit.runners.model.FrameworkMethod.invokeExplosively( > FrameworkMethod.java:47) > at org.junit.internal.runners.statements.InvokeMethod. > evaluate(InvokeMethod.java:17) > at org.junit.internal.runners.statements.RunBefores. > evaluate(RunBefores.java:26) > at org.junit.internal.runners.statements.RunAfters.evaluate( > RunAfters.java:27) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) > at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) > at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) > at org.junit.rules.RunRules.evaluate(RunRules.java:20) > at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) > at org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:78) > at org.junit.runners.BlockJUnit4ClassRunner.runChild( > BlockJUnit4ClassRunner.java:57) > at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) > at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) > at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) > at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) > at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) > at org.junit.internal.runners.statements.RunBefores. > evaluate(RunBefores.java:26) > at org.junit.runners.ParentRunner.run(ParentRunner.java:363) > at org.junit.runner.JUnitCore.run(JUnitCore.java:137) > at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs( > JUnit4IdeaTestRunner.java:117) > at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs( > JUnit4IdeaTestRunner.java:42) > at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart( > JUnitStarter.java:262) > at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:84) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke( > NativeMethodAccessorImpl.java:62) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147) > Caused by: java.io.IOException: Failed to retrieve RMIServer stub: > javax.naming.CommunicationException [Root exception is > java.rmi.ConnectIOException: error during JRMP connection establishment; > nested exception is: > javax.net.ssl.SSLHandshakeException: > sun.security.validator.ValidatorException: > PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target] > at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369) > at javax.management.remote.JMXConnectorFactory.connect( > JMXConnectorFactory.java:270) > at org.apache.geode.management.JMXMBeanDUnitTest. > connectAndValidateAsJmxClient(JMXMBeanDUnitTest.java:202) > at org.apache.geode.management.JMXMBeanDUnitTest. > connectAndValidateAsJmxClient(JMXMBeanDUnitTest.java:179) > at org.apache.geode.management.JMXMBeanDUnitTest.lambda$ > testJMXOverLegacySSL$bb17a952$1(JMXMBeanDUnitTest.java:149) > at org.apache.geode.test.dunit.NamedRunnable.run(NamedRunnable.java:31) > at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at hydra.MethExecutor.executeObject(MethExecutor.java:245) > at org.apache.geode.test.dunit.standalone.RemoteDUnitVM. > executeMethodOnObject(RemoteDUnitVM.java:73) > at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke( > DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:497) > at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323) > at sun.rmi.transport.Transport$1.run(Transport.java:200) > at sun.rmi.transport.Transport$1.run(Transport.java:197) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.Transport.serviceCall(Transport.java:196) > at sun.rmi.transport.tcp.TCPTransport.handleMessages( > TCPTransport.java:568) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0( > TCPTransport.java:826) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$ > 256(TCPTransport.java:683) > at java.security.AccessController.doPrivileged(Native Method) > at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run( > TCPTransport.java:682) > at java.util.concurrent.ThreadPoolExecutor.runWorker( > ThreadPoolExecutor.java:1142) > at java.util.concurrent.ThreadPoolExecutor$Worker.run( > ThreadPoolExecutor.java:617) > at java.lang.Thread.run(Thread.java:745) > Caused by: javax.naming.CommunicationException [Root exception is > java.rmi.ConnectIOException: error during JRMP connection establishment; > nested exception is: > javax.net.ssl.SSLHandshakeException: > sun.security.validator.ValidatorException: > PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target] > at com.sun.jndi.rmi.registry.RegistryContext.lookup( > RegistryContext.java:122) > at com.sun.jndi.toolkit.url.GenericURLContext.lookup( > GenericURLContext.java:205) > at javax.naming.InitialContext.lookup(InitialContext.java:417) > at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI( > RMIConnector.java:1957) > at javax.management.remote.rmi.RMIConnector.findRMIServer( > RMIConnector.java:1924) > at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:287) > ... 26 more > Caused by: java.rmi.ConnectIOException: error during JRMP connection > establishment; nested exception is: > javax.net.ssl.SSLHandshakeException: > sun.security.validator.ValidatorException: > PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target > at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:304) > at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:202) > at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:342) > at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source) > at com.sun.jndi.rmi.registry.RegistryContext.lookup( > RegistryContext.java:118) > ... 31 more > Caused by: javax.net.ssl.SSLHandshakeException: > sun.security.validator.ValidatorException: > PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target > at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) > at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1949) > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:302) > at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:296) > at sun.security.ssl.ClientHandshaker.serverCertificate( > ClientHandshaker.java:1509) > at sun.security.ssl.ClientHandshaker.processMessage( > ClientHandshaker.java:216) > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) > at sun.security.ssl.Handshaker.process_record(Handshaker.java:914) > at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1062) > at sun.security.ssl.SSLSocketImpl.performInitialHandshake( > SSLSocketImpl.java:1375) > at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:747) > at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:123) > at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82) > at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140) > at java.io.DataOutputStream.flush(DataOutputStream.java:123) > at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:229) > ... 35 more > Caused by: sun.security.validator.ValidatorException: PKIX path building > failed: sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:387) > at sun.security.validator.PKIXValidator.engineValidate( > PKIXValidator.java:292) > at sun.security.validator.Validator.validate(Validator.java:260) > at sun.security.ssl.X509TrustManagerImpl.validate( > X509TrustManagerImpl.java:324) > at sun.security.ssl.X509TrustManagerImpl.checkTrusted( > X509TrustManagerImpl.java:229) > at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted( > X509TrustManagerImpl.java:124) > at sun.security.ssl.ClientHandshaker.serverCertificate( > ClientHandshaker.java:1491) > ... 46 more > Caused by: sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target > at sun.security.provider.certpath.SunCertPathBuilder. > build(SunCertPathBuilder.java:146) > at sun.security.provider.certpath.SunCertPathBuilder.engineBuild( > SunCertPathBuilder.java:131) > at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280) > at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:382) > ... 52 more > >
