Hi Jared, Short answer: No, Longer answer: Ask Infra ;)
We are generally running on VMs with multiple Jenkins executors, which means we could be sharing the box with other jobs. We have been piggybacking on other project's systems so far, trying to find 'available' systems. I have a feeling this is going to be an ongoing issue of getting a stable environment until we have another solution, such as requesting dedicated VMs or containers or etc. Open to ideas here... --Mark On Tue, Aug 15, 2017 at 10:59 AM, Jared Stewart <jstew...@pivotal.io> wrote: > It looks like all of our tests using default ports are failing in the > nightly build. Here is my suspicion of what happened: We have some > Acceptance Tests that spin up separate JVMs for Locators and Servers. > Those tests make sure to kill their child processes as part of their > cleanup, but I suspect that when the build times out our test process gets > hard-killed, which would prevent the proper cleanup from taking place. > Does anyone have ssh access into this box to look for orphaned Java > processes? > > Thanks, > Jared > > > Begin forwarded message: > > > > From: Apache Jenkins Server <jenk...@builds.apache.org> > > Subject: Build failed in Jenkins: Geode-nightly #923 > > Date: August 14, 2017 at 8:53:03 PM PDT > > To: dev@geode.apache.org, kmil...@pivotal.io, kh...@apache.org, > bschucha...@pivotal.io, jstew...@pivotal.io, kl...@pivotal.io, > dschnei...@pivotal.io, dbar...@pivotal.io, n...@pivotal.io > > Reply-To: dev@geode.apache.org > > > > See <https://builds.apache.org/job/Geode-nightly/923/display/red > irect?page=changes> > > > > Changes: > > > > [jiliao] GEODE-3328: fix a test failure on windows. > > > > [klund] Revert "GEODE-3328: fix a test failure on windows." > > > > [klund] GEODE-3436: revert recent refactoring of GFSH commands > > > > ------------------------------------------ > > [...truncated 129.88 KB...] > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > > > Caused by: > > java.net.BindException: Failed to create server socket on > null[40,404] > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:783) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:745) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:712) > > at org.apache.geode.internal.cach > e.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > > at org.apache.geode.internal.cach > e.CacheServerImpl.start(CacheServerImpl.java:344) > > at org.apache.geode.modules.sessi > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > > > Caused by: > > java.net.BindException: Address already in use (Bind failed) > > at java.net.PlainSocketImpl.socketBind(Native Method) > > at java.net.AbstractPlainSocketIm > pl.bind(AbstractPlainSocketImpl.java:387) > > at java.net.ServerSocket.bind(ServerSocket.java:375) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:779) > > ... 5 more > > > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testSanity FAILED > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > > > Caused by: > > java.net.BindException: Failed to create server socket on > null[40,404] > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:783) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:745) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:712) > > at org.apache.geode.internal.cach > e.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > > at org.apache.geode.internal.cach > e.CacheServerImpl.start(CacheServerImpl.java:344) > > at org.apache.geode.modules.sessi > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > > > Caused by: > > java.net.BindException: Address already in use (Bind failed) > > at java.net.PlainSocketImpl.socketBind(Native Method) > > at java.net.AbstractPlainSocketIm > pl.bind(AbstractPlainSocketImpl.java:387) > > at java.net.ServerSocket.bind(ServerSocket.java:375) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:779) > > ... 5 more > > > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testInvalidate FAILED > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > > > Caused by: > > java.net.BindException: Failed to create server socket on > null[40,404] > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:783) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:745) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:712) > > at org.apache.geode.internal.cach > e.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > > at org.apache.geode.internal.cach > e.CacheServerImpl.start(CacheServerImpl.java:344) > > at org.apache.geode.modules.sessi > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > > > Caused by: > > java.net.BindException: Address already in use (Bind failed) > > at java.net.PlainSocketImpl.socketBind(Native Method) > > at java.net.AbstractPlainSocketIm > pl.bind(AbstractPlainSocketImpl.java:387) > > at java.net.ServerSocket.bind(ServerSocket.java:375) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:779) > > ... 5 more > > > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testRemoveAttribute FAILED > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > > > Caused by: > > java.net.BindException: Failed to create server socket on > null[40,404] > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:783) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:745) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:712) > > at org.apache.geode.internal.cach > e.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > > at org.apache.geode.internal.cach > e.CacheServerImpl.start(CacheServerImpl.java:344) > > at org.apache.geode.modules.sessi > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > > > Caused by: > > java.net.BindException: Address already in use (Bind failed) > > at java.net.PlainSocketImpl.socketBind(Native Method) > > at java.net.AbstractPlainSocketIm > pl.bind(AbstractPlainSocketImpl.java:387) > > at java.net.ServerSocket.bind(ServerSocket.java:375) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:779) > > ... 5 more > > > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testSessionExpirationByContainer FAILED > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > > > Caused by: > > java.net.BindException: Failed to create server socket on > null[40,404] > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:783) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:745) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:712) > > at org.apache.geode.internal.cach > e.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > > at org.apache.geode.internal.cach > e.CacheServerImpl.start(CacheServerImpl.java:344) > > at org.apache.geode.modules.sessi > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > > > Caused by: > > java.net.BindException: Address already in use (Bind failed) > > at java.net.PlainSocketImpl.socketBind(Native Method) > > at java.net.AbstractPlainSocketIm > pl.bind(AbstractPlainSocketImpl.java:387) > > at java.net.ServerSocket.bind(ServerSocket.java:375) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:779) > > ... 5 more > > > > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > > testSessionPersists1 FAILED > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:325) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60) > > at org.apache.geode.modules.session.Tomcat8SessionsClientServer > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44) > > > > Caused by: > > java.net.BindException: Failed to create server socket on > null[40,404] > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:783) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:745) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:712) > > at org.apache.geode.internal.cach > e.tier.sockets.AcceptorImpl.<init>(AcceptorImpl.java:469) > > at org.apache.geode.internal.cach > e.CacheServerImpl.start(CacheServerImpl.java:344) > > at org.apache.geode.modules.sessi > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65) > > > > Caused by: > > java.net.BindException: Address already in use (Bind failed) > > at java.net.PlainSocketImpl.socketBind(Native Method) > > at java.net.AbstractPlainSocketIm > pl.bind(AbstractPlainSocketImpl.java:387) > > at java.net.ServerSocket.bind(ServerSocket.java:375) > > at org.apache.geode.internal.net. > SocketCreator.createServerSocket(SocketCreator.java:779) > > ... 5 more > > > > 30 tests completed, 15 failed > > :extensions/geode-modules-tomcat8:distributedTest FAILED > > :extensions/geode-modules-tomcat8:integrationTest > > :extensions/session-testing-war:javadoc > > :extensions/session-testing-war:javadocJar > > :extensions/session-testing-war:sourcesJar > > :extensions/session-testing-war:war > > :extensions/session-testing-war:signArchives SKIPPED > > :extensions/session-testing-war:assemble > > :extensions/session-testing-war:compileTestJava UP-TO-DATE > > :extensions/session-testing-war:processTestResources UP-TO-DATE > > :extensions/session-testing-war:testClasses UP-TO-DATE > > :extensions/session-testing-war:checkMissedTests UP-TO-DATE > > :extensions/session-testing-war:spotlessJavaCheck > > :extensions/session-testing-war:spotlessCheck > > :extensions/session-testing-war:test UP-TO-DATE > > :extensions/session-testing-war:check > > :extensions/session-testing-war:build > > :extensions/session-testing-war:distributedTest UP-TO-DATE > > :extensions/session-testing-war:integrationTest UP-TO-DATE > > :geode-assembly:distTar > > :geode-assembly:distZip > > :geode-assembly:writeBuildInfo > > :geode-assembly:srcDistTar > > :geode-assembly:srcDistZip > > :geode-assembly:signArchives SKIPPED > > :geode-assembly:assemble > > :geode-assembly:checkMissedTests > > :geode-assembly:spotlessJavaCheck > > :geode-assembly:spotlessCheck > > :geode-assembly:test > > :geode-assembly:check > > :geode-assembly:build > > :geode-assembly:distributedTest > > > > org.apache.geode.rest.internal.web.controllers.RestAPIsWithSSLDUnitTest > > testSSLWithCipherSuite[1] FAILED > > org.apache.geode.test.dunit.RMIException: While invoking > org.apache.geode.test.dunit.NamedRunnable.run in VM 3 running on Host > asf905.gq1.ygridcore.net with 4 VMs > > at org.apache.geode.test.dunit.VM.invoke(VM.java:387) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:357) > > at org.apache.geode.test.dunit.VM.invoke(VM.java:290) > > at org.apache.geode.rest.internal.web.controllers.RestAPIsWithS > SLDUnitTest.startInfraWithSSL(RestAPIsWithSSLDUnitTest.java:257) > > at org.apache.geode.rest.internal.web.controllers.RestAPIsWithS > SLDUnitTest.testSSLWithCipherSuite(RestAPIsWithSSLDUnitTest.java:651) > > > > Caused by: > > org.apache.geode.ToDataException: toData failed on > PdxSerializable class org.apache.geode.rest.internal > .web.controllers.Person > > at org.apache.geode.internal.Inte > rnalDataSerializer.writePdx(InternalDataSerializer.java:3007) > > at org.apache.geode.internal.Inte > rnalDataSerializer.basicWriteObject(InternalDataSerializer.java:2077) > > at org.apache.geode.DataSerialize > r.writeObject(DataSerializer.java:2936) > > at org.apache.geode.internal.util > .BlobHelper.serializeTo(BlobHelper.java:66) > > at org.apache.geode.internal.cach > e.tier.sockets.Message.serializeAndAddPart(Message.java:396) > > at org.apache.geode.internal.cach > e.tier.sockets.Message.addObjPart(Message.java:340) > > at org.apache.geode.internal.cach > e.tier.sockets.Message.addObjPart(Message.java:319) > > at org.apache.geode.cache.client. > internal.PutOp$PutOpImpl.<init>(PutOp.java:281) > > at org.apache.geode.cache.client. > internal.PutOp.execute(PutOp.java:66) > > at org.apache.geode.cache.client. > internal.ServerRegionProxy.put(ServerRegionProxy.java:159) > > at org.apache.geode.internal.cach > e.LocalRegion.serverPut(LocalRegion.java:2987) > > at org.apache.geode.internal.cach > e.LocalRegion.cacheWriteBeforePut(LocalRegion.java:3096) > > at org.apache.geode.internal.cach > e.ProxyRegionMap.basicPut(ProxyRegionMap.java:222) > > at org.apache.geode.internal.cach > e.LocalRegion.virtualPut(LocalRegion.java:5598) > > at org.apache.geode.internal.cach > e.LocalRegionDataView.putEntry(LocalRegionDataView.java:151) > > at org.apache.geode.internal.cach > e.LocalRegion.basicPut(LocalRegion.java:5027) > > at org.apache.geode.internal.cach > e.LocalRegion.validatedPut(LocalRegion.java:1580) > > at org.apache.geode.internal.cach > e.LocalRegion.put(LocalRegion.java:1567) > > at org.apache.geode.internal.cach > e.AbstractRegion.put(AbstractRegion.java:325) > > at org.apache.geode.rest.internal > .web.controllers.RestAPIsWithSSLDUnitTest.doPutsInClientCach > e(RestAPIsWithSSLDUnitTest.java:184) > > at org.apache.geode.rest.internal > .web.controllers.RestAPIsWithSSLDUnitTest.lambda$startInfraW > ithSSL$1253249a$4(RestAPIsWithSSLDUnitTest.java:257) > > > > Caused by: > > org.apache.geode.cache.client.ServerConnectivityException: > Pool unexpected socket timed out on client connection=Pooled Connection to > asf905.gq1.ygridcore.net:52966: > Connection[asf905.gq1.ygridcore.net:52966]@1232456419). > Server unreachable: could not connect after 1 attempts > > at org.apache.geode.cache.client. > internal.OpExecutorImpl.handleException(OpExecutorImpl.java:798) > > at org.apache.geode.cache.client. > internal.OpExecutorImpl.handleException(OpExecutorImpl.java:623) > > at org.apache.geode.cache.client. > internal.OpExecutorImpl.execute(OpExecutorImpl.java:174) > > at org.apache.geode.cache.client. > internal.OpExecutorImpl.execute(OpExecutorImpl.java:115) > > at org.apache.geode.cache.client. > internal.PoolImpl.execute(PoolImpl.java:775) > > at org.apache.geode.cache.client. > internal.GetPDXIdForEnumOp.execute(GetPDXIdForEnumOp.java:38) > > at org.apache.geode.pdx.internal. > ClientTypeRegistration.processEnumInfoForEnumId(ClientTypeRe > gistration.java:161) > > at org.apache.geode.pdx.internal. > ClientTypeRegistration.getEnumId(ClientTypeRegistration.java:153) > > at org.apache.geode.pdx.internal. > TypeRegistry.getEnumId(TypeRegistry.java:367) > > at org.apache.geode.internal.Inte > rnalDataSerializer.writePdxEnum(InternalDataSerializer.java:2139) > > at org.apache.geode.internal.Inte > rnalDataSerializer.writeUserObject(InternalDataSerializer.java:1529) > > at org.apache.geode.internal.Inte > rnalDataSerializer.writeWellKnownObject(InternalDataSerializer.java:1437) > > at org.apache.geode.internal.Inte > rnalDataSerializer.basicWriteObject(InternalDataSerializer.java:2102) > > at org.apache.geode.pdx.internal. > PdxOutputStream.writeObject(PdxOutputStream.java:82) > > at org.apache.geode.pdx.internal. > PdxWriterImpl.writeObject(PdxWriterImpl.java:329) > > at org.apache.geode.pdx.internal. > PdxWriterImpl.writeObject(PdxWriterImpl.java:319) > > at org.apache.geode.rest.internal > .web.controllers.Person.toData(Person.java:166) > > at org.apache.geode.internal.Inte > rnalDataSerializer.writePdx(InternalDataSerializer.java:2977) > > ... 20 more > > Build timed out (after 1,440 minutes). Marking the build as failed. > > Build was aborted > > Archiving artifacts > > [Fast Archiver] Compressed 338.72 MB of artifacts by 42.4% relative to > #914 > > Recording test results > > Not sending mail to unregistered user ukohlme...@pivotal.io > > Not sending mail to unregistered user gz...@pivotal.io > > Not sending mail to unregistered user lhughesgodf...@pivotal.io > > Not sending mail to unregistered user e...@pivotal.io > > Not sending mail to unregistered user huyn...@gmail.com > > Not sending mail to unregistered user hkhame...@pivotal.io > > Not sending mail to unregistered user jil...@pivotal.io > >
