Re: Build failed in Jenkins: Geode-nightly #923

2017-08-16 Thread Jens Deppe 
I'm working on enabling parallel dunits in docker to work on Jenkins. If
this can happen it should help with some of these problems. However, it
requires fairly beefy systems to run well, otherwise you end up with other
issues around resource starvation. At worst we could just run with a
parallelism of 1 which should help with network isolation but won't help
with speeding up the build unfortunately.

--Jens

On Tue, Aug 15, 2017 at 2:01 PM, Mark Bretl <mbr...@apache.org> wrote:

> Hi Jared,
>
> Short answer: No, Longer answer: Ask Infra ;)
>
> We are generally running on VMs with multiple Jenkins executors, which
> means we could be sharing the box with other jobs. We have been
> piggybacking on other project's systems so far, trying to find 'available'
> systems. I have a feeling this is going to be an ongoing issue of getting a
> stable environment until we have another solution, such as requesting
> dedicated VMs or containers or etc.
>
> Open to ideas here...
>
> --Mark
>
>
> On Tue, Aug 15, 2017 at 10:59 AM, Jared Stewart <jstew...@pivotal.io>
> wrote:
>
> > It looks like all of our tests using default ports are failing in the
> > nightly build.  Here is my suspicion of what happened: We have some
> > Acceptance Tests that spin up separate JVMs for Locators and Servers.
> > Those tests make sure to kill their child processes as part of their
> > cleanup, but I suspect that when the build times out our test process
> gets
> > hard-killed, which would prevent the proper cleanup from taking place.
> > Does anyone have ssh access into this box to look for orphaned Java
> > processes?
> >
> > Thanks,
> > Jared
> >
> > > Begin forwarded message:
> > >
> > > From: Apache Jenkins Server <jenk...@builds.apache.org>
> > > Subject: Build failed in Jenkins: Geode-nightly #923
> > > Date: August 14, 2017 at 8:53:03 PM PDT
> > > To: dev@geode.apache.org, kmil...@pivotal.io, kh...@apache.org,
> > bschucha...@pivotal.io, jstew...@pivotal.io, kl...@pivotal.io,
> > dschnei...@pivotal.io, dbar...@pivotal.io, n...@pivotal.io
> > > Reply-To: dev@geode.apache.org
> > >
> > > See <https://builds.apache.org/job/Geode-nightly/923/display/red
> > irect?page=changes>
> > >
> > > Changes:
> > >
> > > [jiliao] GEODE-3328: fix a test failure on windows.
> > >
> > > [klund] Revert "GEODE-3328: fix a test failure on windows."
> > >
> > > [klund] GEODE-3436: revert recent refactoring of GFSH commands
> > >
> > > --
> > > [...truncated 129.88 KB...]
> > >org.apache.geode.test.dunit.RMIException: While invoking
> > org.apache.geode.modules.session.Tomcat8SessionsClientServer
> > DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host
> > asf905.gq1.ygridcore.net with 4 VMs
> > >at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
> > >at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
> > >at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
> > >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> > DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
> > >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> > DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)
> > >
> > >Caused by:
> > >java.net.BindException: Failed to create server socket on
> > null[40,404]
> > >at org.apache.geode.internal.net.
> > SocketCreator.createServerSocket(SocketCreator.java:783)
> > >at org.apache.geode.internal.net.
> > SocketCreator.createServerSocket(SocketCreator.java:745)
> > >at org.apache.geode.internal.net.
> > SocketCreator.createServerSocket(SocketCreator.java:712)
> > >at org.apache.geode.internal.cach
> > e.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
> > >at org.apache.geode.internal.cach
> > e.CacheServerImpl.start(CacheServerImpl.java:344)
> > >at org.apache.geode.modules.sessi
> > on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f
> > 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)
> > >
> > >Caused by:
> > >java.net.BindException: Address already in use (Bind failed)
> > >at java.net.PlainSocketImpl.socketBind(Native Method)
> > >at java.net.AbstractPlainSocketIm
> > pl.bind(AbstractPlainSocketImpl.java:387)

Re: Build failed in Jenkins: Geode-nightly #923

2017-08-15 Thread Mark Bretl 
Hi Jared,

Short answer: No, Longer answer: Ask Infra ;)

We are generally running on VMs with multiple Jenkins executors, which
means we could be sharing the box with other jobs. We have been
piggybacking on other project's systems so far, trying to find 'available'
systems. I have a feeling this is going to be an ongoing issue of getting a
stable environment until we have another solution, such as requesting
dedicated VMs or containers or etc.

Open to ideas here...

--Mark


On Tue, Aug 15, 2017 at 10:59 AM, Jared Stewart <jstew...@pivotal.io> wrote:

> It looks like all of our tests using default ports are failing in the
> nightly build.  Here is my suspicion of what happened: We have some
> Acceptance Tests that spin up separate JVMs for Locators and Servers.
> Those tests make sure to kill their child processes as part of their
> cleanup, but I suspect that when the build times out our test process gets
> hard-killed, which would prevent the proper cleanup from taking place.
> Does anyone have ssh access into this box to look for orphaned Java
> processes?
>
> Thanks,
> Jared
>
> > Begin forwarded message:
> >
> > From: Apache Jenkins Server <jenk...@builds.apache.org>
> > Subject: Build failed in Jenkins: Geode-nightly #923
> > Date: August 14, 2017 at 8:53:03 PM PDT
> > To: dev@geode.apache.org, kmil...@pivotal.io, kh...@apache.org,
> bschucha...@pivotal.io, jstew...@pivotal.io, kl...@pivotal.io,
> dschnei...@pivotal.io, dbar...@pivotal.io, n...@pivotal.io
> > Reply-To: dev@geode.apache.org
> >
> > See <https://builds.apache.org/job/Geode-nightly/923/display/red
> irect?page=changes>
> >
> > Changes:
> >
> > [jiliao] GEODE-3328: fix a test failure on windows.
> >
> > [klund] Revert "GEODE-3328: fix a test failure on windows."
> >
> > [klund] GEODE-3436: revert recent refactoring of GFSH commands
> >
> > --
> > [...truncated 129.88 KB...]
> >org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host
> asf905.gq1.ygridcore.net with 4 VMs
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)
> >
> >Caused by:
> >java.net.BindException: Failed to create server socket on
> null[40,404]
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:783)
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:745)
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:712)
> >at org.apache.geode.internal.cach
> e.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
> >at org.apache.geode.internal.cach
> e.CacheServerImpl.start(CacheServerImpl.java:344)
> >at org.apache.geode.modules.sessi
> on.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f
> 0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)
> >
> >Caused by:
> >java.net.BindException: Address already in use (Bind failed)
> >at java.net.PlainSocketImpl.socketBind(Native Method)
> >at java.net.AbstractPlainSocketIm
> pl.bind(AbstractPlainSocketImpl.java:387)
> >at java.net.ServerSocket.bind(ServerSocket.java:375)
> >at org.apache.geode.internal.net.
> SocketCreator.createServerSocket(SocketCreator.java:779)
> >... 5 more
> >
> > org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest >
> testSanity FAILED
> >org.apache.geode.test.dunit.RMIException: While invoking
> org.apache.geode.modules.session.Tomcat8SessionsClientServer
> DUnitTest$$Lambda$56/1529418967.call in VM 1 running on Host
> asf905.gq1.ygridcore.net with 4 VMs
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
> >at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
> >at org.apache.geode.modules.session.Tomcat8SessionsClientSer

Build failed in Jenkins: Geode-nightly #923

2017-08-14 Thread Apache Jenkins Server 
See 


Changes:

[jiliao] GEODE-3328: fix a test failure on windows.

[klund] Revert "GEODE-3328: fix a test failure on windows."

[klund] GEODE-3436: revert recent refactoring of GFSH commands

--
[...truncated 129.88 KB...]
org.apache.geode.test.dunit.RMIException: While invoking 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call
 in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs
at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)

Caused by:
java.net.BindException: Failed to create server socket on  null[40,404]
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712)
at 
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
at 
org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)

Caused by:
java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at 
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779)
... 5 more

org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > 
testSanity FAILED
org.apache.geode.test.dunit.RMIException: While invoking 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call
 in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs
at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.postSetUp(Tomcat8SessionsClientServerDUnitTest.java:44)

Caused by:
java.net.BindException: Failed to create server socket on  null[40,404]
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:783)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:745)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:712)
at 
org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.(AcceptorImpl.java:469)
at 
org.apache.geode.internal.cache.CacheServerImpl.start(CacheServerImpl.java:344)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.lambda$setupServer$f0fd67c5$1(Tomcat8SessionsClientServerDUnitTest.java:65)

Caused by:
java.net.BindException: Address already in use (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at 
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at 
org.apache.geode.internal.net.SocketCreator.createServerSocket(SocketCreator.java:779)
... 5 more

org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest > 
testInvalidate FAILED
org.apache.geode.test.dunit.RMIException: While invoking 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest$$Lambda$56/1529418967.call
 in VM 1 running on Host asf905.gq1.ygridcore.net with 4 VMs
at org.apache.geode.test.dunit.VM.invoke(VM.java:387)
at org.apache.geode.test.dunit.VM.invoke(VM.java:357)
at org.apache.geode.test.dunit.VM.invoke(VM.java:325)
at 
org.apache.geode.modules.session.Tomcat8SessionsClientServerDUnitTest.setupServer(Tomcat8SessionsClientServerDUnitTest.java:60)
at