The acceptance tests used to run fine in parallel in the same host. We need docker in docker working to containerize them. We would need to add docker to the docker image and some other fun stuff. Short term you can make the tests run serially. :(
-Jake > On Aug 21, 2018, at 11:04 AM, Dan Smith <dsm...@pivotal.io> wrote: > > Actually, it looks like the problem is that we are *not* using docker > containers for the acceptance tests. Check this out, in > gradle/docker.gradle. Since acceptance tests use the default port, this > means the test are guaranteed to be flaky, especially since we are running > them in parallel: > > // ACCEPTANCE TEST NEEDS DOCKER-COMPOSE TO WORK WITHIN DOCKER FIRST > // acceptanceTest.configure(dockerConfig) > > I'm not sure what changed that is causing the tests to fail more often now, > but maybe a test ordering change? > > -Dan > > > >> On Tue, Aug 21, 2018 at 10:52 AM, Kenneth Howe <kh...@pivotal.io> wrote: >> >> >> >>> On Aug 21, 2018, at 10:44 AM, Kirk Lund <kl...@apache.org> wrote: >>> >>> GEODE-5590 would seem to imply that GfshRule does not have an adequate >> safe >>> guard? If it spawns a server process which binds to the default server >> port >>> and that process persists after the test then we need better tearDown. >>> >> Yes, that does appear to be the case. The current failures are apparently >> due to incomplete >> teardown between tests within a test class. >> >> I am attempting to reproduce the failures on a consistent basis for >> debugging the problem. >> >> >>> Actually I thought we were using Docker to run each AcceptanceTest in >>> isolation. Then when the test finishes the Docker instances goes away. >> Did >>> we stop using Docker for these? >>> >>> On Tue, Aug 21, 2018 at 10:25 AM, Sai Boorlagadda < >> sai.boorlaga...@gmail.com >>>> wrote: >>> >>>> DeployWithLargeJarTest & PutCommandWithJsonTest are flaky on Develop. >>>> >>>> DeployWithLargeJarTest - >>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>> develop/jobs/AcceptanceTest/builds/335 >>>> PutCommandWithJsonTest - >>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>> develop/jobs/AcceptanceTest/builds/334 >>>> >>>> On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda < >>>> sai.boorlaga...@gmail.com> >>>> wrote: >>>> >>>>> The metrics job themselves will be green (as they complete to success) >>>> but >>>>> you can expand the get_metrics task output and see that build#20 >> started >>>>> reporting these failures, so probably these are due to recent changes >> on >>>>> develop. I believe these metrics are from develop CI test runs. >>>>> >>>>>> On Tue, Aug 21, 2018 at 10:15 AM Kirk Lund <kl...@apache.org> wrote: >>>>>> >>>>>> Those metrics show AcceptanceTests consistently GREEN. Do these >> metrics >>>>>> include test failures from pull request precheckin runs like mine? Or >>>> does >>>>>> it just cover CI test runs? >>>>>> >>>>>> On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda < >>>>>> sai.boorlaga...@gmail.com >>>>>>> wrote: >>>>>> >>>>>>> Metrics show these started failing recently. >>>>>>> >>>>>>> >>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ >>>> metrics/jobs/ >>>>>>> GeodeAcceptanceTestMetrics/builds/20 >>>>>>> >>>>>>>> On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund <kl...@apache.org> wrote: >>>>>>>> >>>>>>>> Are PutCommandWithJsonTest and DeployWithLargeJarTest known to be >>>>>> flaky? >>>>>>>> >>>>>>>> My latest pull request failed with these two failures and all I did >>>>>> was >>>>>>>> extract LocalRegion.validateRegionName and improve unit testing of >>>>>>>> RegionNameValidation. No other tests failed for me. >>>>>>>> >>>>>>>>> Task :geode-assembly:acceptanceTest >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:619> >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:620> >>>>>>>> >>>>>> org.apache.geode.management.internal.cli.commands. >>>> PutCommandWithJsonTest >>>>>>>>> putWithJsonString FAILED >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:621> >>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:622> >>>>>>>> at >>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>>>> Method) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:623> >>>>>>>> at >>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>>> NativeConstructorAccessorImpl.java:62) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:624> >>>>>>>> at >>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>>>>>> DelegatingConstructorAccessorImpl.java:45) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:625> >>>>>>>> at >>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. >>>>>>> awaitIfNecessary(GfshScript.java:117) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:626> >>>>>>>> at >>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >>>>>>> GfshRule.java:135) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:627> >>>>>>>> at >>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >>>>>>> GfshScript.java:106) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:628> >>>>>>>> at >>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>> PutCommandWithJsonTest.putWithJsonString( >> PutCommandWithJsonTest.java: >>>> 55) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:629> >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:630> >>>>>>>> >>>>>> org.apache.geode.management.internal.cli.commands. >>>> DeployWithLargeJarTest >>>>>>>>> deployLargeSetOfJars FAILED >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:631> >>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:632> >>>>>>>> at >>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native >>>>>>>> Method) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:633> >>>>>>>> at >>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( >>>>>>> NativeConstructorAccessorImpl.java:62) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:634> >>>>>>>> at >>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( >>>>>>> DelegatingConstructorAccessorImpl.java:45) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:635> >>>>>>>> at >>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. >>>>>>> awaitIfNecessary(GfshScript.java:117) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:636> >>>>>>>> at >>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( >>>>>>> GfshRule.java:135) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:637> >>>>>>>> at >>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( >>>>>>> GfshScript.java:106) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:638> >>>>>>>> at >>>>>>>> org.apache.geode.management.internal.cli.commands. >>>>>>> DeployWithLargeJarTest.deployLargeSetOfJars( >>>> DeployWithLargeJarTest.java: >>>>>>> 41) >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:639> >>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680#L5b60bc1a:640> >>>>>>>>> Task :geode-assembly:acceptanceTest FAILED >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >> >>
