I looked through the gradle files and couldn't figure out where to make the changes. If anyone else knows, please make the changes or let me know if you want to pair with me so I can learn.
On Tue, Aug 21, 2018 at 11:32 AM, Anthony Baker <aba...@pivotal.io> wrote: > By design, acceptanceTest should be run with defaults (including ports). > So if they aren’t run in containers they can’t be run in parallel. > > Anthony > > > > On Aug 21, 2018, at 11:15 AM, Kirk Lund <kl...@apache.org> wrote: > > > > Oops... I mean "orphaned processes" > > > > On Tue, Aug 21, 2018 at 11:13 AM, Kirk Lund <kl...@apache.org> wrote: > > > >> So, we need to disable running them in parallel and improve the tearDown > >> to make sure these tests don't leave an orphaned tests behind. > >> > >> On Tue, Aug 21, 2018 at 11:11 AM, Sai Boorlagadda < > >> sai.boorlaga...@gmail.com> wrote: > >> > >>> jdbc-connector acceptance tests need docker-in-docker (also > >>> docker-compose) > >>> to spin up mysql and postgres. > >>> > >>> On Tue, Aug 21, 2018 at 11:04 AM Dan Smith <dsm...@pivotal.io> wrote: > >>> > >>>> Actually, it looks like the problem is that we are *not* using docker > >>>> containers for the acceptance tests. Check this out, in > >>>> gradle/docker.gradle. Since acceptance tests use the default port, > this > >>>> means the test are guaranteed to be flaky, especially since we are > >>> running > >>>> them in parallel: > >>>> > >>>> // ACCEPTANCE TEST NEEDS DOCKER-COMPOSE TO WORK WITHIN DOCKER FIRST > >>>> // acceptanceTest.configure(dockerConfig) > >>>> > >>>> I'm not sure what changed that is causing the tests to fail more often > >>> now, > >>>> but maybe a test ordering change? > >>>> > >>>> -Dan > >>>> > >>>> > >>>> > >>>> On Tue, Aug 21, 2018 at 10:52 AM, Kenneth Howe <kh...@pivotal.io> > >>> wrote: > >>>> > >>>>> > >>>>> > >>>>>> On Aug 21, 2018, at 10:44 AM, Kirk Lund <kl...@apache.org> wrote: > >>>>>> > >>>>>> GEODE-5590 would seem to imply that GfshRule does not have an > >>> adequate > >>>>> safe > >>>>>> guard? If it spawns a server process which binds to the default > >>> server > >>>>> port > >>>>>> and that process persists after the test then we need better > >>> tearDown. > >>>>>> > >>>>> Yes, that does appear to be the case. The current failures are > >>> apparently > >>>>> due to incomplete > >>>>> teardown between tests within a test class. > >>>>> > >>>>> I am attempting to reproduce the failures on a consistent basis for > >>>>> debugging the problem. > >>>>> > >>>>> > >>>>>> Actually I thought we were using Docker to run each AcceptanceTest > >>> in > >>>>>> isolation. Then when the test finishes the Docker instances goes > >>> away. > >>>>> Did > >>>>>> we stop using Docker for these? > >>>>>> > >>>>>> On Tue, Aug 21, 2018 at 10:25 AM, Sai Boorlagadda < > >>>>> sai.boorlaga...@gmail.com > >>>>>>> wrote: > >>>>>> > >>>>>>> DeployWithLargeJarTest & PutCommandWithJsonTest are flaky on > >>> Develop. > >>>>>>> > >>>>>>> DeployWithLargeJarTest - > >>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ > >>>>>>> develop/jobs/AcceptanceTest/builds/335 > >>>>>>> PutCommandWithJsonTest - > >>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ > >>>>>>> develop/jobs/AcceptanceTest/builds/334 > >>>>>>> > >>>>>>> On Tue, Aug 21, 2018 at 10:18 AM Sai Boorlagadda < > >>>>>>> sai.boorlaga...@gmail.com> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> The metrics job themselves will be green (as they complete to > >>>> success) > >>>>>>> but > >>>>>>>> you can expand the get_metrics task output and see that build#20 > >>>>> started > >>>>>>>> reporting these failures, so probably these are due to recent > >>> changes > >>>>> on > >>>>>>>> develop. I believe these metrics are from develop CI test runs. > >>>>>>>> > >>>>>>>> On Tue, Aug 21, 2018 at 10:15 AM Kirk Lund <kl...@apache.org> > >>> wrote: > >>>>>>>> > >>>>>>>>> Those metrics show AcceptanceTests consistently GREEN. Do these > >>>>> metrics > >>>>>>>>> include test failures from pull request precheckin runs like > >>> mine? > >>>> Or > >>>>>>> does > >>>>>>>>> it just cover CI test runs? > >>>>>>>>> > >>>>>>>>> On Tue, Aug 21, 2018 at 10:09 AM, Sai Boorlagadda < > >>>>>>>>> sai.boorlaga...@gmail.com > >>>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Metrics show these started failing recently. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> https://concourse.apachegeode-ci.info/teams/main/pipelines/ > >>>>>>> metrics/jobs/ > >>>>>>>>>> GeodeAcceptanceTestMetrics/builds/20 > >>>>>>>>>> > >>>>>>>>>> On Tue, Aug 21, 2018 at 10:07 AM Kirk Lund <kl...@apache.org> > >>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Are PutCommandWithJsonTest and DeployWithLargeJarTest known to > >>> be > >>>>>>>>> flaky? > >>>>>>>>>>> > >>>>>>>>>>> My latest pull request failed with these two failures and all I > >>>> did > >>>>>>>>> was > >>>>>>>>>>> extract LocalRegion.validateRegionName and improve unit > >>> testing of > >>>>>>>>>>> RegionNameValidation. No other tests failed for me. > >>>>>>>>>>> > >>>>>>>>>>>> Task :geode-assembly:acceptanceTest > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:619 > >>>>> > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:620 > >>>>> > >>>>>>>>>>> > >>>>>>>>> org.apache.geode.management.internal.cli.commands. > >>>>>>> PutCommandWithJsonTest > >>>>>>>>>>>> putWithJsonString FAILED > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:621 > >>>>> > >>>>>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:622 > >>>>> > >>>>>>>>>>> at > >>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>>>>>>>>>> Method) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:623 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( > >>>>>>>>>> NativeConstructorAccessorImpl.java:62) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:624 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > >>>>>>>>>> DelegatingConstructorAccessorImpl.java:45) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:625 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. > >>>>>>>>>> awaitIfNecessary(GfshScript.java:117) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:626 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( > >>>>>>>>>> GfshRule.java:135) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:627 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( > >>>>>>>>>> GfshScript.java:106) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:628 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.management.internal.cli.commands. > >>>>>>>>>> PutCommandWithJsonTest.putWithJsonString( > >>>>> PutCommandWithJsonTest.java: > >>>>>>> 55) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:629 > >>>>> > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:630 > >>>>> > >>>>>>>>>>> > >>>>>>>>> org.apache.geode.management.internal.cli.commands. > >>>>>>> DeployWithLargeJarTest > >>>>>>>>>>>> deployLargeSetOfJars FAILED > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:631 > >>>>> > >>>>>>>>>>> org.junit.ComparisonFailure: expected:<[0]> but was:<[1]> > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:632 > >>>>> > >>>>>>>>>>> at > >>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > >>>>>>>>>>> Method) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:633 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> sun.reflect.NativeConstructorAccessorImpl.newInstance( > >>>>>>>>>> NativeConstructorAccessorImpl.java:62) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:634 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> sun.reflect.DelegatingConstructorAccessorImpl.newInstance( > >>>>>>>>>> DelegatingConstructorAccessorImpl.java:45) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:635 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript. > >>>>>>>>>> awaitIfNecessary(GfshScript.java:117) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:636 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshRule.execute( > >>>>>>>>>> GfshRule.java:135) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:637 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.test.junit.rules.gfsh.GfshScript.execute( > >>>>>>>>>> GfshScript.java:106) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:638 > >>>>> > >>>>>>>>>>> at > >>>>>>>>>>> org.apache.geode.management.internal.cli.commands. > >>>>>>>>>> DeployWithLargeJarTest.deployLargeSetOfJars( > >>>>>>> DeployWithLargeJarTest.java: > >>>>>>>>>> 41) > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:639 > >>>>> > >>>>>>>>>>> <https://concourse.apachegeode-ci.info/builds/19680# > >>> L5b60bc1a:640 > >>>>> > >>>>>>>>>>>> Task :geode-assembly:acceptanceTest FAILED > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>> > >>>>> > >>>> > >>> > >> > >> > >
