Re: Very red CI -> Hold merges, please

[Kirk Lund]

I merged it in.

On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:

> I think we should go ahead and merge in
> https://github.com/apache/geode/pull/3172 since it resolves the
> GfshConsoleModeUnitTest UnitTest failures.
>
> On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag  wrote:
>
>> FYI, I have just merged a ci timeout fix to increase the timeout for
>> geode-benchmarks to 4h. This does not influence any geode modules.
>>
>> Regards
>> Naba
>>
>> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann 
>> wrote:
>>
>> > Hi folks,
>> >
>> > Our CI is very red since ~24 hours
>> > <
>> >
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
>> > >.
>> > It looks like a substantial new issue was introduced.
>> >
>> > Can we hold off on merging new changes to the develop branch till this
>> > issue is resolved?
>> >
>> > Thank you all!
>> >
>>
>

Re: Very red CI -> Hold merges, please

[Kirk Lund]

Sorry, go ahead and revert the commit and reopen the PR.

On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:

> I was still working on a fix...
>
> On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
>
> > I merged it in.
> >
> > On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:
> >
> > > I think we should go ahead and merge in
> > > https://github.com/apache/geode/pull/3172 since it resolves the
> > > GfshConsoleModeUnitTest UnitTest failures.
> > >
> > > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag  wrote:
> > >
> > >> FYI, I have just merged a ci timeout fix to increase the timeout for
> > >> geode-benchmarks to 4h. This does not influence any geode modules.
> > >>
> > >> Regards
> > >> Naba
> > >>
> > >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann  >
> > >> wrote:
> > >>
> > >> > Hi folks,
> > >> >
> > >> > Our CI is very red since ~24 hours
> > >> > <
> > >> >
> > >>
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> > >> > >.
> > >> > It looks like a substantial new issue was introduced.
> > >> >
> > >> > Can we hold off on merging new changes to the develop branch till
> this
> > >> > issue is resolved?
> > >> >
> > >> > Thank you all!
> > >> >
> > >>
> > >
> >
>

Re: Very red CI -> Hold merges, please

[Nabarun Nag]

FYI, I have just merged a ci timeout fix to increase the timeout for
geode-benchmarks to 4h. This does not influence any geode modules.

Regards
Naba

On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann 
wrote:

> Hi folks,
>
> Our CI is very red since ~24 hours
> <
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> >.
> It looks like a substantial new issue was introduced.
>
> Can we hold off on merging new changes to the develop branch till this
> issue is resolved?
>
> Thank you all!
>

Very red CI -> Hold merges, please

[Alexander Murmann]

Hi folks,

Our CI is very red since ~24 hours
.
It looks like a substantial new issue was introduced.

Can we hold off on merging new changes to the develop branch till this
issue is resolved?

Thank you all!

Re: Very red CI -> Hold merges, please

[Kirk Lund]

I think we should go ahead and merge in
https://github.com/apache/geode/pull/3172 since it resolves the
GfshConsoleModeUnitTest UnitTest failures.

On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag  wrote:

> FYI, I have just merged a ci timeout fix to increase the timeout for
> geode-benchmarks to 4h. This does not influence any geode modules.
>
> Regards
> Naba
>
> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann 
> wrote:
>
> > Hi folks,
> >
> > Our CI is very red since ~24 hours
> > <
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> > >.
> > It looks like a substantial new issue was introduced.
> >
> > Can we hold off on merging new changes to the develop branch till this
> > issue is resolved?
> >
> > Thank you all!
> >
>

Re: Very red CI -> Hold merges, please

[Jens Deppe]

I was still working on a fix...

On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:

> I merged it in.
>
> On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:
>
> > I think we should go ahead and merge in
> > https://github.com/apache/geode/pull/3172 since it resolves the
> > GfshConsoleModeUnitTest UnitTest failures.
> >
> > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag  wrote:
> >
> >> FYI, I have just merged a ci timeout fix to increase the timeout for
> >> geode-benchmarks to 4h. This does not influence any geode modules.
> >>
> >> Regards
> >> Naba
> >>
> >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann 
> >> wrote:
> >>
> >> > Hi folks,
> >> >
> >> > Our CI is very red since ~24 hours
> >> > <
> >> >
> >>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> >> > >.
> >> > It looks like a substantial new issue was introduced.
> >> >
> >> > Can we hold off on merging new changes to the develop branch till this
> >> > issue is resolved?
> >> >
> >> > Thank you all!
> >> >
> >>
> >
>

Re: Very red CI -> Hold merges, please

[Bruce Schuchardt]

I'm looking into the locator test failures.  I recently changed those 
tests to make them not depend on the old JUnit4DistributedTestCase and 
they lost some settings along the way.  There's also a real problem with 
Geode that this test is exposing because of that change, so I'd like to 
NOT fix these failures by changing the test's settings.


Unfortunately I need to run some serious regression tests to make sure I 
haven't broken anything.  That will delay any fix until tomorrow at the 
earliest.


On 2/7/19 1:46 PM, Alexander Murmann wrote:

Awesome! Things look a lot better now. It seems like the only consistent
issue now is in DistributedTestOpenJDK11 where the LocatorDUnitTest
`testMultipleLocatorsRestartingAtSameTimeWithMissingServers` has been
failing since 2 days or so. Is anyone looking at that?

On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:


No worries. I think I have a better fix now. At least the builds are moving
again.

On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:


Sorry, go ahead and revert the commit and reopen the PR.

On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:


I was still working on a fix...

On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:


I merged it in.

On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:


I think we should go ahead and merge in
https://github.com/apache/geode/pull/3172 since it resolves the
GfshConsoleModeUnitTest UnitTest failures.

On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 

wrote:

FYI, I have just merged a ci timeout fix to increase the timeout

for

geode-benchmarks to 4h. This does not influence any geode modules.

Regards
Naba

On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <

amurm...@apache.org

wrote:


Hi folks,

Our CI is very red since ~24 hours
<


https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372

.

It looks like a substantial new issue was introduced.

Can we hold off on merging new changes to the develop branch

till

this

issue is resolved?

Thank you all!

Re: Very red CI -> Hold merges, please

[Kirk Lund]

Hmm, and that was another false search hit in Jira! Searching for
WANRollingUpgradeNewSenderProcessOldEvent in Jira brings up GEODE-3967
which apparently does NOT involve that test. So, maybe we found another
flaky test.

Jira search seems to not work very well.

On Thu, Feb 7, 2019 at 2:24 PM Kirk Lund  wrote:

> The UpgradeTest failures on your latest commit for this PR are
> WANRollingUpgradeNewSenderProcessOldEvent which seems to be a reoccurrence
> of [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967). I
> recommend having Gester take a look at that these failures. He marked
> [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967) as
> resolved on Jan 9th.
>
> On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:
>
>> No worries. I think I have a better fix now. At least the builds are
>> moving
>> again.
>>
>> On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:
>>
>> > Sorry, go ahead and revert the commit and reopen the PR.
>> >
>> > On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:
>> >
>> > > I was still working on a fix...
>> > >
>> > > On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
>> > >
>> > > > I merged it in.
>> > > >
>> > > > On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:
>> > > >
>> > > > > I think we should go ahead and merge in
>> > > > > https://github.com/apache/geode/pull/3172 since it resolves the
>> > > > > GfshConsoleModeUnitTest UnitTest failures.
>> > > > >
>> > > > > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 
>> wrote:
>> > > > >
>> > > > >> FYI, I have just merged a ci timeout fix to increase the timeout
>> for
>> > > > >> geode-benchmarks to 4h. This does not influence any geode
>> modules.
>> > > > >>
>> > > > >> Regards
>> > > > >> Naba
>> > > > >>
>> > > > >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
>> > amurm...@apache.org
>> > > >
>> > > > >> wrote:
>> > > > >>
>> > > > >> > Hi folks,
>> > > > >> >
>> > > > >> > Our CI is very red since ~24 hours
>> > > > >> > <
>> > > > >> >
>> > > > >>
>> > > >
>> > >
>> >
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
>> > > > >> > >.
>> > > > >> > It looks like a substantial new issue was introduced.
>> > > > >> >
>> > > > >> > Can we hold off on merging new changes to the develop branch
>> till
>> > > this
>> > > > >> > issue is resolved?
>> > > > >> >
>> > > > >> > Thank you all!
>> > > > >> >
>> > > > >>
>> > > > >
>> > > >
>> > >
>> >
>>
>

Re: Very red CI -> Hold merges, please

[Jens Deppe]

No worries. I think I have a better fix now. At least the builds are moving
again.

On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:

> Sorry, go ahead and revert the commit and reopen the PR.
>
> On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:
>
> > I was still working on a fix...
> >
> > On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
> >
> > > I merged it in.
> > >
> > > On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:
> > >
> > > > I think we should go ahead and merge in
> > > > https://github.com/apache/geode/pull/3172 since it resolves the
> > > > GfshConsoleModeUnitTest UnitTest failures.
> > > >
> > > > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag  wrote:
> > > >
> > > >> FYI, I have just merged a ci timeout fix to increase the timeout for
> > > >> geode-benchmarks to 4h. This does not influence any geode modules.
> > > >>
> > > >> Regards
> > > >> Naba
> > > >>
> > > >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
> amurm...@apache.org
> > >
> > > >> wrote:
> > > >>
> > > >> > Hi folks,
> > > >> >
> > > >> > Our CI is very red since ~24 hours
> > > >> > <
> > > >> >
> > > >>
> > >
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> > > >> > >.
> > > >> > It looks like a substantial new issue was introduced.
> > > >> >
> > > >> > Can we hold off on merging new changes to the develop branch till
> > this
> > > >> > issue is resolved?
> > > >> >
> > > >> > Thank you all!
> > > >> >
> > > >>
> > > >
> > >
> >
>

Re: Very red CI -> Hold merges, please

[Kirk Lund]

The UpgradeTest failures on your latest commit for this PR are
WANRollingUpgradeNewSenderProcessOldEvent which seems to be a reoccurrence
of [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967). I
recommend having Gester take a look at that these failures. He marked
[GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967) as resolved
on Jan 9th.

On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:

> No worries. I think I have a better fix now. At least the builds are moving
> again.
>
> On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:
>
> > Sorry, go ahead and revert the commit and reopen the PR.
> >
> > On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:
> >
> > > I was still working on a fix...
> > >
> > > On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
> > >
> > > > I merged it in.
> > > >
> > > > On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:
> > > >
> > > > > I think we should go ahead and merge in
> > > > > https://github.com/apache/geode/pull/3172 since it resolves the
> > > > > GfshConsoleModeUnitTest UnitTest failures.
> > > > >
> > > > > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 
> wrote:
> > > > >
> > > > >> FYI, I have just merged a ci timeout fix to increase the timeout
> for
> > > > >> geode-benchmarks to 4h. This does not influence any geode modules.
> > > > >>
> > > > >> Regards
> > > > >> Naba
> > > > >>
> > > > >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
> > amurm...@apache.org
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi folks,
> > > > >> >
> > > > >> > Our CI is very red since ~24 hours
> > > > >> > <
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> > > > >> > >.
> > > > >> > It looks like a substantial new issue was introduced.
> > > > >> >
> > > > >> > Can we hold off on merging new changes to the develop branch
> till
> > > this
> > > > >> > issue is resolved?
> > > > >> >
> > > > >> > Thank you all!
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Re: Very red CI -> Hold merges, please

[Alexander Murmann]

Bruce, would it make sense to for now revert the suspect change to the
test? At that point we should be back to full green and we all can without
a doubt go back to our usual flow of merging to develop.

Thoughts?

On Thu, Feb 7, 2019 at 2:37 PM Kirk Lund  wrote:

> Hmm, and that was another false search hit in Jira! Searching for
> WANRollingUpgradeNewSenderProcessOldEvent in Jira brings up GEODE-3967
> which apparently does NOT involve that test. So, maybe we found another
> flaky test.
>
> Jira search seems to not work very well.
>
> On Thu, Feb 7, 2019 at 2:24 PM Kirk Lund  wrote:
>
> > The UpgradeTest failures on your latest commit for this PR are
> > WANRollingUpgradeNewSenderProcessOldEvent which seems to be a
> reoccurrence
> > of [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967). I
> > recommend having Gester take a look at that these failures. He marked
> > [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967) as
> > resolved on Jan 9th.
> >
> > On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:
> >
> >> No worries. I think I have a better fix now. At least the builds are
> >> moving
> >> again.
> >>
> >> On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:
> >>
> >> > Sorry, go ahead and revert the commit and reopen the PR.
> >> >
> >> > On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:
> >> >
> >> > > I was still working on a fix...
> >> > >
> >> > > On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
> >> > >
> >> > > > I merged it in.
> >> > > >
> >> > > > On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund 
> wrote:
> >> > > >
> >> > > > > I think we should go ahead and merge in
> >> > > > > https://github.com/apache/geode/pull/3172 since it resolves the
> >> > > > > GfshConsoleModeUnitTest UnitTest failures.
> >> > > > >
> >> > > > > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 
> >> wrote:
> >> > > > >
> >> > > > >> FYI, I have just merged a ci timeout fix to increase the
> timeout
> >> for
> >> > > > >> geode-benchmarks to 4h. This does not influence any geode
> >> modules.
> >> > > > >>
> >> > > > >> Regards
> >> > > > >> Naba
> >> > > > >>
> >> > > > >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
> >> > amurm...@apache.org
> >> > > >
> >> > > > >> wrote:
> >> > > > >>
> >> > > > >> > Hi folks,
> >> > > > >> >
> >> > > > >> > Our CI is very red since ~24 hours
> >> > > > >> > <
> >> > > > >> >
> >> > > > >>
> >> > > >
> >> > >
> >> >
> >>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> >> > > > >> > >.
> >> > > > >> > It looks like a substantial new issue was introduced.
> >> > > > >> >
> >> > > > >> > Can we hold off on merging new changes to the develop branch
> >> till
> >> > > this
> >> > > > >> > issue is resolved?
> >> > > > >> >
> >> > > > >> > Thank you all!
> >> > > > >> >
> >> > > > >>
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
>

Re: Very red CI -> Hold merges, please

[Alexander Murmann]

Awesome! Things look a lot better now. It seems like the only consistent
issue now is in DistributedTestOpenJDK11 where the LocatorDUnitTest
`testMultipleLocatorsRestartingAtSameTimeWithMissingServers` has been
failing since 2 days or so. Is anyone looking at that?

On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:

> No worries. I think I have a better fix now. At least the builds are moving
> again.
>
> On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:
>
> > Sorry, go ahead and revert the commit and reopen the PR.
> >
> > On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:
> >
> > > I was still working on a fix...
> > >
> > > On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
> > >
> > > > I merged it in.
> > > >
> > > > On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund  wrote:
> > > >
> > > > > I think we should go ahead and merge in
> > > > > https://github.com/apache/geode/pull/3172 since it resolves the
> > > > > GfshConsoleModeUnitTest UnitTest failures.
> > > > >
> > > > > On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 
> wrote:
> > > > >
> > > > >> FYI, I have just merged a ci timeout fix to increase the timeout
> for
> > > > >> geode-benchmarks to 4h. This does not influence any geode modules.
> > > > >>
> > > > >> Regards
> > > > >> Naba
> > > > >>
> > > > >> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
> > amurm...@apache.org
> > > >
> > > > >> wrote:
> > > > >>
> > > > >> > Hi folks,
> > > > >> >
> > > > >> > Our CI is very red since ~24 hours
> > > > >> > <
> > > > >> >
> > > > >>
> > > >
> > >
> >
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> > > > >> > >.
> > > > >> > It looks like a substantial new issue was introduced.
> > > > >> >
> > > > >> > Can we hold off on merging new changes to the develop branch
> till
> > > this
> > > > >> > issue is resolved?
> > > > >> >
> > > > >> > Thank you all!
> > > > >> >
> > > > >>
> > > > >
> > > >
> > >
> >
>

Adding integrationTest src set to geode-dunit

[Kirk Lund]

The usual geode src sets like integrationTest don't already exist in some
modules such as geode-dunit.

I'm trying to write a new IntegrationTest but simply creating the
directories and placing a new .java file in it doesn't seem to work.

geode-dunit/src/integrationTest/java/org/apache/geode/test/junit/rules/DiskDirRuleIntegrationTest.java

I've looked at geode-dunit/build.gradle and it's not clear what to add
there if anything. The only src sets currently in geode-dunit are:

geode-dunit/src/distributedTest
geode-dunit/src/main
geode-dunit/src/test

How do I add integrationTest so that it a) compiles, b) is added to my IJ
project, and 3) actually runs in CI and precheckin?

Re: Adding integrationTest src set to geode-dunit

[Robert Houghton]

Can I work with you on this tomorrow?

On Thu, Feb 7, 2019, 15:09 Kirk Lund  wrote:

> The usual geode src sets like integrationTest don't already exist in some
> modules such as geode-dunit.
>
> I'm trying to write a new IntegrationTest but simply creating the
> directories and placing a new .java file in it doesn't seem to work.
>
>
> geode-dunit/src/integrationTest/java/org/apache/geode/test/junit/rules/DiskDirRuleIntegrationTest.java
>
> I've looked at geode-dunit/build.gradle and it's not clear what to add
> there if anything. The only src sets currently in geode-dunit are:
>
> geode-dunit/src/distributedTest
> geode-dunit/src/main
> geode-dunit/src/test
>
> How do I add integrationTest so that it a) compiles, b) is added to my IJ
> project, and 3) actually runs in CI and precheckin?
>

Re: Very red CI -> Hold merges, please

[Owen Nichols]

Pipeline is back to green now.  Thank you to everyone who stepped up to get 
things back on track. 

If you had PR checks fail this week, please re-trigger them (by making an empty 
commit).

> On Feb 7, 2019, at 4:20 PM, Alexander Murmann  wrote:
> 
> Bruce, would it make sense to for now revert the suspect change to the
> test? At that point we should be back to full green and we all can without
> a doubt go back to our usual flow of merging to develop.
> 
> Thoughts?
> 
> On Thu, Feb 7, 2019 at 2:37 PM Kirk Lund  wrote:
> 
>> Hmm, and that was another false search hit in Jira! Searching for
>> WANRollingUpgradeNewSenderProcessOldEvent in Jira brings up GEODE-3967
>> which apparently does NOT involve that test. So, maybe we found another
>> flaky test.
>> 
>> Jira search seems to not work very well.
>> 
>> On Thu, Feb 7, 2019 at 2:24 PM Kirk Lund  wrote:
>> 
>>> The UpgradeTest failures on your latest commit for this PR are
>>> WANRollingUpgradeNewSenderProcessOldEvent which seems to be a
>> reoccurrence
>>> of [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967). I
>>> recommend having Gester take a look at that these failures. He marked
>>> [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967) as
>>> resolved on Jan 9th.
>>> 
>>> On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:
>>> 
 No worries. I think I have a better fix now. At least the builds are
 moving
 again.
 
 On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:
 
> Sorry, go ahead and revert the commit and reopen the PR.
> 
> On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe  wrote:
> 
>> I was still working on a fix...
>> 
>> On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
>> 
>>> I merged it in.
>>> 
>>> On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund 
>> wrote:
>>> 
 I think we should go ahead and merge in
 https://github.com/apache/geode/pull/3172 since it resolves the
 GfshConsoleModeUnitTest UnitTest failures.
 
 On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 
 wrote:
 
> FYI, I have just merged a ci timeout fix to increase the
>> timeout
 for
> geode-benchmarks to 4h. This does not influence any geode
 modules.
> 
> Regards
> Naba
> 
> On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
> amurm...@apache.org
>>> 
> wrote:
> 
>> Hi folks,
>> 
>> Our CI is very red since ~24 hours
>> <
>> 
> 
>>> 
>> 
> 
 
>> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
>>> .
>> It looks like a substantial new issue was introduced.
>> 
>> Can we hold off on merging new changes to the develop branch
 till
>> this
>> issue is resolved?
>> 
>> Thank you all!
>> 
> 
 
>>> 
>> 
> 
 
>>> 
>> 

Re: Very red CI -> Hold merges, please

[Xiaojian Zhou]

WANRollingUpgradeNewSenderProcessOldEvent is not related with GEODE-3967.
I wonder why search guided us to GEODE-3967.

Regards
Gester

On Thu, Feb 7, 2019 at 8:34 PM Owen Nichols  wrote:

> Pipeline is back to green now.  Thank you to everyone who stepped up to
> get things back on track.
>
> If you had PR checks fail this week, please re-trigger them (by making an
> empty commit).
>
> > On Feb 7, 2019, at 4:20 PM, Alexander Murmann 
> wrote:
> >
> > Bruce, would it make sense to for now revert the suspect change to the
> > test? At that point we should be back to full green and we all can
> without
> > a doubt go back to our usual flow of merging to develop.
> >
> > Thoughts?
> >
> > On Thu, Feb 7, 2019 at 2:37 PM Kirk Lund  wrote:
> >
> >> Hmm, and that was another false search hit in Jira! Searching for
> >> WANRollingUpgradeNewSenderProcessOldEvent in Jira brings up GEODE-3967
> >> which apparently does NOT involve that test. So, maybe we found another
> >> flaky test.
> >>
> >> Jira search seems to not work very well.
> >>
> >> On Thu, Feb 7, 2019 at 2:24 PM Kirk Lund  wrote:
> >>
> >>> The UpgradeTest failures on your latest commit for this PR are
> >>> WANRollingUpgradeNewSenderProcessOldEvent which seems to be a
> >> reoccurrence
> >>> of [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967). I
> >>> recommend having Gester take a look at that these failures. He marked
> >>> [GEODE-3967](https://issues.apache.org/jira/browse/GEODE-3967) as
> >>> resolved on Jan 9th.
> >>>
> >>> On Thu, Feb 7, 2019 at 12:37 PM Jens Deppe  wrote:
> >>>
>  No worries. I think I have a better fix now. At least the builds are
>  moving
>  again.
> 
>  On Thu, Feb 7, 2019 at 12:11 PM Kirk Lund  wrote:
> 
> > Sorry, go ahead and revert the commit and reopen the PR.
> >
> > On Thu, Feb 7, 2019 at 11:36 AM Jens Deppe 
> wrote:
> >
> >> I was still working on a fix...
> >>
> >> On Thu, Feb 7, 2019 at 11:31 AM Kirk Lund  wrote:
> >>
> >>> I merged it in.
> >>>
> >>> On Thu, Feb 7, 2019 at 11:28 AM Kirk Lund 
> >> wrote:
> >>>
>  I think we should go ahead and merge in
>  https://github.com/apache/geode/pull/3172 since it resolves the
>  GfshConsoleModeUnitTest UnitTest failures.
> 
>  On Thu, Feb 7, 2019 at 9:57 AM Nabarun Nag 
>  wrote:
> 
> > FYI, I have just merged a ci timeout fix to increase the
> >> timeout
>  for
> > geode-benchmarks to 4h. This does not influence any geode
>  modules.
> >
> > Regards
> > Naba
> >
> > On Thu, Feb 7, 2019 at 9:32 AM Alexander Murmann <
> > amurm...@apache.org
> >>>
> > wrote:
> >
> >> Hi folks,
> >>
> >> Our CI is very red since ~24 hours
> >> <
> >>
> >
> >>>
> >>
> >
> 
> >>
> https://concourse.apachegeode-ci.info/teams/main/pipelines/apache-develop-main/jobs/UnitTestOpenJDK11/builds/372
> >>> .
> >> It looks like a substantial new issue was introduced.
> >>
> >> Can we hold off on merging new changes to the develop branch
>  till
> >> this
> >> issue is resolved?
> >>
> >> Thank you all!
> >>
> >
> 
> >>>
> >>
> >
> 
> >>>
> >>
>
>

Re: GfshConsoleModeUnitTest is failing in UnitTest

[Jens Deppe]

The problem is caused by some change in test ordering (unit tests are
running in parallel) which is resulting in environmental pollution.

Probably Government shutdown related... :P

On Wed, Feb 6, 2019 at 7:31 PM Jens Deppe  wrote:

> I didn't touch anything in this area, but nevertheless I reverted my
> commit which preceded this failure (referenced in Owen's message). However,
> the failure is still happening.
>
> --Jens
>
> On Wed, Feb 6, 2019 at 6:20 PM Owen Nichols  wrote:
>
>> There is a ticket for this,
>> https://issues.apache.org/jira/browse/GEODE-6370
>>
>> The problem “seems" to have started with this change:
>>
>> commit 7f10f3a297eeeb473c3f16fff9465df9b404a21c
>> Date:   Wed Feb 6 08:33:53 2019 -0800
>>
>> GEODE-6364: Deploy of invalid jar file does not write file contents
>> to config on locator (#3164)
>>
>> However, the unit test failure we’re seeing consistently now never
>> appeared in the precheckin runs for PR 3164...
>>
>>
>> > On Feb 6, 2019, at 4:20 PM, Kirk Lund  wrote:
>> >
>> > Did someone break unit tests?
>> >
>> > I have 3 PRs with unit tests failing in GfshConsoleModeUnitTest:
>> >
>> > org.apache.geode.management.internal.cli.shell.GfshConsoleModeUnitTest >
>> > consoleModeShouldRedirectOnlyJDKLoggers FAILED
>> >java.lang.AssertionError:
>> >Expecting:
>> >  <"org.apache.geode.management.internal.cli.LogWrapper">
>> >not to end with:
>> >  <"LogWrapper">
>> >at
>> >
>> org.apache.geode.management.internal.cli.shell.GfshConsoleModeUnitTest.consoleModeShouldRedirectOnlyJDKLoggers(GfshConsoleModeUnitTest.java:54)
>> >
>> > Does anyone know which commit broke this test? Can we please revert that
>> > commit?
>>
>>

Re: GfshConsoleModeUnitTest is failing in UnitTest

[Jens Deppe]

Oh, feel free to review https://github.com/apache/geode/pull/3172.

Better suggestions on how to 'fix' the problem are most welcome.

On Thu, Feb 7, 2019 at 6:49 AM Jens Deppe  wrote:

> The problem is caused by some change in test ordering (unit tests are
> running in parallel) which is resulting in environmental pollution.
>
> Probably Government shutdown related... :P
>
> On Wed, Feb 6, 2019 at 7:31 PM Jens Deppe  wrote:
>
>> I didn't touch anything in this area, but nevertheless I reverted my
>> commit which preceded this failure (referenced in Owen's message). However,
>> the failure is still happening.
>>
>> --Jens
>>
>> On Wed, Feb 6, 2019 at 6:20 PM Owen Nichols  wrote:
>>
>>> There is a ticket for this,
>>> https://issues.apache.org/jira/browse/GEODE-6370
>>>
>>> The problem “seems" to have started with this change:
>>>
>>> commit 7f10f3a297eeeb473c3f16fff9465df9b404a21c
>>> Date:   Wed Feb 6 08:33:53 2019 -0800
>>>
>>> GEODE-6364: Deploy of invalid jar file does not write file contents
>>> to config on locator (#3164)
>>>
>>> However, the unit test failure we’re seeing consistently now never
>>> appeared in the precheckin runs for PR 3164...
>>>
>>>
>>> > On Feb 6, 2019, at 4:20 PM, Kirk Lund  wrote:
>>> >
>>> > Did someone break unit tests?
>>> >
>>> > I have 3 PRs with unit tests failing in GfshConsoleModeUnitTest:
>>> >
>>> > org.apache.geode.management.internal.cli.shell.GfshConsoleModeUnitTest
>>> >
>>> > consoleModeShouldRedirectOnlyJDKLoggers FAILED
>>> >java.lang.AssertionError:
>>> >Expecting:
>>> >  <"org.apache.geode.management.internal.cli.LogWrapper">
>>> >not to end with:
>>> >  <"LogWrapper">
>>> >at
>>> >
>>> org.apache.geode.management.internal.cli.shell.GfshConsoleModeUnitTest.consoleModeShouldRedirectOnlyJDKLoggers(GfshConsoleModeUnitTest.java:54)
>>> >
>>> > Does anyone know which commit broke this test? Can we please revert
>>> that
>>> > commit?
>>>
>>>