Re: [DISCUSS] Clean build takes 10minutes to complete now

[Udo Kohlmeyer]

I think that the moving the old version out of the compilation step is
paramount. Could the download of the artifact be made a dependency of the
BackwardCompatibilityTest?

--Udo

On Fri, Sep 15, 2017 at 1:08 PM, Jason Huynh  wrote:

> For the original issue, where the old version is pulling down the old
> versions during compilation time, we have a pull request to use the maven
> repo: https://github.com/apache/geode/pull/790
>
> The rest of the tests added for the session state are marked as @Category
> ({DistributedTest.class, BackwardCompatibilityTest.class})
>  geode-old-versions happens to be required at compile time.
>
> The old versions will now (after the pull request is checked in) pull down
> the zip into the local repo once.  Unzipping will still be required but
> that should be a lot shorter than downloading.
>
> Whether we should move the old versions out of the compile task is a
> different issue...
>
>
>
> On Fri, Sep 15, 2017 at 8:55 AM Kirk Lund  wrote:
>
> > The actual tests marked with UnitTest category are actually pretty good.
> > They all run in just over one minute and almost all of them use Mockito
> to
> > isolate one class. I remember seeing one newer Lucene UnitTest that
> touches
> > File System which should be recategorized as IntegrationTest.
> >
> > If we could move the pulling down of previous versions of Geode out of
> the
> > main build+unit-test target, that would help a lot.
> >
> > Even prior to the pulling down of previous versions for backwards compat
> > testing, the main build (without unit-test) was too slow and I think it's
> > because our project is a little too complex for what Gradle is designed
> to
> > handle.
> >
> > Code generation and javadocs are two of the tasks in our main build
> > (without unit-test) that contributes to it taking too long.
> >
> > Also, the way Gradle handles junit categories is designed and coded very
> > inefficiently -- if we could change their junit runner to use
> > FastClasspathScanner to find all tests containing the targeted junit
> > category annotation then that would speed up all of our testing targets
> > immensely. Any testing target that forks JVMs runs super slow due to the
> > way they handle categories (this effects IntegrationTest,
> DistributedTest,
> > FlakyTest).
> >
> > On Fri, Sep 15, 2017 at 8:44 AM, Alexander Murmann 
> > wrote:
> >
> > > I fully agree with Udo here. The main build should be for Unit tests.
> Our
> > > "Unit Tests" are already exercising much more of the system than they
> > > should. Adding unit tests that not only too much or our current code
> but
> > > also old code is moving us in the wrong direction. Let's keep the
> tests,
> > > but please appropriately mark them as IntegrationTest.
> > >
> > > On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
> > > wrote:
> > >
> > > > My apologies, I might gotten the commit reason incorrect. I just know
> > > that
> > > > downloading the older product version every time is becoming painful.
> > > > Yes, sometimes it is faster than other times, but imo, this is not
> > > > something that should be part of the main build path.
> > > >
> > > > Backwards compat or integration testing should not be running as part
> > of
> > > > the main build task.
> > > >
> > > > --Udo
> > > >
> > > > On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag 
> wrote:
> > > >
> > > > > As we are working on fixing this issue, some extra parameters may
> > help
> > > > the
> > > > > build to get bit quicker on your machine.
> > > > >
> > > > > using -xjavadoc -xdoc
> > > > > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > > > > BUILD SUCCESSFUL
> > > > > Total time: 2 mins 2.729 secs
> > > > >
> > > > >
> > > > > Also, I think as Jason mentioned that the slow down is due to full
> > > > product
> > > > > download for session state tests. LuceneSearchWithRollingUpgrade
> DUnit
> > > > > tests
> > > > > were added  in July. Please do correct me if I am wrong.
> > > > >
> > > > > Regards
> > > > > Nabarun
> > > > >
> > > > >
> > > > > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann <
> > > amurm...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > Could we make it so that these tests for now are only run as part
> > of
> > > > > > pre-checkin till we got this ironed out and then revisit this?
> > > > > >
> > > > > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > > > > bschucha...@pivotal.io>
> > > > > > wrote:
> > > > > >
> > > > > > > The geode-old-versions module was originally created to pull in
> > old
> > > > > > > version jar files into your gradle cache.  This happened only
> > once
> > > > and
> > > > > > you
> > > > > > > were good to go.  I don't think that part should be backed out
> as
> > > it
> > > > > has
> > > > > > > minimal impact and is not affecting build time.
> > > > > > >
> > > > > > > The recent changes for lucene testing seem to be 

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Jason Huynh]

For the original issue, where the old version is pulling down the old
versions during compilation time, we have a pull request to use the maven
repo: https://github.com/apache/geode/pull/790

The rest of the tests added for the session state are marked as @Category
({DistributedTest.class, BackwardCompatibilityTest.class})
 geode-old-versions happens to be required at compile time.

The old versions will now (after the pull request is checked in) pull down
the zip into the local repo once.  Unzipping will still be required but
that should be a lot shorter than downloading.

Whether we should move the old versions out of the compile task is a
different issue...



On Fri, Sep 15, 2017 at 8:55 AM Kirk Lund  wrote:

> The actual tests marked with UnitTest category are actually pretty good.
> They all run in just over one minute and almost all of them use Mockito to
> isolate one class. I remember seeing one newer Lucene UnitTest that touches
> File System which should be recategorized as IntegrationTest.
>
> If we could move the pulling down of previous versions of Geode out of the
> main build+unit-test target, that would help a lot.
>
> Even prior to the pulling down of previous versions for backwards compat
> testing, the main build (without unit-test) was too slow and I think it's
> because our project is a little too complex for what Gradle is designed to
> handle.
>
> Code generation and javadocs are two of the tasks in our main build
> (without unit-test) that contributes to it taking too long.
>
> Also, the way Gradle handles junit categories is designed and coded very
> inefficiently -- if we could change their junit runner to use
> FastClasspathScanner to find all tests containing the targeted junit
> category annotation then that would speed up all of our testing targets
> immensely. Any testing target that forks JVMs runs super slow due to the
> way they handle categories (this effects IntegrationTest, DistributedTest,
> FlakyTest).
>
> On Fri, Sep 15, 2017 at 8:44 AM, Alexander Murmann 
> wrote:
>
> > I fully agree with Udo here. The main build should be for Unit tests. Our
> > "Unit Tests" are already exercising much more of the system than they
> > should. Adding unit tests that not only too much or our current code but
> > also old code is moving us in the wrong direction. Let's keep the tests,
> > but please appropriately mark them as IntegrationTest.
> >
> > On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
> > wrote:
> >
> > > My apologies, I might gotten the commit reason incorrect. I just know
> > that
> > > downloading the older product version every time is becoming painful.
> > > Yes, sometimes it is faster than other times, but imo, this is not
> > > something that should be part of the main build path.
> > >
> > > Backwards compat or integration testing should not be running as part
> of
> > > the main build task.
> > >
> > > --Udo
> > >
> > > On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:
> > >
> > > > As we are working on fixing this issue, some extra parameters may
> help
> > > the
> > > > build to get bit quicker on your machine.
> > > >
> > > > using -xjavadoc -xdoc
> > > > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > > > BUILD SUCCESSFUL
> > > > Total time: 2 mins 2.729 secs
> > > >
> > > >
> > > > Also, I think as Jason mentioned that the slow down is due to full
> > > product
> > > > download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> > > > tests
> > > > were added  in July. Please do correct me if I am wrong.
> > > >
> > > > Regards
> > > > Nabarun
> > > >
> > > >
> > > > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann <
> > amurm...@pivotal.io>
> > > > wrote:
> > > >
> > > > > Could we make it so that these tests for now are only run as part
> of
> > > > > pre-checkin till we got this ironed out and then revisit this?
> > > > >
> > > > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > > > bschucha...@pivotal.io>
> > > > > wrote:
> > > > >
> > > > > > The geode-old-versions module was originally created to pull in
> old
> > > > > > version jar files into your gradle cache.  This happened only
> once
> > > and
> > > > > you
> > > > > > were good to go.  I don't think that part should be backed out as
> > it
> > > > has
> > > > > > minimal impact and is not affecting build time.
> > > > > >
> > > > > > The recent changes for lucene testing seem to be pulling in full
> > > > > > installations of old versions and these are deleted as part of
> the
> > > > > "clean"
> > > > > > gradle task.  That's causing them to be downloaded again each
> time
> > > you
> > > > > do a
> > > > > > clean  Dan put changes in place so that the files aren't
> > > > > downloaded
> > > > > > again if you build without cleaning but clearly more needs to be
> > done
> > > > in
> > > > > > this area.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On 9/11/17 11:23 AM, 

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Kirk Lund]

The actual tests marked with UnitTest category are actually pretty good.
They all run in just over one minute and almost all of them use Mockito to
isolate one class. I remember seeing one newer Lucene UnitTest that touches
File System which should be recategorized as IntegrationTest.

If we could move the pulling down of previous versions of Geode out of the
main build+unit-test target, that would help a lot.

Even prior to the pulling down of previous versions for backwards compat
testing, the main build (without unit-test) was too slow and I think it's
because our project is a little too complex for what Gradle is designed to
handle.

Code generation and javadocs are two of the tasks in our main build
(without unit-test) that contributes to it taking too long.

Also, the way Gradle handles junit categories is designed and coded very
inefficiently -- if we could change their junit runner to use
FastClasspathScanner to find all tests containing the targeted junit
category annotation then that would speed up all of our testing targets
immensely. Any testing target that forks JVMs runs super slow due to the
way they handle categories (this effects IntegrationTest, DistributedTest,
FlakyTest).

On Fri, Sep 15, 2017 at 8:44 AM, Alexander Murmann 
wrote:

> I fully agree with Udo here. The main build should be for Unit tests. Our
> "Unit Tests" are already exercising much more of the system than they
> should. Adding unit tests that not only too much or our current code but
> also old code is moving us in the wrong direction. Let's keep the tests,
> but please appropriately mark them as IntegrationTest.
>
> On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
> wrote:
>
> > My apologies, I might gotten the commit reason incorrect. I just know
> that
> > downloading the older product version every time is becoming painful.
> > Yes, sometimes it is faster than other times, but imo, this is not
> > something that should be part of the main build path.
> >
> > Backwards compat or integration testing should not be running as part of
> > the main build task.
> >
> > --Udo
> >
> > On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:
> >
> > > As we are working on fixing this issue, some extra parameters may help
> > the
> > > build to get bit quicker on your machine.
> > >
> > > using -xjavadoc -xdoc
> > > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > > BUILD SUCCESSFUL
> > > Total time: 2 mins 2.729 secs
> > >
> > >
> > > Also, I think as Jason mentioned that the slow down is due to full
> > product
> > > download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> > > tests
> > > were added  in July. Please do correct me if I am wrong.
> > >
> > > Regards
> > > Nabarun
> > >
> > >
> > > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann <
> amurm...@pivotal.io>
> > > wrote:
> > >
> > > > Could we make it so that these tests for now are only run as part of
> > > > pre-checkin till we got this ironed out and then revisit this?
> > > >
> > > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > > bschucha...@pivotal.io>
> > > > wrote:
> > > >
> > > > > The geode-old-versions module was originally created to pull in old
> > > > > version jar files into your gradle cache.  This happened only once
> > and
> > > > you
> > > > > were good to go.  I don't think that part should be backed out as
> it
> > > has
> > > > > minimal impact and is not affecting build time.
> > > > >
> > > > > The recent changes for lucene testing seem to be pulling in full
> > > > > installations of old versions and these are deleted as part of the
> > > > "clean"
> > > > > gradle task.  That's causing them to be downloaded again each time
> > you
> > > > do a
> > > > > clean  Dan put changes in place so that the files aren't
> > > > downloaded
> > > > > again if you build without cleaning but clearly more needs to be
> done
> > > in
> > > > > this area.
> > > > >
> > > > >
> > > > >
> > > > > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> > > > >
> > > > >> Agreed, integration tests should not be part of the build process.
> > > This
> > > > >> is clearly an integration test.
> > > > >>
> > > > >> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer 
> wrote:
> > > > >>>
> > > > >>> Hi there,
> > > > >>>
> > > > >>> With a recent addition to the build scripts, to test lucene
> > backwards
> > > > >>> compatibility, a step was added to download a previous version of
> > > > GEODE.
> > > > >>>
> > > > >>> This is causing longer build times now, which is a real
> > distraction.
> > > In
> > > > >>> cases where one would like to work on a branch, rebase that on
> > > develop
> > > > and
> > > > >>> merge that, this step becomes a real time hog.
> > > > >>>
> > > > >>> I request that we remove this default behavior from a clean build
> > > until
> > > > >>> we have a better solution to this issue.
> > > > >>>
> > > > >>> I also believe that if anyone wants 

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Alexander Murmann]

I fully agree with Udo here. The main build should be for Unit tests. Our
"Unit Tests" are already exercising much more of the system than they
should. Adding unit tests that not only too much or our current code but
also old code is moving us in the wrong direction. Let's keep the tests,
but please appropriately mark them as IntegrationTest.

On Tue, Sep 12, 2017 at 9:30 AM, Udo Kohlmeyer 
wrote:

> My apologies, I might gotten the commit reason incorrect. I just know that
> downloading the older product version every time is becoming painful.
> Yes, sometimes it is faster than other times, but imo, this is not
> something that should be part of the main build path.
>
> Backwards compat or integration testing should not be running as part of
> the main build task.
>
> --Udo
>
> On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:
>
> > As we are working on fixing this issue, some extra parameters may help
> the
> > build to get bit quicker on your machine.
> >
> > using -xjavadoc -xdoc
> > Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> > BUILD SUCCESSFUL
> > Total time: 2 mins 2.729 secs
> >
> >
> > Also, I think as Jason mentioned that the slow down is due to full
> product
> > download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> > tests
> > were added  in July. Please do correct me if I am wrong.
> >
> > Regards
> > Nabarun
> >
> >
> > On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann 
> > wrote:
> >
> > > Could we make it so that these tests for now are only run as part of
> > > pre-checkin till we got this ironed out and then revisit this?
> > >
> > > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> > bschucha...@pivotal.io>
> > > wrote:
> > >
> > > > The geode-old-versions module was originally created to pull in old
> > > > version jar files into your gradle cache.  This happened only once
> and
> > > you
> > > > were good to go.  I don't think that part should be backed out as it
> > has
> > > > minimal impact and is not affecting build time.
> > > >
> > > > The recent changes for lucene testing seem to be pulling in full
> > > > installations of old versions and these are deleted as part of the
> > > "clean"
> > > > gradle task.  That's causing them to be downloaded again each time
> you
> > > do a
> > > > clean  Dan put changes in place so that the files aren't
> > > downloaded
> > > > again if you build without cleaning but clearly more needs to be done
> > in
> > > > this area.
> > > >
> > > >
> > > >
> > > > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> > > >
> > > >> Agreed, integration tests should not be part of the build process.
> > This
> > > >> is clearly an integration test.
> > > >>
> > > >> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> > > >>>
> > > >>> Hi there,
> > > >>>
> > > >>> With a recent addition to the build scripts, to test lucene
> backwards
> > > >>> compatibility, a step was added to download a previous version of
> > > GEODE.
> > > >>>
> > > >>> This is causing longer build times now, which is a real
> distraction.
> > In
> > > >>> cases where one would like to work on a branch, rebase that on
> > develop
> > > and
> > > >>> merge that, this step becomes a real time hog.
> > > >>>
> > > >>> I request that we remove this default behavior from a clean build
> > until
> > > >>> we have a better solution to this issue.
> > > >>>
> > > >>> I also believe that if anyone wants to add behavior like this into
> > the
> > > >>> default build, that it at least is discussed on the dev list before
> > > >>> implementing this.
> > > >>>
> > > >>> --Udo
> > > >>>
> > > >>>
> > > >
> > >
> >
>
>
>
> --
> Kindest Regards
> -
> *Udo Kohlmeyer* | *Snr Solutions Architect* |*Pivotal*
> *Mobile:* +61 409-279-160 | ukohlme...@pivotal.io
> 
> www.pivotal.io
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Udo Kohlmeyer]

My apologies, I might gotten the commit reason incorrect. I just know that
downloading the older product version every time is becoming painful.
Yes, sometimes it is faster than other times, but imo, this is not
something that should be part of the main build path.

Backwards compat or integration testing should not be running as part of
the main build task.

--Udo

On Tue, Sep 12, 2017 at 9:05 AM, Nabarun Nag  wrote:

> As we are working on fixing this issue, some extra parameters may help the
> build to get bit quicker on your machine.
>
> using -xjavadoc -xdoc
> Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
> BUILD SUCCESSFUL
> Total time: 2 mins 2.729 secs
>
>
> Also, I think as Jason mentioned that the slow down is due to full product
> download for session state tests. LuceneSearchWithRollingUpgradeDUnit
> tests
> were added  in July. Please do correct me if I am wrong.
>
> Regards
> Nabarun
>
>
> On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann 
> wrote:
>
> > Could we make it so that these tests for now are only run as part of
> > pre-checkin till we got this ironed out and then revisit this?
> >
> > On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt <
> bschucha...@pivotal.io>
> > wrote:
> >
> > > The geode-old-versions module was originally created to pull in old
> > > version jar files into your gradle cache.  This happened only once and
> > you
> > > were good to go.  I don't think that part should be backed out as it
> has
> > > minimal impact and is not affecting build time.
> > >
> > > The recent changes for lucene testing seem to be pulling in full
> > > installations of old versions and these are deleted as part of the
> > "clean"
> > > gradle task.  That's causing them to be downloaded again each time you
> > do a
> > > clean  Dan put changes in place so that the files aren't
> > downloaded
> > > again if you build without cleaning but clearly more needs to be done
> in
> > > this area.
> > >
> > >
> > >
> > > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> > >
> > >> Agreed, integration tests should not be part of the build process.
> This
> > >> is clearly an integration test.
> > >>
> > >> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> > >>>
> > >>> Hi there,
> > >>>
> > >>> With a recent addition to the build scripts, to test lucene backwards
> > >>> compatibility, a step was added to download a previous version of
> > GEODE.
> > >>>
> > >>> This is causing longer build times now, which is a real distraction.
> In
> > >>> cases where one would like to work on a branch, rebase that on
> develop
> > and
> > >>> merge that, this step becomes a real time hog.
> > >>>
> > >>> I request that we remove this default behavior from a clean build
> until
> > >>> we have a better solution to this issue.
> > >>>
> > >>> I also believe that if anyone wants to add behavior like this into
> the
> > >>> default build, that it at least is discussed on the dev list before
> > >>> implementing this.
> > >>>
> > >>> --Udo
> > >>>
> > >>>
> > >
> >
>



-- 
Kindest Regards
-
*Udo Kohlmeyer* | *Snr Solutions Architect* |*Pivotal*
*Mobile:* +61 409-279-160 | ukohlme...@pivotal.io

www.pivotal.io

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Nabarun Nag]

As we are working on fixing this issue, some extra parameters may help the
build to get bit quicker on your machine.

using -xjavadoc -xdoc
Eg: ./gradlew clean build -Dskip.tests=true -xjavadoc -xdocs
BUILD SUCCESSFUL
Total time: 2 mins 2.729 secs


Also, I think as Jason mentioned that the slow down is due to full product
download for session state tests. LuceneSearchWithRollingUpgradeDUnit tests
were added  in July. Please do correct me if I am wrong.

Regards
Nabarun


On Tue, Sep 12, 2017 at 11:47 AM Alexander Murmann 
wrote:

> Could we make it so that these tests for now are only run as part of
> pre-checkin till we got this ironed out and then revisit this?
>
> On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt 
> wrote:
>
> > The geode-old-versions module was originally created to pull in old
> > version jar files into your gradle cache.  This happened only once and
> you
> > were good to go.  I don't think that part should be backed out as it has
> > minimal impact and is not affecting build time.
> >
> > The recent changes for lucene testing seem to be pulling in full
> > installations of old versions and these are deleted as part of the
> "clean"
> > gradle task.  That's causing them to be downloaded again each time you
> do a
> > clean  Dan put changes in place so that the files aren't
> downloaded
> > again if you build without cleaning but clearly more needs to be done in
> > this area.
> >
> >
> >
> > On 9/11/17 11:23 AM, Jacob Barrett wrote:
> >
> >> Agreed, integration tests should not be part of the build process. This
> >> is clearly an integration test.
> >>
> >> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> >>>
> >>> Hi there,
> >>>
> >>> With a recent addition to the build scripts, to test lucene backwards
> >>> compatibility, a step was added to download a previous version of
> GEODE.
> >>>
> >>> This is causing longer build times now, which is a real distraction. In
> >>> cases where one would like to work on a branch, rebase that on develop
> and
> >>> merge that, this step becomes a real time hog.
> >>>
> >>> I request that we remove this default behavior from a clean build until
> >>> we have a better solution to this issue.
> >>>
> >>> I also believe that if anyone wants to add behavior like this into the
> >>> default build, that it at least is discussed on the dev list before
> >>> implementing this.
> >>>
> >>> --Udo
> >>>
> >>>
> >
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Alexander Murmann]

Could we make it so that these tests for now are only run as part of
pre-checkin till we got this ironed out and then revisit this?

On Tue, Sep 12, 2017 at 8:32 AM, Bruce Schuchardt 
wrote:

> The geode-old-versions module was originally created to pull in old
> version jar files into your gradle cache.  This happened only once and you
> were good to go.  I don't think that part should be backed out as it has
> minimal impact and is not affecting build time.
>
> The recent changes for lucene testing seem to be pulling in full
> installations of old versions and these are deleted as part of the "clean"
> gradle task.  That's causing them to be downloaded again each time you do a
> clean  Dan put changes in place so that the files aren't downloaded
> again if you build without cleaning but clearly more needs to be done in
> this area.
>
>
>
> On 9/11/17 11:23 AM, Jacob Barrett wrote:
>
>> Agreed, integration tests should not be part of the build process. This
>> is clearly an integration test.
>>
>> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
>>>
>>> Hi there,
>>>
>>> With a recent addition to the build scripts, to test lucene backwards
>>> compatibility, a step was added to download a previous version of GEODE.
>>>
>>> This is causing longer build times now, which is a real distraction. In
>>> cases where one would like to work on a branch, rebase that on develop and
>>> merge that, this step becomes a real time hog.
>>>
>>> I request that we remove this default behavior from a clean build until
>>> we have a better solution to this issue.
>>>
>>> I also believe that if anyone wants to add behavior like this into the
>>> default build, that it at least is discussed on the dev list before
>>> implementing this.
>>>
>>> --Udo
>>>
>>>
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Bruce Schuchardt]

The geode-old-versions module was originally created to pull in old 
version jar files into your gradle cache.  This happened only once and 
you were good to go.  I don't think that part should be backed out as it 
has minimal impact and is not affecting build time.


The recent changes for lucene testing seem to be pulling in full 
installations of old versions and these are deleted as part of the 
"clean" gradle task.  That's causing them to be downloaded again each 
time you do a clean  Dan put changes in place so that the files 
aren't downloaded again if you build without cleaning but clearly more 
needs to be done in this area.



On 9/11/17 11:23 AM, Jacob Barrett wrote:

Agreed, integration tests should not be part of the build process. This is 
clearly an integration test.


On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:

Hi there,

With a recent addition to the build scripts, to test lucene backwards 
compatibility, a step was added to download a previous version of GEODE.

This is causing longer build times now, which is a real distraction. In cases 
where one would like to work on a branch, rebase that on develop and merge 
that, this step becomes a real time hog.

I request that we remove this default behavior from a clean build until we have 
a better solution to this issue.

I also believe that if anyone wants to add behavior like this into the default 
build, that it at least is discussed on the dev list before implementing this.

--Udo

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Jens Deppe]

Right. I've added the full distribution to be uploaded as part of the
'uploadArchives' task. Here is how you can set up your gradle environment
to retrieve it.

Add the apache snapshot repo to your repositories:

repositories {
  maven {
url 'https://repository.apache.org/content/repositories/snapshots'
  }
}


Define the dependency:

dependencies {
  compile 'org.apache.geode:apache-geode:1.3.0-SNAPSHOT'
}


Then you can reference the actual file, from the gradle cache, with this
snippet:

def geodeDistZip = configurations.compile.find {
  it.path.contains('org.apache.geode/apache-geode')
}

Currently there is only a 1.3.0-SNAPSHOT build available.

--Jens


On Mon, Sep 11, 2017 at 4:25 PM, Jason Huynh  wrote:

> I was speaking with Jens and he is working on getting the full product
> install up to the apache maven repo.  We could then use this instead of
> downloading the zip files manually.  This would allow gradle to cache the
> full product install in the local repo (similar to the other jars that are
> being pulled down)
>
> I think this is referring to the download of the full product install for
> the Session State tests and not the Lucene tests?  I don't believe any of
> the session state tests are being run as part of the build and it is the
> download that is potentially taking a long time (which gets cleaned and
> downloaded after every clean build)
>
> The test themselves are marked as DistributedTest, however I don't think
> our gradle test task can use these annotations to fire off specific gradle
> tasks.  So it is currently lumped in with compiling geode-old-versions.
>
> Just for reference, my build is taking 8 minutes but the downloading of the
> product install is taking 30 seconds on my machine.  I don't think this
> will solve the entire 10 minute build...
>
>
>
>
>
>
> On Mon, Sep 11, 2017 at 11:23 AM Jacob Barrett 
> wrote:
>
> > Agreed, integration tests should not be part of the build process. This
> is
> > clearly an integration test.
> >
> > > On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> > >
> > > Hi there,
> > >
> > > With a recent addition to the build scripts, to test lucene backwards
> > compatibility, a step was added to download a previous version of GEODE.
> > >
> > > This is causing longer build times now, which is a real distraction. In
> > cases where one would like to work on a branch, rebase that on develop
> and
> > merge that, this step becomes a real time hog.
> > >
> > > I request that we remove this default behavior from a clean build until
> > we have a better solution to this issue.
> > >
> > > I also believe that if anyone wants to add behavior like this into the
> > default build, that it at least is discussed on the dev list before
> > implementing this.
> > >
> > > --Udo
> > >
> >
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Jason Huynh]

I was speaking with Jens and he is working on getting the full product
install up to the apache maven repo.  We could then use this instead of
downloading the zip files manually.  This would allow gradle to cache the
full product install in the local repo (similar to the other jars that are
being pulled down)

I think this is referring to the download of the full product install for
the Session State tests and not the Lucene tests?  I don't believe any of
the session state tests are being run as part of the build and it is the
download that is potentially taking a long time (which gets cleaned and
downloaded after every clean build)

The test themselves are marked as DistributedTest, however I don't think
our gradle test task can use these annotations to fire off specific gradle
tasks.  So it is currently lumped in with compiling geode-old-versions.

Just for reference, my build is taking 8 minutes but the downloading of the
product install is taking 30 seconds on my machine.  I don't think this
will solve the entire 10 minute build...






On Mon, Sep 11, 2017 at 11:23 AM Jacob Barrett  wrote:

> Agreed, integration tests should not be part of the build process. This is
> clearly an integration test.
>
> > On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> >
> > Hi there,
> >
> > With a recent addition to the build scripts, to test lucene backwards
> compatibility, a step was added to download a previous version of GEODE.
> >
> > This is causing longer build times now, which is a real distraction. In
> cases where one would like to work on a branch, rebase that on develop and
> merge that, this step becomes a real time hog.
> >
> > I request that we remove this default behavior from a clean build until
> we have a better solution to this issue.
> >
> > I also believe that if anyone wants to add behavior like this into the
> default build, that it at least is discussed on the dev list before
> implementing this.
> >
> > --Udo
> >
>

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Jacob Barrett]

Agreed, integration tests should not be part of the build process. This is 
clearly an integration test.

> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> 
> Hi there,
> 
> With a recent addition to the build scripts, to test lucene backwards 
> compatibility, a step was added to download a previous version of GEODE.
> 
> This is causing longer build times now, which is a real distraction. In cases 
> where one would like to work on a branch, rebase that on develop and merge 
> that, this step becomes a real time hog.
> 
> I request that we remove this default behavior from a clean build until we 
> have a better solution to this issue.
> 
> I also believe that if anyone wants to add behavior like this into the 
> default build, that it at least is discussed on the dev list before 
> implementing this.
> 
> --Udo
> 

Re: [DISCUSS] Clean build takes 10minutes to complete now

[Mark Hanson]

+1
> On Sep 11, 2017, at 11:00 AM, Udo Kohlmeyer  wrote:
> 
> Hi there,
> 
> With a recent addition to the build scripts, to test lucene backwards 
> compatibility, a step was added to download a previous version of GEODE.
> 
> This is causing longer build times now, which is a real distraction. In cases 
> where one would like to work on a branch, rebase that on develop and merge 
> that, this step becomes a real time hog.
> 
> I request that we remove this default behavior from a clean build until we 
> have a better solution to this issue.
> 
> I also believe that if anyone wants to add behavior like this into the 
> default build, that it at least is discussed on the dev list before 
> implementing this.
> 
> --Udo
> 

[DISCUSS] Clean build takes 10minutes to complete now

[Udo Kohlmeyer]

Hi there,

With a recent addition to the build scripts, to test lucene backwards 
compatibility, a step was added to download a previous version of GEODE.


This is causing longer build times now, which is a real distraction. In 
cases where one would like to work on a branch, rebase that on develop 
and merge that, this step becomes a real time hog.


I request that we remove this default behavior from a clean build until 
we have a better solution to this issue.


I also believe that if anyone wants to add behavior like this into the 
default build, that it at least is discussed on the dev list before 
implementing this.


--Udo