Re: [Rdkit-devel] travis build timeouts

2017-06-13 Thread Greg Landrum 
On Tue, Jun 13, 2017 at 3:24 PM, Brian Kelley  wrote:

> I'm unsure if that staging will help, aren't the builds and tests already
> done in parallel effectively?
>

Kind of. There are currently two jobs that run in parallel:
- job 1: build the core library, the python 2.7 wrappers, and the java
wrappers and test everything
- job 2: build the core library, the python 3.5 wrappers, and the java
wrappers and test everything

Those have no dependencies on each other. I would hope that by pulling the
core library and test builds out into a separate job and then using the
artifacts (=libraries) from that for the python and java builds we could
shorten the individual jobs.

-greg




> On Tue, Jun 13, 2017 at 9:20 AM, Greg Landrum 
> wrote:
>
>> Given that the current variability is almost a factor of two I don't
>> think smaller changes are going to help.
>>
>> What I'm likely going to try is doing staged builds:
>> - stage 1: build and test core library
>> - stage 2a: build and test python 2.7 wrappers
>> - stage 2b: build and test python 3.5 wrappers
>> - stage 2c: build and test java wrappers
>>
>> the three "stage 2" things can run in parallel.
>>
>> From the bit of reading I did on the train this morning, it looks like
>> this is not that horrible to do in jenkins; just need to invest a bit of
>> time.
>>
>> -greg
>>
>>
>> On Tue, Jun 13, 2017 at 3:12 PM, Brian Kelley 
>> wrote:
>>
>>> I tried using ninja for the builds in stead of make, after an evening of
>>> not building I successfully had two builds:
>>>
>>> https://travis-ci.org/bp-kelley/rdkit/builds/242399233
>>>
>>> Commit:
>>>
>>> https://github.com/bp-kelley/rdkit/blob/baac00bc1eb7f6641301
>>> 1aed239413fe74da3388/.travis.yml
>>>
>>> Note that the default ninja build is #cores + 2 which blew out the
>>> memory on the VMs so I used -j3 we could try this in the makefile as well.
>>>
>>> I also think we could shave off a few minutes by
>>>
>>>   1. building in the root conda env
>>>   2. having only one conda install for *everything*
>>>
>>> If you like I can try these fixes.
>>>
>>> Cheers,
>>>   Brian
>>>
>>> On Tue, Jun 13, 2017 at 3:44 AM, Greg Landrum 
>>> wrote:
>>>
 Dear all,

 We're getting more and more timeouts with RDKit builds on travis. I'm
 reasonably sure that this is due to some variability in the performance of
 the VMs that the builds end up running on. A good demonstration of that is
 this particular build (on a branch where I'm playing with simplifying the
 travis builds), which works fine for py35 but times out for py27:
 https://travis-ci.org/rdkit/rdkit/builds/242301061?utm_sourc
 e=email_medium=notification

 If you look into the details you'll see that the main compile "only"
 takes 1475 seconds for the python 3.5 build but it is killed after 2541
 seconds in the python2.7 build. Given that it's the same code being built,
 it seems like this more or less has to be infrastructure variability.

 I'll continue to explore, particularly breaking the build into stages,
 but if anyone has suggestions on how to improve this please let me know.

 -greg


 
 --
 Check out the vibrant tech community on one of the world's most
 engaging tech sites, Slashdot.org! http://sdm.link/slashdot
 ___
 Rdkit-devel mailing list
 Rdkit-devel@lists.sourceforge.net
 https://lists.sourceforge.net/lists/listinfo/rdkit-devel


>>>
>>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Rdkit-devel mailing list
Rdkit-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rdkit-devel

Re: [Rdkit-devel] travis build timeouts

2017-06-13 Thread Greg Landrum 
Given that the current variability is almost a factor of two I don't think
smaller changes are going to help.

What I'm likely going to try is doing staged builds:
- stage 1: build and test core library
- stage 2a: build and test python 2.7 wrappers
- stage 2b: build and test python 3.5 wrappers
- stage 2c: build and test java wrappers

the three "stage 2" things can run in parallel.

>From the bit of reading I did on the train this morning, it looks like this
is not that horrible to do in jenkins; just need to invest a bit of time.

-greg


On Tue, Jun 13, 2017 at 3:12 PM, Brian Kelley  wrote:

> I tried using ninja for the builds in stead of make, after an evening of
> not building I successfully had two builds:
>
> https://travis-ci.org/bp-kelley/rdkit/builds/242399233
>
> Commit:
>
> https://github.com/bp-kelley/rdkit/blob/baac00bc1eb7f66413011aed239413
> fe74da3388/.travis.yml
>
> Note that the default ninja build is #cores + 2 which blew out the memory
> on the VMs so I used -j3 we could try this in the makefile as well.
>
> I also think we could shave off a few minutes by
>
>   1. building in the root conda env
>   2. having only one conda install for *everything*
>
> If you like I can try these fixes.
>
> Cheers,
>   Brian
>
> On Tue, Jun 13, 2017 at 3:44 AM, Greg Landrum 
> wrote:
>
>> Dear all,
>>
>> We're getting more and more timeouts with RDKit builds on travis. I'm
>> reasonably sure that this is due to some variability in the performance of
>> the VMs that the builds end up running on. A good demonstration of that is
>> this particular build (on a branch where I'm playing with simplifying the
>> travis builds), which works fine for py35 but times out for py27:
>> https://travis-ci.org/rdkit/rdkit/builds/242301061?utm_sourc
>> e=email_medium=notification
>>
>> If you look into the details you'll see that the main compile "only"
>> takes 1475 seconds for the python 3.5 build but it is killed after 2541
>> seconds in the python2.7 build. Given that it's the same code being built,
>> it seems like this more or less has to be infrastructure variability.
>>
>> I'll continue to explore, particularly breaking the build into stages,
>> but if anyone has suggestions on how to improve this please let me know.
>>
>> -greg
>>
>>
>> 
>> --
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> ___
>> Rdkit-devel mailing list
>> Rdkit-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/rdkit-devel
>>
>>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Rdkit-devel mailing list
Rdkit-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rdkit-devel

Re: [Rdkit-devel] travis build timeouts

2017-06-13 Thread Brian Kelley 
I tried using ninja for the builds in stead of make, after an evening of
not building I successfully had two builds:

https://travis-ci.org/bp-kelley/rdkit/builds/242399233

Commit:

https://github.com/bp-kelley/rdkit/blob/baac00bc1eb7f66413011aed239413fe74da3388/.travis.yml

Note that the default ninja build is #cores + 2 which blew out the memory
on the VMs so I used -j3 we could try this in the makefile as well.

I also think we could shave off a few minutes by

  1. building in the root conda env
  2. having only one conda install for *everything*

If you like I can try these fixes.

Cheers,
  Brian

On Tue, Jun 13, 2017 at 3:44 AM, Greg Landrum 
wrote:

> Dear all,
>
> We're getting more and more timeouts with RDKit builds on travis. I'm
> reasonably sure that this is due to some variability in the performance of
> the VMs that the builds end up running on. A good demonstration of that is
> this particular build (on a branch where I'm playing with simplifying the
> travis builds), which works fine for py35 but times out for py27:
> https://travis-ci.org/rdkit/rdkit/builds/242301061?utm_
> source=email_medium=notification
>
> If you look into the details you'll see that the main compile "only" takes
> 1475 seconds for the python 3.5 build but it is killed after 2541 seconds
> in the python2.7 build. Given that it's the same code being built, it seems
> like this more or less has to be infrastructure variability.
>
> I'll continue to explore, particularly breaking the build into stages, but
> if anyone has suggestions on how to improve this please let me know.
>
> -greg
>
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Rdkit-devel mailing list
> Rdkit-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/rdkit-devel
>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Rdkit-devel mailing list
Rdkit-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/rdkit-devel