Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

I have overlay2 and super fast disk I/O (memory cheat + SSD), just the CPU freq is not high. The CPU is a Broadwell and actually it has lot more core (E5-2630V4). Even a 5 year old gamer CPU can be 2 times faster on a single core, but cannot compete with all of the cores ;-) This machine have

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 26 September 2017 at 07:34, Attila Fazekas wrote: > decompressing those registry tar.gz takes ~0.5 min on 2.2 GHz CPU. > > Fully pulling all container takes something like ~4.5 min (from localhost, > one leaf request at a time), > but on the gate vm we usually have 4

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

decompressing those registry tar.gz takes ~0.5 min on 2.2 GHz CPU. Fully pulling all container takes something like ~4.5 min (from localhost, one leaf request at a time), but on the gate vm we usually have 4 core, so it is possible to go bellow 2 min with better pulling strategy, unless we hit

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 22 September 2017 at 17:21, Paul Belanger wrote: > On Fri, Sep 22, 2017 at 02:31:20PM +, Jeremy Stanley wrote: >> On 2017-09-22 15:04:43 +0200 (+0200), Attila Fazekas wrote: >> > "if DevStack gets custom images prepped to make its jobs >> > run faster, won't

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On Fri, Sep 22, 2017 at 02:31:20PM +, Jeremy Stanley wrote: > On 2017-09-22 15:04:43 +0200 (+0200), Attila Fazekas wrote: > > "if DevStack gets custom images prepped to make its jobs > > run faster, won't Triple-O, Kolla, et cetera want the same and where > > do we draw that line?). " > > > >

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 22 September 2017 at 11:45, Clark Boylan wrote: > On Fri, Sep 22, 2017, at 08:58 AM, Michał Jastrzębski wrote: >> Another, more revolutionary (for good or ill) alternative would be to >> move gates to run Kolla instead of DevStack. We're working towards >> registry of

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On Fri, Sep 22, 2017, at 08:58 AM, Michał Jastrzębski wrote: > Another, more revolutionary (for good or ill) alternative would be to > move gates to run Kolla instead of DevStack. We're working towards > registry of images, and we support most of openstack services now. If > we enable mixed

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 22 September 2017 at 07:31, Jeremy Stanley wrote: > On 2017-09-22 15:04:43 +0200 (+0200), Attila Fazekas wrote: >> "if DevStack gets custom images prepped to make its jobs >> run faster, won't Triple-O, Kolla, et cetera want the same and where >> do we draw that line?). " >>

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 2017-09-22 15:04:43 +0200 (+0200), Attila Fazekas wrote: > "if DevStack gets custom images prepped to make its jobs > run faster, won't Triple-O, Kolla, et cetera want the same and where > do we draw that line?). " > > IMHO we can try to have only one big image per distribution, > where the

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

"if DevStack gets custom images prepped to make its jobs run faster, won't Triple-O, Kolla, et cetera want the same and where do we draw that line?). " IMHO we can try to have only one big image per distribution, where the packages are the union of the packages requested by all team, minus the

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 2017-09-20 15:17:28 +0200 (+0200), Attila Fazekas wrote: [...] > The image building was the good old working solution and unless > the image build become a super expensive thing, this is still the > best option. [...] It became a super expensive thing, and that's the main reason we stopped

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On Wed, Sep 20, 2017 at 3:11 AM, Ian Wienand wrote: > On 09/20/2017 09:30 AM, David Moreau Simard wrote: > >> At what point does it become beneficial to build more than one image per >> OS >> that is more aggressively tuned/optimized for a particular purpose ? >> > > ... and

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 09/20/2017 09:30 AM, David Moreau Simard wrote: At what point does it become beneficial to build more than one image per OS that is more aggressively tuned/optimized for a particular purpose ? ... and we can put -dsvm- in the jobs names to indicate it should run on these nodes :) Older

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On Tue, Sep 19, 2017 at 9:03 AM, Jeremy Stanley wrote: > > In order to reduce image sizes and the time it takes to build > images, once we had local package caches in each provider we stopped > pre-retrieving packages onto the images. Is the time spent at this > stage mostly

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 09/19/2017 11:03 PM, Jeremy Stanley wrote: On 2017-09-19 14:15:53 +0200 (+0200), Attila Fazekas wrote: [...] The jobs does 120..220 sec apt-get install and packages defined /files/debs/general are missing from the images before starting the job. Is the time spent at this stage mostly

Re: [openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

On 2017-09-19 14:15:53 +0200 (+0200), Attila Fazekas wrote: [...] > Let's start with the first obvious difference compared to the old-time > jobs.: > The jobs does 120..220 sec apt-get install and packages defined > /files/debs/general are missing from the images before starting the job. > > We

[openstack-dev] [devstack] Why do we apt-get install NEW files/debs/general at job time ?

The gate-tempest-dsvm-neutron-full-ubuntu-xenial job is 20..30 min slower than it supposed to be/used to be. The extra time has multiple reasons and it is not because we test more :( . Usually we are just less smart than before. Huge time increment is visible in devstack as well. devstack is