Re: [openstack-dev] [tripleo] tripleo-test-cloud-rh2 local mirror server

[Derek Higgins]

On 27 July 2016 at 17:52, Paul Belanger  wrote:
> On Wed, Jul 27, 2016 at 02:54:00PM +0100, Derek Higgins wrote:
>> On 21 July 2016 at 23:04, Paul Belanger  wrote:
>> > Greetings,
>> >
>> > I write today to see how I can remove this server from 
>> > tripleo-test-cloud-rh2. I
>> > have an open patch[1] currently to migrate tripleo-ci to use our AFS 
>> > mirrors for
>> > centos and epel.  However, I'm still struggling to see what else you are 
>> > using
>> > the local mirror for.
>> >
>> > From what I see, there appears to be some puppet modules in the mirror?
>> >
>> > The reason I am doing this work, is to help bring tripleo inline with
>> > openstack-infra tooling.  There shouldn't be the need for a project to 
>> > maintain
>> > its own infrastructure outside of openstack-infra.  If so, I see that as 
>> > some
>> > sort of a failure between the project and openstack-infra.   And with that 
>> > in
>> > mind, I am here to help fix that.
>> >
>> > For the most part, I think we have everything currently in place to 
>> > migrate away
>> > from your locally mirror. I just need some help figuring what else is left 
>> > and
>> > then delete it.
>>
>> Hi Paul,
>> The mirror server hosts 3 sets of data used in CI long with a cron
>> a job aimed at promoting trunk repositories,
>> The first you've already mentioned, there is a list of puppet modules
>> hosted here, we soon hope to move to packaged puppet modules so the
>> need for this will go away.
>>
> Ya, I was looking at an open review to rework this. If we moved these puppet
> modules to tarballs over git repos, I think we could mirror them pretty easy
> into our AFS mirrors.  Them being git repos requires more work because some
> policies around git repos.

We wont need to do anything here, the patch to move away from git
repos will be instead using the rdo packaged puppet modules, so we
wont need anything from infra for this, we just end up using the rdo
repository like we do for all other openstack projects.

>
>> The second is a mirror of the centos cloud images, these are updated
>> hourly by the centos-cloud-images cronjob[1], I guess these could be
>> easily replaced with the AFS server
>>
> So 2 things here.
>
> 1) I've reached out to CentOS asking to enable rsync support on
> http://cloud.centos.org/ if they do that, I can easily enable rsync for it.

Great

>
> 2) What about moving away from the centos diskimage-builder element and switch
> to centos-minimal element. I have an open review for this, but need help on
> actually testing this.  It moves away from using the cloud image, and instead
> uses yumdownloader to prebuild the images.

Its possible, but I think out of scope for a general ci thread, its
more of a tripleo decision so maybe needs its own thread to get a
wider audience.

>
>> Then we come to the parts where it will probably be more tricky to
>> move away from our own server
>>
>> o cached images - our nightly periodic jobs run tripleo ci with
>> master/HEAD for all openstack projects (using the most recent rdo
>> trunk repository), if the jobs pass then we upload the overcloud-full
>> and ipa images to the mirror server along with logging what jobs
>> passed, this happens at the end of toci_instack.sh[2], nothing else
>> happens at this point the files are just uploaded nothing starts using
>> them yet.
>>
> I suggest we move this to tarballs.o.o for now, this is what other projects 
> are
> doing.  I believe we are also considering moving this process into AFS too.

Ok, its an option worth looking at if we could make it work.

>
>> o promote script - hourly we then run the promote script[3], this
>> script is whats responsible for the promotion of the master rdo
>> repository that is used by tripleo ci (and devs), it checks to see if
>> images have been updated to the mirror server by the periodic jobs,
>> and if all of the jobs we care about (currently
>> periodic-tripleo-ci-centos-7-ovb-ha
>> periodic-tripleo-ci-centos-7-ovb-nonha[4]) passed then it does 2
>> things
>>   1. updates the current-tripleo link on the mirror server[5]
>>   2. updates the current-tripleo link on the rdo trunk server[6]
>> By doing this we ensure that the the current-tripleo link on the rdo
>> trunk server is always pointing to something that has passed tripleo
>> ci jobs, and that tripleo ci is using cached images that were built
>> using this repository
>>
> Okay, I think we need to dive more into this. It might be possible to make 
> this
> a post job or use mirror-update.openstack.org

A post job might work if it could find the status of all the perioc
jobs, I'm not
familiar with mirror-update.openstack.org what does it do?

>
>> We've had to run this promote script on the mirror server as the
>> individual jobs run independently and in oder to make the promote
>> decision we needed somewhere that is aware of the status of all the
>> jobs
>>
>> Hope this answers your questions,
>> Derek.
>>
>> 

Re: [openstack-dev] [tripleo] tripleo-test-cloud-rh2 local mirror server

[Paul Belanger]

On Wed, Jul 27, 2016 at 02:54:00PM +0100, Derek Higgins wrote:
> On 21 July 2016 at 23:04, Paul Belanger  wrote:
> > Greetings,
> >
> > I write today to see how I can remove this server from 
> > tripleo-test-cloud-rh2. I
> > have an open patch[1] currently to migrate tripleo-ci to use our AFS 
> > mirrors for
> > centos and epel.  However, I'm still struggling to see what else you are 
> > using
> > the local mirror for.
> >
> > From what I see, there appears to be some puppet modules in the mirror?
> >
> > The reason I am doing this work, is to help bring tripleo inline with
> > openstack-infra tooling.  There shouldn't be the need for a project to 
> > maintain
> > its own infrastructure outside of openstack-infra.  If so, I see that as 
> > some
> > sort of a failure between the project and openstack-infra.   And with that 
> > in
> > mind, I am here to help fix that.
> >
> > For the most part, I think we have everything currently in place to migrate 
> > away
> > from your locally mirror. I just need some help figuring what else is left 
> > and
> > then delete it.
> 
> Hi Paul,
> The mirror server hosts 3 sets of data used in CI long with a cron
> a job aimed at promoting trunk repositories,
> The first you've already mentioned, there is a list of puppet modules
> hosted here, we soon hope to move to packaged puppet modules so the
> need for this will go away.
> 
Ya, I was looking at an open review to rework this. If we moved these puppet
modules to tarballs over git repos, I think we could mirror them pretty easy
into our AFS mirrors.  Them being git repos requires more work because some
policies around git repos.

> The second is a mirror of the centos cloud images, these are updated
> hourly by the centos-cloud-images cronjob[1], I guess these could be
> easily replaced with the AFS server
> 
So 2 things here.

1) I've reached out to CentOS asking to enable rsync support on
http://cloud.centos.org/ if they do that, I can easily enable rsync for it.

2) What about moving away from the centos diskimage-builder element and switch
to centos-minimal element. I have an open review for this, but need help on
actually testing this.  It moves away from using the cloud image, and instead
uses yumdownloader to prebuild the images.

> Then we come to the parts where it will probably be more tricky to
> move away from our own server
> 
> o cached images - our nightly periodic jobs run tripleo ci with
> master/HEAD for all openstack projects (using the most recent rdo
> trunk repository), if the jobs pass then we upload the overcloud-full
> and ipa images to the mirror server along with logging what jobs
> passed, this happens at the end of toci_instack.sh[2], nothing else
> happens at this point the files are just uploaded nothing starts using
> them yet.
> 
I suggest we move this to tarballs.o.o for now, this is what other projects are
doing.  I believe we are also considering moving this process into AFS too.

> o promote script - hourly we then run the promote script[3], this
> script is whats responsible for the promotion of the master rdo
> repository that is used by tripleo ci (and devs), it checks to see if
> images have been updated to the mirror server by the periodic jobs,
> and if all of the jobs we care about (currently
> periodic-tripleo-ci-centos-7-ovb-ha
> periodic-tripleo-ci-centos-7-ovb-nonha[4]) passed then it does 2
> things
>   1. updates the current-tripleo link on the mirror server[5]
>   2. updates the current-tripleo link on the rdo trunk server[6]
> By doing this we ensure that the the current-tripleo link on the rdo
> trunk server is always pointing to something that has passed tripleo
> ci jobs, and that tripleo ci is using cached images that were built
> using this repository
> 
Okay, I think we need to dive more into this. It might be possible to make this
a post job or use mirror-update.openstack.org

> We've had to run this promote script on the mirror server as the
> individual jobs run independently and in oder to make the promote
> decision we needed somewhere that is aware of the status of all the
> jobs
> 
> Hope this answers your questions,
> Derek.
> 
> [1] - 
> http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n40
> [2] - 
> http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n198
> [3] - 
> http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/promote.sh
> [4] - 
> http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n51
> [5] - http://8.43.87.241/builds/current-tripleo/
> [6] - 
> http://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/
> 
> >
> > [1] https://review.openstack.org/#/c/326143/
> >
> > __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe: 

Re: [openstack-dev] [tripleo] tripleo-test-cloud-rh2 local mirror server

[Derek Higgins]

On 21 July 2016 at 23:04, Paul Belanger  wrote:
> Greetings,
>
> I write today to see how I can remove this server from 
> tripleo-test-cloud-rh2. I
> have an open patch[1] currently to migrate tripleo-ci to use our AFS mirrors 
> for
> centos and epel.  However, I'm still struggling to see what else you are using
> the local mirror for.
>
> From what I see, there appears to be some puppet modules in the mirror?
>
> The reason I am doing this work, is to help bring tripleo inline with
> openstack-infra tooling.  There shouldn't be the need for a project to 
> maintain
> its own infrastructure outside of openstack-infra.  If so, I see that as some
> sort of a failure between the project and openstack-infra.   And with that in
> mind, I am here to help fix that.
>
> For the most part, I think we have everything currently in place to migrate 
> away
> from your locally mirror. I just need some help figuring what else is left and
> then delete it.

Hi Paul,
The mirror server hosts 3 sets of data used in CI long with a cron
a job aimed at promoting trunk repositories,
The first you've already mentioned, there is a list of puppet modules
hosted here, we soon hope to move to packaged puppet modules so the
need for this will go away.

The second is a mirror of the centos cloud images, these are updated
hourly by the centos-cloud-images cronjob[1], I guess these could be
easily replaced with the AFS server

Then we come to the parts where it will probably be more tricky to
move away from our own server

o cached images - our nightly periodic jobs run tripleo ci with
master/HEAD for all openstack projects (using the most recent rdo
trunk repository), if the jobs pass then we upload the overcloud-full
and ipa images to the mirror server along with logging what jobs
passed, this happens at the end of toci_instack.sh[2], nothing else
happens at this point the files are just uploaded nothing starts using
them yet.

o promote script - hourly we then run the promote script[3], this
script is whats responsible for the promotion of the master rdo
repository that is used by tripleo ci (and devs), it checks to see if
images have been updated to the mirror server by the periodic jobs,
and if all of the jobs we care about (currently
periodic-tripleo-ci-centos-7-ovb-ha
periodic-tripleo-ci-centos-7-ovb-nonha[4]) passed then it does 2
things
  1. updates the current-tripleo link on the mirror server[5]
  2. updates the current-tripleo link on the rdo trunk server[6]
By doing this we ensure that the the current-tripleo link on the rdo
trunk server is always pointing to something that has passed tripleo
ci jobs, and that tripleo ci is using cached images that were built
using this repository

We've had to run this promote script on the mirror server as the
individual jobs run independently and in oder to make the promote
decision we needed somewhere that is aware of the status of all the
jobs

Hope this answers your questions,
Derek.

[1] - 
http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n40
[2] - 
http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/toci_instack.sh#n198
[3] - 
http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/promote.sh
[4] - 
http://git.openstack.org/cgit/openstack-infra/tripleo-ci/tree/scripts/mirror-server/mirror-server.pp#n51
[5] - http://8.43.87.241/builds/current-tripleo/
[6] - 
http://buildlogs.centos.org/centos/7/cloud/x86_64/rdo-trunk-master-tripleo/

>
> [1] https://review.openstack.org/#/c/326143/
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [tripleo] tripleo-test-cloud-rh2 local mirror server

[Paul Belanger]

Greetings,

I write today to see how I can remove this server from tripleo-test-cloud-rh2. I
have an open patch[1] currently to migrate tripleo-ci to use our AFS mirrors for
centos and epel.  However, I'm still struggling to see what else you are using
the local mirror for.

>From what I see, there appears to be some puppet modules in the mirror?

The reason I am doing this work, is to help bring tripleo inline with
openstack-infra tooling.  There shouldn't be the need for a project to maintain
its own infrastructure outside of openstack-infra.  If so, I see that as some
sort of a failure between the project and openstack-infra.   And with that in
mind, I am here to help fix that.

For the most part, I think we have everything currently in place to migrate away
from your locally mirror. I just need some help figuring what else is left and
then delete it.

[1] https://review.openstack.org/#/c/326143/

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev