Re: [PATCH] s3-mirror: sync s3-sync-path script with ideas from s3.sh
On Saturday, March 28, 2020 8:35:00 PM CEST Kevin Fenzi wrote: > > I would like to turn on the cloudfront mirror again to see if COPR still > > breaks. Any objections? > > None here, +1 +1, thank you! Pavel ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
CPE Weekly: 2020-03-28
--- title: CPE Weekly status email tags: CPE Weekly, email --- # CPE Weekly: 2020-03-06 Background: The Community Platform Engineering group is the Red Hat team combining IT and release engineering from Fedora and CentOS. Our goal is to keep core servers and services running and maintained, build releases, and other strategic tasks that need more dedicated time than volunteers can give. For better communication, we will be giving weekly reports to the CentOS and Fedora communities about the general tasks and work being done. Also for better communication between our groups we have created #redhat-cpe on Freenode IRC! Please feel free to catch us there, a mail has landed on both the CentOS and Fedora devel lists with context here. ## Fedora Updates * Freeze is over! * New version of pagure (5.9.0) has been released and deployed in staging and on pagure.io * Document being worked on about how to onboard new CI systems in Fedora - this is a work in progress! https://hackmd.io/5gk_kHFhSR6sKa1huPA-4Q ### Request for Review * Fedora magazine: Article proposals about Silverblue rebase to F32 https://pagure.io/fedora-magazine-proposals/issue/59 * Packit integration in the-new-hotness https://github.com/packit-service/packit/issues/689 * KeepassXC flatpak issue https://pagure.io/flatpak-module-tools/issue/6 ### Data Centre Move * Communishift will be unavailable from 2020-04-13 until 2020-05-08 * Check out our detailed move shedule here https://hackmd.io/R3EkjzVyTG2TYwQvkfzYrA?sync=&type= * Covid-19 has seen restrictions added to the data centres we are moving from and to, however we are unaffected as of yet for the move. * If or when our timelines become affected, we will inform you immediately of any outages, downtimes, etc a delay could cause. * Thank you for your understanding at this particularly uncertain and worrying times. ### AAA Replacement * We are nearly complete in our phase one development! * Below are some of the features the team have developed since beginning the project in mid-Jan * UI where people register and login * Add groups function * Change personal details * Enroll OPT * Reset password * View group owner details * Use a search engine * Our next steps in this project is to demo to the team for feedback and then continue to develop CentOS authentication, more user focused features, request for more feedback and test, test test! ### CI/CD * Monitor-gating is now running in production and has already caught a couple of issues with bodhi (both in stg and in prod)! * Rpmautospec * This is in review as a Fedora package: https://bugzilla.redhat.com/show_bug.cgi?id=1816124 * Work progressing on Koji tagging plugin (post-build), full use case support for bumping releases * The team hope to deploy this in staging soon! ### Sustaining Team * Mbbox * Some progress on CRD for koji-builder and koji-hub components has been made this week * Bodhi 5.2.2 released * Some issues with celery tasks & rawhide monitoring has been super useful with this. * Compose Tracker enhancement * Tagging issues have been resolved * Ability to ping maintainers * Fedora Minimal Compose * Odcs-backend-releng01 has been provisioned to enable testing ## CentOS Updates ### CentOS * ppc64le and aarch64, 8 and 8-stream nodes now available in cico for tenants to checkout. -- Email sent to ci-user list * New signing for SIGs (through https://cbs.centos.org) live this week! ### CentOS Stream * Qt5.12 pushed in response to an internal request * NetworkManager re-imported ### Other Updates GitForge Decision * After evaluating over 300 user stories from multiple stakeholders we have aligned on a decision for the Gitforge that CPE will operate for the coming years. We are opting for Gitlab for our dist git and project hosting and will continue to run pagure.io with community assistance. * Check out our GitForge decision on the Fedora Community blog https://communityblog.fedoraproject.org/ * And at the CentOS blog page https://blog.centos.org/2020/03/git-forge-decision/ * Keep an eye out for mails in the coming months to the devel lists as we plan transitions and next steps with GitLab * We would like to express our sincere thank you to all who contributed requirements to us! As always, feedback is welcome, and we will continue to look at ways to improve the delivery and readability of this weekly report. Have a great weekend! Aoife Source: https://hackmd.io/8iV7PilARSG68Tqv8CzKOQ -- Aoife Moloney Product Owner Community Platform Engineering Team Red Hat EMEA Communications House Cork Road Waterford ___ infrastructure mailing list -- infrastructure@lists.fedoraproject.org To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraprojec
Re: [PATCH] s3-mirror: sync s3-sync-path script with ideas from s3.sh
On Sat, Mar 28, 2020 at 11:09:42AM +0100, Adrian Reber wrote:
> On Fri, Mar 27, 2020 at 05:14:48PM -0700, Kevin Fenzi wrote:
> > On Fri, Mar 27, 2020 at 12:06:22PM +0100, Pavel Raiskup wrote:
> > > On Friday, March 27, 2020 11:34:28 AM CET Adrian Reber wrote:
> > > > The global 'exclude' has '--exclude "*/repodata/*"' and you are using
> > > > "${excludes[@]}" everywhere. In all three syncs. This looks like
> > > > 'repodata/*' will never synced.
> > >
> > > Good catch! Thank you for the review. I am attaching updated patch.
> > >
> > > Pavel
> >
> > Looks ok to me from a quick glance... lets give it a try. :)
> >
> > I think also we should remove the test releases one. It's currently
> > failing the invalidate step (because it's repodata is not in the same
> > place as the updates repos) and the only reason we would want say
> > 32_Beta to be there is so we could point people to download isos from
> > it. The repodata will never change.
>
> Just to confirm from the MirrorManager side. MirrorManager is not aware
> of any repomd.xml files under releases/test. For 32 MirrorManager points
> to development/32 or updates/32 and updates/testing/32.
>
> So MirrorManager will not redirect any clients to releases/test except
> for ISOs.
Yeah, so perhaps we should consider just syncing the isos?
And it only needs to run once, so perhaps this could be a manually run
script/playbook.
kevin
signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: [PATCH] s3-mirror: sync s3-sync-path script with ideas from s3.sh
On Sat, Mar 28, 2020 at 11:04:51AM +0100, Adrian Reber wrote:
> On Fri, Mar 27, 2020 at 05:14:48PM -0700, Kevin Fenzi wrote:
> > On Fri, Mar 27, 2020 at 12:06:22PM +0100, Pavel Raiskup wrote:
> > > On Friday, March 27, 2020 11:34:28 AM CET Adrian Reber wrote:
> > > > The global 'exclude' has '--exclude "*/repodata/*"' and you are using
> > > > "${excludes[@]}" everywhere. In all three syncs. This looks like
> > > > 'repodata/*' will never synced.
> > >
> > > Good catch! Thank you for the review. I am attaching updated patch.
> > >
> > > Pavel
> >
> > Looks ok to me from a quick glance... lets give it a try. :)
> >
> > I think also we should remove the test releases one. It's currently
> > failing the invalidate step (because it's repodata is not in the same
> > place as the updates repos) and the only reason we would want say
> > 32_Beta to be there is so we could point people to download isos from
> > it. The repodata will never change.
>
> Accessing cloudfront I can now see the correct header:
>
> cache-control: max-age=60
>
> and the age header never gets larger than 60
>
> I can see 'age: 59', but each request after that gives me an
>
> x-cache: RefreshHit from cloudfront
>
> From the header returned by cloudfront it seems we are now never seeing
> repomd.xml files older than 1 minute.
>
> I would like to turn on the cloudfront mirror again to see if COPR still
> breaks. Any objections?
None here, +1
kevin
signature.asc
Description: PGP signature
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: [PATCH] s3-mirror: sync s3-sync-path script with ideas from s3.sh
On Fri, Mar 27, 2020 at 05:14:48PM -0700, Kevin Fenzi wrote:
> On Fri, Mar 27, 2020 at 12:06:22PM +0100, Pavel Raiskup wrote:
> > On Friday, March 27, 2020 11:34:28 AM CET Adrian Reber wrote:
> > > The global 'exclude' has '--exclude "*/repodata/*"' and you are using
> > > "${excludes[@]}" everywhere. In all three syncs. This looks like
> > > 'repodata/*' will never synced.
> >
> > Good catch! Thank you for the review. I am attaching updated patch.
> >
> > Pavel
>
> Looks ok to me from a quick glance... lets give it a try. :)
>
> I think also we should remove the test releases one. It's currently
> failing the invalidate step (because it's repodata is not in the same
> place as the updates repos) and the only reason we would want say
> 32_Beta to be there is so we could point people to download isos from
> it. The repodata will never change.
Just to confirm from the MirrorManager side. MirrorManager is not aware
of any repomd.xml files under releases/test. For 32 MirrorManager points
to development/32 or updates/32 and updates/testing/32.
So MirrorManager will not redirect any clients to releases/test except
for ISOs.
Adrian
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
Re: [PATCH] s3-mirror: sync s3-sync-path script with ideas from s3.sh
On Fri, Mar 27, 2020 at 05:14:48PM -0700, Kevin Fenzi wrote:
> On Fri, Mar 27, 2020 at 12:06:22PM +0100, Pavel Raiskup wrote:
> > On Friday, March 27, 2020 11:34:28 AM CET Adrian Reber wrote:
> > > The global 'exclude' has '--exclude "*/repodata/*"' and you are using
> > > "${excludes[@]}" everywhere. In all three syncs. This looks like
> > > 'repodata/*' will never synced.
> >
> > Good catch! Thank you for the review. I am attaching updated patch.
> >
> > Pavel
>
> Looks ok to me from a quick glance... lets give it a try. :)
>
> I think also we should remove the test releases one. It's currently
> failing the invalidate step (because it's repodata is not in the same
> place as the updates repos) and the only reason we would want say
> 32_Beta to be there is so we could point people to download isos from
> it. The repodata will never change.
Accessing cloudfront I can now see the correct header:
cache-control: max-age=60
and the age header never gets larger than 60
I can see 'age: 59', but each request after that gives me an
x-cache: RefreshHit from cloudfront
From the header returned by cloudfront it seems we are now never seeing
repomd.xml files older than 1 minute.
I would like to turn on the cloudfront mirror again to see if COPR still
breaks. Any objections?
Adrian
___
infrastructure mailing list -- infrastructure@lists.fedoraproject.org
To unsubscribe send an email to infrastructure-le...@lists.fedoraproject.org
Fedora Code of Conduct:
https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives:
https://lists.fedoraproject.org/archives/list/infrastructure@lists.fedoraproject.org
