On 15/02/2021 09:51, Rohit Yadav wrote:
Hi Nathan,
Thanks for reporting, I've been managing the rpm builds/repos on the server and
I wasn't aware of this issue.
I checked and found there's a hourly cron job that updates rpm repo metadata
using:
createrepo --update --workers 1 --baseurl <other options/paths>
Based on your suggestion, I've changed the script to include: "createrepo --update
--retain-old-md <rest of the args...>".
@Wido Hollander<mailto:w...@pcextreme.nl> @Gabriel Beims
Bräscher<mailto:gabr...@pcextreme.nl> - any reason why we have the cron job to update
repo metadata?
No, I think it's just an oversight. It was setup and I don't think it
was very well thought of.
The CRON is a very simple Shell script which probably can use some
attention.
Wido
Regards.
________________________________
From: Nathan McGarvey <nathanmcgar...@gmail.com>
Sent: Sunday, February 14, 2021 08:32
To: dev@cloudstack.apache.org <dev@cloudstack.apache.org>
Subject: RPM Repository Metadata
To whom this reaches (@widodh, perhaps?):
First of all, thank you for building binary distributions (rpm, deb)
of CloudStack.
I am attempting to create a downstream rsync mirror of the
http://download.cloudstack.org/ centos/rhel repos. (Namely, centos and
systemvm) and noticed two oddities:
1. The frequency with which the metadata is being rebuilt is
astronomical. E.g.
http://download.cloudstack.org/centos/8/4.15/repodata/ looks like it is
being fully rebuilt every hour, though the RPMs contained within haven't
been updated in over a month. Was this supposed to have a
--retain-old-md or --retain-old-md-by-age flag? The default for things
like RHEL 8 is 48 hours for metadata expiry, so re-generating the entire
repo every hour can (and does) cause caching issues.
2. The metadata contained in
http://download.cloudstack.org/centos/8/4.15/repodata/repomd.xml makes
it virtually impossible to mirror since it points the <location> tag at
http://cloudstack.apt-get.eu/
a. Most RPM repos (E.g. The CentOS official ones, just point the
<location> tag to the repodata/<hash>> of the data type without external
links via relative URI. (See
http://mirror.centos.org/centos-8/8.3.2011/BaseOS/x86_64/os/repodata/repomd.xml
, for example.)
b. Putting the xml:base in there effectively makes it not a
mirror since anyone pointed at their local mirror will actually redirect
to whatever xml:base is set to.
FWIW: It looks like cloudstack.apt-get.eu and
download.cloudstack.org are *probably* the same host, which likely means
the xml:base being set at all may not actually be doing anything useful.
Please pardon my ignorance if these technical configurations are
intentional and have already been discussed as I am new poster to this
thread.
Thanks,
-Nathan McGarvey
rohit.ya...@shapeblue.com
www.shapeblue.com
3 London Bridge Street, 3rd floor, News Building, London SE1 9SGUK
@shapeblue
