Hi everyone,
Until very recently, most of the caches of various repositories ('remote
repositories') we operate on repo.jenkins-ci.org (Maven Central, JCenter, and a
bunch of others) were configured to use plain HTTP to download artifacts. This
has now been fixed where possible. While I didn't find any unexplainable
differences during a rudimentary check of the dependencies used in the most
popular Jenkins plugins, this still isn't a great situation and I'd like to get
rid of these existing caches. These repos are part of the virtual repository
repo.jenkins-ci.org/public referenced from pom.xml files, so are involved in
dependency resolution from Jenkins components like plugins.
Note that I wrote 'unexplainable differences'. Our caches have, in some cases,
diverged from upstream. All occurrences of this that I've seen can be traced to
a project re-releasing an artifact after it was already cached by us.
Wiping our caches would therefore result in changes to some artifacts, or
possibly lose them altogether. Not wiping caches OTOH isn't a reasonable
alternative either if we want to be able to trust these artifacts. Manually
confirming our caches are identical, and, in case of differences, confirm
they're legitimate, would be extremely time consuming, given the number of
cached artifacts[1], and we still wouldn't know what to do about any artifacts
cached by us that are gone from upstream[2].
My suggestion would be that we configure our current cache repositories to be
"blacked out". That's an Artifactory feature that allows repositories to
remain, but disables downloads and artifact resolution. In their place, we
would add new remote repositories to the same upstream repos, but configured to
use HTTPS from the start. (Alternatively, we could remove these old remote
repositories from the 'public' virtual repository referenced in pom.xml files,
largely accomplishing the same goal.)
If we actually encounter problems due to unintended, major changes to
artifacts, we will be able to restore them from these blacked out caches to a
"manual cache" repository with higher precedence than whatever would be
available from remote repositories.
WDYT?
Daniel
(And while we're at it: Do we want our caches to expire, or have consistent
builds in favor of alignment with upstream re-releases?)
1: JCenter: 1M files / 360.37 GB; Maven Central: 5.6M files / 1.45 TB
2: I gave up on this task for our cache of Atlassian's repo after 76830 cached
artifacts, of which 29 actually existed upstream.
--
You received this message because you are subscribed to the Google Groups
"Jenkins Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/jenkinsci-dev/832077AA-3BA3-4780-8A79-6C098F57D566%40beckweb.net.
For more options, visit https://groups.google.com/d/optout.
