Following up,
I've implemented a new Jenkins job that re-pulls all current couchdbdev
images on each docker node every night. The job takes 12 minutes to run.
Once a week, it also runs the "pull them all" set of images I mentioned
a few months ago, to keep our images on Docker Hub from disappearing.
(That run takes about an hour and happens Sunday nights, randomly,
between 0-7h UTC.)
With this we can drop the "alwaysPull true" in the Jenkinsfiles, which
should speed up PRs and builds by a couple minutes minutes each.
Those PRs are up here:
https://github.com/apache/couchdb/pull/3234
https://github.com/apache/couchdb/pull/3233
which need +1s to land.
That leaves a single problem scenario, namely when the couchdbdev images
are updated and Docker stubbornly refuses to pull the latest image,
causing build issues. In that case, simply log into our Jenkins and
click "Build Now" here:
https://ci-couchdb.apache.org/job/jenkins-cm1/job/Update%20Docker%20Containers/
which forces the image removal and re-pull.
I hope this is enough to avoid any rate limiting problems.
-Joan "Gir, stop singing this instant!" Touzet
On 2020-10-29 12:05 a.m., Joan Touzet wrote:
I just posted about this on the ASF-wide bui...@apache.org list:
https://lists.apache.org/thread.html/r5ccf60da8072b3c2b587152256ebaf6a0e7b81182d5e240a2b2a0f02%40%3Cbuilds.apache.org%3E
TL;DR: We're not immune, even with our build machines, and the new
limits kick in Monday.
We can remove some of our workarounds for badly-cached images (at the
cost of lots of pain every time the couchdbci build environment image
changes), but on a busy day we'll probably still hit the limit.
Let's see if ASF Infra comes through quickly with a pull-through caching
registry of sufficient size. If so we can make very minor tweaks to our
Jenkinsfiles, and go back to life as normal.
-Joan "Doom doom, doomy doomy doom" Touzet
