26.05.2023 13:19, Daniel P. Berrangé wrote:
We just (re)discovered that our gitlab rules don't work nicely with
pipelines running from stable staging branches. Every pipeline gets
published with the 'latest' tag, whether its the main staging branch
or one of the stable staging branches. If pipelines for multiple
staging branches run concurrently they'll get very confused and end
up using the wrong container content. eg a 8.0 stable job will get
run with a container from the development branch, or vica-verca.
With this series we dynamically change the tag so that the 'staging'
branch still uses 'latest', but the stable 'staging-X.Y' branaches
use a 'staging-X-Y' container tag.
We also let the container tag be set explicitly via the new variable
QEMU_CI_CONTAINER_TAG
to facilitate CI testing, the new variable
QEMU_CI_UPSTREAM
can be set to the fork namespace, to allow contributors to run a
pipeline as if their fork were upstream.
Daniel, can you describe in a bit more detail (or refer to some text
to read) about how this whole thing works, aka the "big picture"?
It smells like we're doing huge wasteful job here, but it might be
just because I don't understand how it works.
Can't we prepare all containers separately and independently of regular
qemu commits, and just use the prepared container images every time
we run tests?
Also, why can't several tests (from several different pipelines, maybe
run for different branches) use the same images (if we don't re-prepare
them on each and every test run)?
I understand different branches might have different requirements for
containers, like using older version of some OS, etc, - this is done
by naming the container images appropriately, like debian-latest (for
master) vs debian-bullseye (for stable-7.2) etc.
Still, preparing container images might be done separately from regular
commits, like when ci config changes or when new version of something
(fedora/redhat/etc) is released.
How does it all works, why we need to couple container creation to
regular commits and re-create containers on every test run?
Thank you!
/mjt
