On Mon, Jun 01, 2020 at 04:51:19PM +0200, Pavel Hrdina wrote: > On Fri, May 29, 2020 at 03:00:39PM +0200, Andrea Bolognani wrote: > > Branch: https://gitlab.com/abologna/libvirt/-/tree/ci-full-gitlab-registry > > Pipeline: https://gitlab.com/abologna/libvirt/pipelines/150891361 > > > > This is what we're already doing with the subprojects we've migrated > > to GitLab CI and, as of earlier today, all projects under the > > libosinfo umbrella. > > > > Once this is merged, we can stop publishing container images on Quay > > and archive the libvirt-dockerfiles repository. > > > > Patch 3/5 has been trimmed in order to comply with the size limits > > of the mailing list. You can grab the unabridged version with > > > > $ git fetch https://gitlab.com/abologna/libvirt ci-full-gitlab-registry > > This is a lot of files and lines of text/code. I was wondering about > building the dockerfiles as part of the container_job_definition. > > To me it seems like a lot of duplication and a lot of noise in the > future if we decide to change the dockerfiles generation. The main > difference that I can think of is that with files in repository we > need to regenerate all the dockerfiles to apply changes made in > libvirt-ci but with the automatic generation we would have that for > free.
The key reason for keeping the dockerfiles in the libvirt.git repo and NOT auto-generating them on the fly is to ensure the CI process is self-contained, with no dependancy on external moving parts in other git repos. If you automatically generated dockerfiles from libvirt-ci.git, then you end up with unstable CI when changes in libvirt.git need to be made in lock-step with changes in libvirt-ci.git. If you change libvirt.git first, CI will break if it runs before libvirt-ci.git is updated. If you change libvirt-ci.git first, then CI will break if it runs before libvirt.git is updated. This is a no-win situation. This is especially painful when you consider that a user's fork of libvirt.git may not updated to current master. Or consider a user who needs to make changes to libvirt.git that require updated dockerfiles and needs to be able to test them before any change in libvirt-ci.git is present. We've seen these problems many times with our current Jenkins setup where CI breaks for a period when we have to do matching updates between both libvirt-ci.git & libvirt.git. With dockerfiles kept in libvirt.git we know that the containers we're building will always contain exactly what we need. This also makes it easy for users to experiment with changes, as they can modify the dockerfiles directly to add/remove pieces. Such changes can be propagated back to libvirt-ci.git out of band. > Both approaches have some benefits and drawbacks and I guess we could > have some variable to prevent automatic generation of dockerfiles to > make sure that unwanted changes in libvirt-ci doesn't affect CI for > all libvirt repositories, on the other hand it would automatically > check that changes to libvirt-ci doesn't break anything. Changes to libvirt-ci that affect the dockerfiles, should come with a URL pointer to a merge request against the affected project(s), showing the succesful CI run with updated dockerfiles. > I personally don't like the need to introduce 5000+ lines just for > compilation testing. That's a tiny proportion of the code we have in libvirt.git, so IMHO it is not worth worrying about. The benefits of having the CI self contained far outweigh the downside. Essentially we are prioritizing the main libvirt.git as that is the primary content from the POV of most contributors. Regards, Daniel -- |: https://berrange.com -o- https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o- https://fstop138.berrange.com :| |: https://entangle-photo.org -o- https://www.instagram.com/dberrange :|
