Martin Pitt: > Agreed. Let's keep things simple and not overdesign this. The more complicated > the Restrictions: schema becomes, the more edge cases you will find [1]. > I daresay autopkgtests became so successful because they are structurally so > simple and easy to explain ("declare list, give dependencies, run program, > exit > code 0 and no stderr") > > Martin > > [1] > https://mikehadlow.blogspot.com/2012/05/configuration-complexity-clock.html
I 100% agree with this. The devil is in the details when working to keep things simple. I really think that we can get salsa's gitlab-ci setup to be an almost transparent extension of autopkgtest for the vast majority of cases. So we need to figure out exactly what details tests need to control and what can be entirely outside of autopkgtest. I've been working with custom CI scripts, Jenkins, Travis-CI, and GitLab-CI for many years now, and I think GitLab-CI is an example that we can really learn from. They did a good job of making it simple as compared to what it can do. Travis-CI is simpler to get started with, but you rapidly run into its limitations. Jenkins can do lots of things, but is always seemed overcomplicated and confused me. One key idea that makes GitLab-CI good is that they make it a thin layer on top of Docker. I've only used GitLab-CI via Docker and `gitlab-runner exec docker`. It also supports VirtualBox, Parallels, SSH, Shell, and Kubernetes: https://docs.gitlab.com/runner/executors/README.html They have added container-only config options like 'image:' and 'services:' that are just ignored by other runners. It also uses a freeform system of tagging runners and jobs for managing test requirements.