On Mon, Oct 25, 2021 at 06:33:53PM -0700, Stefano Stabellini wrote: > On Mon, 25 Oct 2021, Anthony PERARD wrote: > > There is something I'm missing, how is it a problem to have a container > > that is a bit bigger? What sort of problem could we have to deal with? > > It takes time to clone the container in the gitlab-ci, the bigger the > container the more time it takes. It is fetched over the network. Now we > are fetching qemu (as part of the container) 10 times during the build > although it is not needed.
I guess the issue would be more like we don't do enough caching with our gitlab runners. So package installation is just a workaround. > But we do have a severe problem at the moment with external sources: our > "git clones" keep failing during the build on x86. That is definitely > something worth improving (see my other email thread on the subject) and > it is the main problem affecting gitlab-ci at the moment, I keep having > to restart jobs almost daily to get the overall pipeline to "pass". > > If you have any ideas on how to stop fetching things using "git" from > external repositories in gitlab-ci that would be fantastic :-) > The only thing I could think of to "fix it" is moving all external repos > to gitlab repositories mirrors. I don't think that would work, I've seen the initial clone/fetch of a job fail as well, so from gitlab. If we could have a cache of those external resources closer to the runners, that would be better. > > > I am not entirely sure what is the best solution overall, but for this > > > series at this stage I would prefer to keep the same strategy used for > > > the ARM tests (i.e. reuse the debian unstable build container and > > > apt-get the few missing packages.) If we do change the way we do it, I > > > would rather change both x86 and ARM at the same time. > > > > I'm pretty sure the best strategy would be to do as little as possible > > during a job, download as little as possible and possibly cache as much > > as possible and do as much as possible ahead of time. Feel free to > > change the Arm test, but I don't think it is necessary to change the Arm > > test at the same time as introducing an x86 test. > > I agree. > > At the same time it would be nice to follow the same strategy between > x86 and ARM going forward: if one optimization is made for one, it is > also made for the other. Probably better, yes. Thanks, -- Anthony PERARD
