Hi Martin, Thank you for that. A few remarks and questions: - how much time does the "docker build" step takes? We can possibly speed things up by push images to the dockerhub, as they don't need to change very often. - any reason why you clone pkg-varnish-cache in each job? The idea was to have it cloned once in tar-pkg-tools for consistency and reproducibility, which we lose here. - do we want to change things for the amd64 platforms for the sake of consistency?
-- Guillaume Quintard On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov <martin.grigo...@gmail.com> wrote: > Hi, > > > On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov <martin.grigo...@gmail.com> > wrote: > >> Hi, >> >> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov < >> martin.grigo...@gmail.com> wrote: >> >>> Hi Guillaume, >>> >>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard < >>> guilla...@varnish-software.com> wrote: >>> >>>> Hi, >>>> >>>> Offering arm64 packages requires a few things: >>>> - arm64-compatible code (all good in >>>> https://github.com/varnishcache/varnish-cache) >>>> - arm64-compatible package framework (all good in >>>> https://github.com/varnishcache/pkg-varnish-cache) >>>> - infrastructure to build the packages (uhoh, see below) >>>> - infrastructure to store and deliver ( >>>> https://packagecloud.io/varnishcache) >>>> >>>> So, everything is in place, expect for the third point. At the moment, >>>> there are two concurrent CI implementations: >>>> - travis: >>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml It's >>>> the historical one, and currently only runs compilation+test for OSX >>>> >>> >>> Actually it tests Linux AMD64 and ARM64 too. >>> >>> >>>> - circleci: >>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml >>>> the >>>> new kid on the block, that builds all the packages and distchecks for all >>>> the packaged platforms >>>> >>>> The issue is that cirecleci doesn't support arm64 containers (for >>>> now?), so we would need to re-implement the packaging logic in Travis. It's >>>> not a big problem, but it's currently not a priority on my side. >>>> >>>> However, I am totally ready to provide help if someone wants to take >>>> that up. The added benefit it that Travis would be able to handle >>>> everything and we can retire the circleci experiment >>>> >>> >>> I will take a look in the coming days and ask you if I need help! >>> >> >> I've took a look at the current setup and here is what I've found as >> problems and possible solutions: >> >> 1) Circle CI >> 1.1) problem - the 'machine' and 'Docker' executors run on x86_64, so >> there is no way to build the packages in a "native" environment >> 1.2) possible solutions >> 1.2.1) use multiarch cross build >> 1.2.2) use 'machine' executor that registers QEMU via >> https://hub.docker.com/r/multiarch/qemu-user-static/ and then builds and >> runs a custom Docker image that executes a shell script with the build steps >> It will look something like >> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 >> but >> instead of uploading the Docker image as a last step it will run it. >> The RPM and DEB build related code from current config.yml will be >> extracted into shell scripts which will be copied in the custom Docker >> images >> >> From these two possible ways I have better picture in my head how to do >> 1.2.2, but I don't mind going deep in 1.2.1 if this is what you'd prefer. >> > > I've decided to stay with Circle CI and use 'machine' executor with QEMU. > > The changed config.yml could be seen at > https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci > and > the build at > https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8 > The builds on x86 arch take 3-4 mins, but for aarch64 (emulation!) ~40mins > For now the jobs just build the .deb & .rpm packages for CentOS 7 and > Ubuntu 18.04, both amd64 and aarch64. > TODOs: > - migrate Alpine > - store the packages as CircleCI artifacts > - anything else that is still missing > > Adding more architectures would be as easy as adding a new Dockerfile with > a base image from the respective type. > > Martin > > >> 2) Travis CI >> 2.1) problems >> 2.1.1) generally Travis is slower than Circle! >> Althought if we use CircleCI 'machine' executor it will be slower than >> the current 'Docker' executor! >> 2.1.2) Travis supports only Ubuntu >> Current setup at CircleCI uses CentOS 7. >> I guess the build steps won't have problems on Ubuntu. >> >> 3) GitHub Actions >> GH Actions does not support ARM64 but it supports self hosted ARM64 >> runners >> 3.1) The problem is that there is no way to make a self hosted runner >> really private. I.e. if someone forks Varnish Cache any commit in the fork >> will trigger builds on the arm64 node. There is no way to reserve the >> runner only for commits against >> https://github.com/varnishcache/varnish-cache >> >> Do you see other problems or maybe different ways ? >> Do you have preferences which way to go ? >> >> Regards, >> Martin >> >> >>> >>> Regards, >>> Martin >>> >>> >>>> >>>> -- >>>> Guillaume Quintard >>>> _______________________________________________ >>>> varnish-dev mailing list >>>> varnish-dev@varnish-cache.org >>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >>>> >>>
_______________________________________________ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev
