Hi Guillaume, On Mon, Mar 23, 2020 at 8:01 PM Guillaume Quintard < guilla...@varnish-software.com> wrote:
> Hi Martin, > > Thank you for that. > A few remarks and questions: > - how much time does the "docker build" step takes? We can possibly speed > things up by push images to the dockerhub, as they don't need to change > very often. > Definitely such optimization would be a good thing to do! At the moment, with 'machine' executor it fetches the base image and then builds all the Docker layers again and again. Here are the timings: 1) Spinning up a VM - around 10secs 2) prepare env variables - 0secs 3) checkout code (varnish-cache) - 5secs 4) activate QEMU - 2secs 5) build packages 5.1) x86 deb - 3m 30secs 5.2) x86 rpm - 2m 50secs 5.3) aarch64 rpm - 35mins 5.4) aarch64 deb - 45mins > - any reason why you clone pkg-varnish-cache in each job? The idea was to > have it cloned once in tar-pkg-tools for consistency and reproducibility, > which we lose here. > I will extract the common steps once I see it working. This is my first CircleCI project and I still find my ways in it! > - do we want to change things for the amd64 platforms for the sake of > consistency? > So far there is nothing specific for amd4 or aarch64, except the base Docker images. For example make-deb-packages.sh is reused for both amd64 and aarch64 builds. Same for -rpm- and now for -apk- (alpine). Once I feel the change is almost finished I will open a Pull Request for more comments! Martin > > -- > Guillaume Quintard > > > On Mon, Mar 23, 2020 at 6:25 AM Martin Grigorov <martin.grigo...@gmail.com> > wrote: > >> Hi, >> >> >> On Wed, Mar 18, 2020 at 5:31 PM Martin Grigorov < >> martin.grigo...@gmail.com> wrote: >> >>> Hi, >>> >>> On Thu, Mar 12, 2020 at 4:35 PM Martin Grigorov < >>> martin.grigo...@gmail.com> wrote: >>> >>>> Hi Guillaume, >>>> >>>> On Thu, Mar 12, 2020 at 3:23 PM Guillaume Quintard < >>>> guilla...@varnish-software.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> Offering arm64 packages requires a few things: >>>>> - arm64-compatible code (all good in >>>>> https://github.com/varnishcache/varnish-cache) >>>>> - arm64-compatible package framework (all good in >>>>> https://github.com/varnishcache/pkg-varnish-cache) >>>>> - infrastructure to build the packages (uhoh, see below) >>>>> - infrastructure to store and deliver ( >>>>> https://packagecloud.io/varnishcache) >>>>> >>>>> So, everything is in place, expect for the third point. At the moment, >>>>> there are two concurrent CI implementations: >>>>> - travis: >>>>> https://github.com/varnishcache/varnish-cache/blob/master/.travis.yml It's >>>>> the historical one, and currently only runs compilation+test for OSX >>>>> >>>> >>>> Actually it tests Linux AMD64 and ARM64 too. >>>> >>>> >>>>> - circleci: >>>>> https://github.com/varnishcache/varnish-cache/blob/master/.circleci/config.yml >>>>> the >>>>> new kid on the block, that builds all the packages and distchecks for all >>>>> the packaged platforms >>>>> >>>>> The issue is that cirecleci doesn't support arm64 containers (for >>>>> now?), so we would need to re-implement the packaging logic in Travis. >>>>> It's >>>>> not a big problem, but it's currently not a priority on my side. >>>>> >>>>> However, I am totally ready to provide help if someone wants to take >>>>> that up. The added benefit it that Travis would be able to handle >>>>> everything and we can retire the circleci experiment >>>>> >>>> >>>> I will take a look in the coming days and ask you if I need help! >>>> >>> >>> I've took a look at the current setup and here is what I've found as >>> problems and possible solutions: >>> >>> 1) Circle CI >>> 1.1) problem - the 'machine' and 'Docker' executors run on x86_64, so >>> there is no way to build the packages in a "native" environment >>> 1.2) possible solutions >>> 1.2.1) use multiarch cross build >>> 1.2.2) use 'machine' executor that registers QEMU via >>> https://hub.docker.com/r/multiarch/qemu-user-static/ and then builds >>> and runs a custom Docker image that executes a shell script with the build >>> steps >>> It will look something like >>> https://github.com/yukimochi-containers/alpine-vpnserver/blob/69bb0a612c9df3e4ba78064d114751b760f0df9d/.circleci/config.yml#L19-L38 >>> but >>> instead of uploading the Docker image as a last step it will run it. >>> The RPM and DEB build related code from current config.yml will be >>> extracted into shell scripts which will be copied in the custom Docker >>> images >>> >>> From these two possible ways I have better picture in my head how to do >>> 1.2.2, but I don't mind going deep in 1.2.1 if this is what you'd prefer. >>> >> >> I've decided to stay with Circle CI and use 'machine' executor with QEMU. >> >> The changed config.yml could be seen at >> https://github.com/martin-g/varnish-cache/tree/feature/aarch64-packages/.circleci >> and >> the build at >> https://app.circleci.com/pipelines/github/martin-g/varnish-cache/71/workflows/3a275d79-62a9-48b4-9aef-1585de1c87c8 >> The builds on x86 arch take 3-4 mins, but for aarch64 (emulation!) ~40mins >> For now the jobs just build the .deb & .rpm packages for CentOS 7 and >> Ubuntu 18.04, both amd64 and aarch64. >> TODOs: >> - migrate Alpine >> - store the packages as CircleCI artifacts >> - anything else that is still missing >> >> Adding more architectures would be as easy as adding a new Dockerfile >> with a base image from the respective type. >> >> Martin >> >> >>> 2) Travis CI >>> 2.1) problems >>> 2.1.1) generally Travis is slower than Circle! >>> Althought if we use CircleCI 'machine' executor it will be slower than >>> the current 'Docker' executor! >>> 2.1.2) Travis supports only Ubuntu >>> Current setup at CircleCI uses CentOS 7. >>> I guess the build steps won't have problems on Ubuntu. >>> >>> 3) GitHub Actions >>> GH Actions does not support ARM64 but it supports self hosted ARM64 >>> runners >>> 3.1) The problem is that there is no way to make a self hosted runner >>> really private. I.e. if someone forks Varnish Cache any commit in the fork >>> will trigger builds on the arm64 node. There is no way to reserve the >>> runner only for commits against >>> https://github.com/varnishcache/varnish-cache >>> >>> Do you see other problems or maybe different ways ? >>> Do you have preferences which way to go ? >>> >>> Regards, >>> Martin >>> >>> >>>> >>>> Regards, >>>> Martin >>>> >>>> >>>>> >>>>> -- >>>>> Guillaume Quintard >>>>> _______________________________________________ >>>>> varnish-dev mailing list >>>>> varnish-dev@varnish-cache.org >>>>> https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev >>>>> >>>>
_______________________________________________ varnish-dev mailing list varnish-dev@varnish-cache.org https://www.varnish-cache.org/lists/mailman/listinfo/varnish-dev