I'm not removing it, I am adding an in host crossbuild process. Results I have show that doing the full cross-building on the host is 10 minutes faster (and not twice faster like I said previously) that the 2 hours it takes with docker on prometheus/prometheus ( https://app.circleci.com/jobs/github/prometheus/prometheus/25973).
The real improvement (without sharing go cache across builds) comes from removing the "-a" flag in the go build flags. On Sun, 16 Feb 2020 at 16:17, Julien Pivotto <[email protected]> wrote: > What is the reasoning behind the removal of the docker build? > > ----- Original Message ----- > From: Sylvain Rabot <[email protected]> > To: Tobias Schmidt <[email protected]> > Cc: Simon Pasquier <[email protected]>, Ben Kochie <[email protected]>, > Chris Marchbanks <[email protected]>, Matthias Rampke < > [email protected]>, Krasimir Georgiev <[email protected]>, Prometheus > Developers <[email protected]> > Sent: Sun, 16 Feb 2020 14:19:55 +0100 (CET) > Subject: Re: [prometheus-developers] Reduce list of GOOS/GOARCH crossbuilt > for each PR > > On Sun, 16 Feb 2020 at 13:41, Tobias Schmidt <[email protected]> wrote: > > > > > > > On Sun, Feb 16, 2020 at 11:38 AM Sylvain Rabot <[email protected]> > > wrote: > > > >> I've also been looking into it. > >> > >> The sharing the go build cache would be the ultimate solution but for > >> prometheus, for all current goos/goarch, it accounts for more than 6GB > and > >> I believe that is too much for circleci to handle (they recommend > keeping > >> cache under 500MB). > >> > >> However I made a PR on promu that replace current crossbuild process > with > >> one that (amongst other things) does not use docker anymore and it seems > >> that it *halves* the duration of building all current goos/goarch: > >> > > > > How does this work for node_exporter with its CGO usage? > > > > I don't know, I'll try it. But if it does not work, I moved the docker > cross-building in a separate promu command ("promu crossbuild-docker") so > we could still use that for node_exporter. > > > > > > > >> > >> - https://github.com/prometheus/promu/pull/177 > >> - https://github.com/prometheus/prometheus/pull/6824 > >> - > >> > https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f > >> > >> Regards. > >> > >> On Fri, 14 Feb 2020 at 14:48, Simon Pasquier <[email protected]> > wrote: > >> > >>> FWIW I'm looking into this. From my understanding, the bulk of the > >>> issue is that "promu crossbuild" can't reuse the Go build cache > >>> between CI runs hence it rebuilds everything from scratch every time. > >>> > >>> On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie <[email protected]> wrote: > >>> > > >>> > Part of the problem is that we run the builds in serial. We have a > lot > >>> more compute capacity in our CircleCI for running more docker tasks in > >>> parallel, but promu doesn't know how to distribute the work. > >>> > > >>> > But I also think we could cut the build step down for PRs. But I > think > >>> we should keep the full build in master. > >>> > > >>> > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks < > >>> [email protected]> wrote: > >>> >> > >>> >> I also support this, waiting 2-3 hours for the build job to finish > is > >>> frustrating. I know that building on 32 bit architectures does not > catch > >>> all issues, specifically the alignment bug using the atomic package. > >>> Perhaps add at least one 32 bit build on the pull request though? > >>> >> > >>> >> Is it worth it to build everything on every master, or should a > build > >>> all job be added to the nightly build? I agree that we should build > >>> everything on a cadence more frequent than a release. > >>> >> > >>> >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus > >>> Developers <[email protected]> wrote: > >>> >>> > >>> >>> I would build everything on master, that way we catch before > >>> starting a release if there is something wrong. > >>> >>> > >>> >>> /MR > >>> >>> > >>> >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot < > >>> [email protected]> wrote: > >>> >>>> > >>> >>>> I did not say it but I was speaking of prometheus/prometheus, I > >>> haven't checked others repos for their full cross-building time. > >>> >>>> > >>> >>>> I think we can come up with a minimal list of GOOS/GOARCH for PRs > >>> but, if you think building the complete list on tags only is not > enough, we > >>> could do it on tags & master. > >>> >>>> > >>> >>>> If we were to choose to build the complete list for tags only I > >>> would suggest to build this for PRs: > >>> >>>> > >>> >>>> - linux/amd64 > >>> >>>> - linux/386 > >>> >>>> - linux/arm > >>> >>>> - linux/arm64 > >>> >>>> - darwin/amd64 > >>> >>>> - windows/amd64 > >>> >>>> - freebsd/amd64 > >>> >>>> - openbsd/amd64 > >>> >>>> - netbsd/amd64 > >>> >>>> - dragonfly/amd64 > >>> >>>> > >>> >>>> If we were to choose to build the complete list for tags & master > >>> then I would suggest an even more reduced one: > >>> >>>> > >>> >>>> - linux/amd64 > >>> >>>> - darwin/amd64 > >>> >>>> - windows/amd64 > >>> >>>> - freebsd/amd64 > >>> >>>> > >>> >>>> Regards. > >>> >>>> > >>> >>>> On Tue, 11 Feb 2020 at 23:17, Matthias Rampke <[email protected]> > >>> wrote: > >>> >>>>> > >>> >>>>> There are some exceptions like node exporter where it's important > >>> that all variants at least build, but that has a custom setup already. > >>> >>>>> > >>> >>>>> What would be a sufficient subset? Do we need to worry about > >>> endianness and 32 bit architectures, or would just building not catch > >>> issues specific to these anyway? > >>> >>>>> > >>> >>>>> /MR > >>> >>>>> > >>> >>>>> On Tue, 11 Feb 2020, 22:50 Krasimir Georgiev, < > [email protected]> > >>> wrote: > >>> >>>>>> > >>> >>>>>> I think that is a very good idea. > >>> >>>>>> > >>> >>>>>> On Feb 11 2020, at 11:19 pm, Sylvain Rabot < > >>> [email protected]> wrote: > >>> >>>>>> > >>> >>>>>> Hi, > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> I'm wondering if we could reduce the list of GOOS/GOARCH that > are > >>> crossbuilt for every PR by circle-ci. > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> The building of the complete list seems like a waste of time & > >>> resources to me. > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Maybe we could select a few and only build the complete list > when > >>> building tags ? > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> Regards. > >>> >>>>>> > >>> >>>>>> > >>> >>>>>> -- > >>> >>>>>> Sylvain Rabot <[email protected]> > >>> >>>>>> > >>> >>>>>> -- > >>> >>>>>> You received this message because you are subscribed to the > >>> Google Groups "Prometheus Developers" group. > >>> >>>>>> To unsubscribe from this group and stop receiving emails from > it, > >>> send an email to [email protected]. > >>> >>>>>> To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FJKyVj_gq-hgVgyyVbJ%3D-pECFqcPK-QviXmKB1R-oAgg%40mail.gmail.com > >>> . > >>> >>>>>> > >>> >>>>>> -- > >>> >>>>>> You received this message because you are subscribed to the > >>> Google Groups "Prometheus Developers" group. > >>> >>>>>> To unsubscribe from this group and stop receiving emails from > it, > >>> send an email to [email protected]. > >>> >>>>>> To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/prometheus-developers/2508CDF1-CC2A-4AC6-B9EE-D68B53AFF166%40getmailspring.com > >>> . > >>> >>>> > >>> >>>> > >>> >>>> > >>> >>>> -- > >>> >>>> Sylvain Rabot <[email protected]> > >>> >>> > >>> >>> -- > >>> >>> You received this message because you are subscribed to the Google > >>> Groups "Prometheus Developers" group. > >>> >>> To unsubscribe from this group and stop receiving emails from it, > >>> send an email to [email protected]. > >>> >>> To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5V6MiMGH32a4OB0KMfJmV7FBZbJWEe6HZ-z9%2BmiOkqusQ%40mail.gmail.com > >>> . > >>> >> > >>> >> -- > >>> >> You received this message because you are subscribed to the Google > >>> Groups "Prometheus Developers" group. > >>> >> To unsubscribe from this group and stop receiving emails from it, > >>> send an email to [email protected]. > >>> >> To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/prometheus-developers/CANVFovU3e_avaLtSnTwYzYz5v-Aj7M_VZSi%3DA05i90T3txZ6EA%40mail.gmail.com > >>> . > >>> > > >>> > -- > >>> > You received this message because you are subscribed to the Google > >>> Groups "Prometheus Developers" group. > >>> > To unsubscribe from this group and stop receiving emails from it, > send > >>> an email to [email protected]. > >>> > To view this discussion on the web visit > >>> > https://groups.google.com/d/msgid/prometheus-developers/CABbyFmr98ZO%2B0Tgw%2BY4XkjsTMJ68g0J_aKDBEzPMsc7E4Un%2BJg%40mail.gmail.com > >>> . > >>> > >>> > >> > >> -- > >> Sylvain Rabot <[email protected]> > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Prometheus Developers" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an > >> email to [email protected]. > >> To view this discussion on the web visit > >> > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GSQea1ATq6eE9crF73kWOsW_kRXB5nROPDHDhemCyvgA%40mail.gmail.com > >> < > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GSQea1ATq6eE9crF73kWOsW_kRXB5nROPDHDhemCyvgA%40mail.gmail.com?utm_medium=email&utm_source=footer > > > >> . > >> > > > > -- > Sylvain Rabot <[email protected]> > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1ExeX9Hd01A%3D5GJ%3Dth63WDz5Vev0eDzF-bRTxq1tAugyQ%40mail.gmail.com > . > > -- Sylvain Rabot <[email protected]> -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CADjtP1EcxdQSi10BaA949Pnj4%3DZJULF-yy%3DODCLOLrm6mmxQWQ%40mail.gmail.com.
