I've also been looking into it. The sharing the go build cache would be the ultimate solution but for prometheus, for all current goos/goarch, it accounts for more than 6GB and I believe that is too much for circleci to handle (they recommend keeping cache under 500MB).
However I made a PR on promu that replace current crossbuild process with one that (amongst other things) does not use docker anymore and it seems that it *halves* the duration of building all current goos/goarch: - https://github.com/prometheus/promu/pull/177 - https://github.com/prometheus/prometheus/pull/6824 - https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f Regards. On Fri, 14 Feb 2020 at 14:48, Simon Pasquier <[email protected]> wrote: > FWIW I'm looking into this. From my understanding, the bulk of the > issue is that "promu crossbuild" can't reuse the Go build cache > between CI runs hence it rebuilds everything from scratch every time. > > On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie <[email protected]> wrote: > > > > Part of the problem is that we run the builds in serial. We have a lot > more compute capacity in our CircleCI for running more docker tasks in > parallel, but promu doesn't know how to distribute the work. > > > > But I also think we could cut the build step down for PRs. But I think > we should keep the full build in master. > > > > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks <[email protected]> > wrote: > >> > >> I also support this, waiting 2-3 hours for the build job to finish is > frustrating. I know that building on 32 bit architectures does not catch > all issues, specifically the alignment bug using the atomic package. > Perhaps add at least one 32 bit build on the pull request though? > >> > >> Is it worth it to build everything on every master, or should a build > all job be added to the nightly build? I agree that we should build > everything on a cadence more frequent than a release. > >> > >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus > Developers <[email protected]> wrote: > >>> > >>> I would build everything on master, that way we catch before starting > a release if there is something wrong. > >>> > >>> /MR > >>> > >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot <[email protected]> > wrote: > >>>> > >>>> I did not say it but I was speaking of prometheus/prometheus, I > haven't checked others repos for their full cross-building time. > >>>> > >>>> I think we can come up with a minimal list of GOOS/GOARCH for PRs > but, if you think building the complete list on tags only is not enough, we > could do it on tags & master. > >>>> > >>>> If we were to choose to build the complete list for tags only I would > suggest to build this for PRs: > >>>> > >>>> - linux/amd64 > >>>> - linux/386 > >>>> - linux/arm > >>>> - linux/arm64 > >>>> - darwin/amd64 > >>>> - windows/amd64 > >>>> - freebsd/amd64 > >>>> - openbsd/amd64 > >>>> - netbsd/amd64 > >>>> - dragonfly/amd64 > >>>> > >>>> If we were to choose to build the complete list for tags & master > then I would suggest an even more reduced one: > >>>> > >>>> - linux/amd64 > >>>> - darwin/amd64 > >>>> - windows/amd64 > >>>> - freebsd/amd64 > >>>> > >>>> Regards. > >>>> > >>>> On Tue, 11 Feb 2020 at 23:17, Matthias Rampke <[email protected]> > wrote: > >>>>> > >>>>> There are some exceptions like node exporter where it's important > that all variants at least build, but that has a custom setup already. > >>>>> > >>>>> What would be a sufficient subset? Do we need to worry about > endianness and 32 bit architectures, or would just building not catch > issues specific to these anyway? > >>>>> > >>>>> /MR > >>>>> > >>>>> On Tue, 11 Feb 2020, 22:50 Krasimir Georgiev, <[email protected]> > wrote: > >>>>>> > >>>>>> I think that is a very good idea. > >>>>>> > >>>>>> On Feb 11 2020, at 11:19 pm, Sylvain Rabot <[email protected]> > wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> > >>>>>> I'm wondering if we could reduce the list of GOOS/GOARCH that are > crossbuilt for every PR by circle-ci. > >>>>>> > >>>>>> > >>>>>> The building of the complete list seems like a waste of time & > resources to me. > >>>>>> > >>>>>> > >>>>>> Maybe we could select a few and only build the complete list when > building tags ? > >>>>>> > >>>>>> > >>>>>> Regards. > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Sylvain Rabot <[email protected]> > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected]. > >>>>>> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FJKyVj_gq-hgVgyyVbJ%3D-pECFqcPK-QviXmKB1R-oAgg%40mail.gmail.com > . > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, > send an email to [email protected]. > >>>>>> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/2508CDF1-CC2A-4AC6-B9EE-D68B53AFF166%40getmailspring.com > . > >>>> > >>>> > >>>> > >>>> -- > >>>> Sylvain Rabot <[email protected]> > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > >>> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5V6MiMGH32a4OB0KMfJmV7FBZbJWEe6HZ-z9%2BmiOkqusQ%40mail.gmail.com > . > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > >> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CANVFovU3e_avaLtSnTwYzYz5v-Aj7M_VZSi%3DA05i90T3txZ6EA%40mail.gmail.com > . > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CABbyFmr98ZO%2B0Tgw%2BY4XkjsTMJ68g0J_aKDBEzPMsc7E4Un%2BJg%40mail.gmail.com > . > > -- Sylvain Rabot <[email protected]> -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GSQea1ATq6eE9crF73kWOsW_kRXB5nROPDHDhemCyvgA%40mail.gmail.com.
