I've also been looking into it. The sharing the go build cache would be the ultimate solution but for prometheus, for all current goos/goarch, it accounts for more than 6GB and I believe that is too much for circleci to handle (they recommend keeping cache under 500MB).
However I made a PR on promu that replace current crossbuild process with one that (amongst other things) does not use docker anymore and it seems that it *halves* the duration of building all current goos/goarch: - https://github.com/prometheus/promu/pull/177 - https://github.com/prometheus/prometheus/pull/6824 - https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f Regards. On Fri, 14 Feb 2020 at 14:48, Simon Pasquier <spasq...@redhat.com> wrote: > FWIW I'm looking into this. From my understanding, the bulk of the > issue is that "promu crossbuild" can't reuse the Go build cache > between CI runs hence it rebuilds everything from scratch every time. > > On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie <sup...@gmail.com> wrote: > > > > Part of the problem is that we run the builds in serial. We have a lot > more compute capacity in our CircleCI for running more docker tasks in > parallel, but promu doesn't know how to distribute the work. > > > > But I also think we could cut the build step down for PRs. But I think > we should keep the full build in master. > > > > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks <csmarchba...@gmail.com> > wrote: > >> > >> I also support this, waiting 2-3 hours for the build job to finish is > frustrating. I know that building on 32 bit architectures does not catch > all issues, specifically the alignment bug using the atomic package. > Perhaps add at least one 32 bit build on the pull request though? > >> > >> Is it worth it to build everything on every master, or should a build > all job be added to the nightly build? I agree that we should build > everything on a cadence more frequent than a release. > >> > >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus > Developers <prometheus-developers@googlegroups.com> wrote: > >>> > >>> I would build everything on master, that way we catch before starting > a release if there is something wrong. > >>> > >>> /MR > >>> > >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot <sylv...@abstraction.fr> > wrote: > >>>> > >>>> I did not say it but I was speaking of prometheus/prometheus, I > haven't checked others repos for their full cross-building time. > >>>> > >>>> I think we can come up with a minimal list of GOOS/GOARCH for PRs > but, if you think building the complete list on tags only is not enough, we > could do it on tags & master. > >>>> > >>>> If we were to choose to build the complete list for tags only I would > suggest to build this for PRs: > >>>> > >>>> - linux/amd64 > >>>> - linux/386 > >>>> - linux/arm > >>>> - linux/arm64 > >>>> - darwin/amd64 > >>>> - windows/amd64 > >>>> - freebsd/amd64 > >>>> - openbsd/amd64 > >>>> - netbsd/amd64 > >>>> - dragonfly/amd64 > >>>> > >>>> If we were to choose to build the complete list for tags & master > then I would suggest an even more reduced one: > >>>> > >>>> - linux/amd64 > >>>> - darwin/amd64 > >>>> - windows/amd64 > >>>> - freebsd/amd64 > >>>> > >>>> Regards. > >>>> > >>>> On Tue, 11 Feb 2020 at 23:17, Matthias Rampke <m...@soundcloud.com> > wrote: > >>>>> > >>>>> There are some exceptions like node exporter where it's important > that all variants at least build, but that has a custom setup already. > >>>>> > >>>>> What would be a sufficient subset? Do we need to worry about > endianness and 32 bit architectures, or would just building not catch > issues specific to these anyway? > >>>>> > >>>>> /MR > >>>>> > >>>>> On Tue, 11 Feb 2020, 22:50 Krasimir Georgiev, <kgeor...@redhat.com> > wrote: > >>>>>> > >>>>>> I think that is a very good idea. > >>>>>> > >>>>>> On Feb 11 2020, at 11:19 pm, Sylvain Rabot <sylv...@abstraction.fr> > wrote: > >>>>>> > >>>>>> Hi, > >>>>>> > >>>>>> > >>>>>> I'm wondering if we could reduce the list of GOOS/GOARCH that are > crossbuilt for every PR by circle-ci. > >>>>>> > >>>>>> > >>>>>> The building of the complete list seems like a waste of time & > resources to me. > >>>>>> > >>>>>> > >>>>>> Maybe we could select a few and only build the complete list when > building tags ? > >>>>>> > >>>>>> > >>>>>> Regards. > >>>>>> > >>>>>> > >>>>>> -- > >>>>>> Sylvain Rabot <sylv...@abstraction.fr> > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, > send an email to prometheus-developers+unsubscr...@googlegroups.com. > >>>>>> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FJKyVj_gq-hgVgyyVbJ%3D-pECFqcPK-QviXmKB1R-oAgg%40mail.gmail.com > . > >>>>>> > >>>>>> -- > >>>>>> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >>>>>> To unsubscribe from this group and stop receiving emails from it, > send an email to prometheus-developers+unsubscr...@googlegroups.com. > >>>>>> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/2508CDF1-CC2A-4AC6-B9EE-D68B53AFF166%40getmailspring.com > . > >>>> > >>>> > >>>> > >>>> -- > >>>> Sylvain Rabot <sylv...@abstraction.fr> > >>> > >>> -- > >>> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >>> To unsubscribe from this group and stop receiving emails from it, send > an email to prometheus-developers+unsubscr...@googlegroups.com. > >>> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5V6MiMGH32a4OB0KMfJmV7FBZbJWEe6HZ-z9%2BmiOkqusQ%40mail.gmail.com > . > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > >> To unsubscribe from this group and stop receiving emails from it, send > an email to prometheus-developers+unsubscr...@googlegroups.com. > >> To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CANVFovU3e_avaLtSnTwYzYz5v-Aj7M_VZSi%3DA05i90T3txZ6EA%40mail.gmail.com > . > > > > -- > > You received this message because you are subscribed to the Google > Groups "Prometheus Developers" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to prometheus-developers+unsubscr...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CABbyFmr98ZO%2B0Tgw%2BY4XkjsTMJ68g0J_aKDBEzPMsc7E4Un%2BJg%40mail.gmail.com > . > > -- Sylvain Rabot <sylv...@abstraction.fr> -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to prometheus-developers+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GSQea1ATq6eE9crF73kWOsW_kRXB5nROPDHDhemCyvgA%40mail.gmail.com.
