On Sun, Feb 16, 2020 at 11:38 AM Sylvain Rabot <[email protected]> wrote:
> I've also been looking into it. > > The sharing the go build cache would be the ultimate solution but for > prometheus, for all current goos/goarch, it accounts for more than 6GB and > I believe that is too much for circleci to handle (they recommend keeping > cache under 500MB). > > However I made a PR on promu that replace current crossbuild process with > one that (amongst other things) does not use docker anymore and it seems > that it *halves* the duration of building all current goos/goarch: > How does this work for node_exporter with its CGO usage? > > - https://github.com/prometheus/promu/pull/177 > - https://github.com/prometheus/prometheus/pull/6824 > - > https://app.circleci.com/github/prometheus/prometheus/pipelines/838ffe3c-b738-430b-8cf0-c707842e7e8f/workflows/dd50dabc-04e3-4b48-8a36-c85456668c2f > > Regards. > > On Fri, 14 Feb 2020 at 14:48, Simon Pasquier <[email protected]> wrote: > >> FWIW I'm looking into this. From my understanding, the bulk of the >> issue is that "promu crossbuild" can't reuse the Go build cache >> between CI runs hence it rebuilds everything from scratch every time. >> >> On Thu, Feb 13, 2020 at 5:36 PM Ben Kochie <[email protected]> wrote: >> > >> > Part of the problem is that we run the builds in serial. We have a lot >> more compute capacity in our CircleCI for running more docker tasks in >> parallel, but promu doesn't know how to distribute the work. >> > >> > But I also think we could cut the build step down for PRs. But I think >> we should keep the full build in master. >> > >> > On Wed, Feb 12, 2020 at 3:50 PM Chris Marchbanks < >> [email protected]> wrote: >> >> >> >> I also support this, waiting 2-3 hours for the build job to finish is >> frustrating. I know that building on 32 bit architectures does not catch >> all issues, specifically the alignment bug using the atomic package. >> Perhaps add at least one 32 bit build on the pull request though? >> >> >> >> Is it worth it to build everything on every master, or should a build >> all job be added to the nightly build? I agree that we should build >> everything on a cadence more frequent than a release. >> >> >> >> On Wed, Feb 12, 2020 at 1:58 AM 'Matthias Rampke' via Prometheus >> Developers <[email protected]> wrote: >> >>> >> >>> I would build everything on master, that way we catch before starting >> a release if there is something wrong. >> >>> >> >>> /MR >> >>> >> >>> On Wed, Feb 12, 2020 at 8:37 AM Sylvain Rabot <[email protected]> >> wrote: >> >>>> >> >>>> I did not say it but I was speaking of prometheus/prometheus, I >> haven't checked others repos for their full cross-building time. >> >>>> >> >>>> I think we can come up with a minimal list of GOOS/GOARCH for PRs >> but, if you think building the complete list on tags only is not enough, we >> could do it on tags & master. >> >>>> >> >>>> If we were to choose to build the complete list for tags only I >> would suggest to build this for PRs: >> >>>> >> >>>> - linux/amd64 >> >>>> - linux/386 >> >>>> - linux/arm >> >>>> - linux/arm64 >> >>>> - darwin/amd64 >> >>>> - windows/amd64 >> >>>> - freebsd/amd64 >> >>>> - openbsd/amd64 >> >>>> - netbsd/amd64 >> >>>> - dragonfly/amd64 >> >>>> >> >>>> If we were to choose to build the complete list for tags & master >> then I would suggest an even more reduced one: >> >>>> >> >>>> - linux/amd64 >> >>>> - darwin/amd64 >> >>>> - windows/amd64 >> >>>> - freebsd/amd64 >> >>>> >> >>>> Regards. >> >>>> >> >>>> On Tue, 11 Feb 2020 at 23:17, Matthias Rampke <[email protected]> >> wrote: >> >>>>> >> >>>>> There are some exceptions like node exporter where it's important >> that all variants at least build, but that has a custom setup already. >> >>>>> >> >>>>> What would be a sufficient subset? Do we need to worry about >> endianness and 32 bit architectures, or would just building not catch >> issues specific to these anyway? >> >>>>> >> >>>>> /MR >> >>>>> >> >>>>> On Tue, 11 Feb 2020, 22:50 Krasimir Georgiev, <[email protected]> >> wrote: >> >>>>>> >> >>>>>> I think that is a very good idea. >> >>>>>> >> >>>>>> On Feb 11 2020, at 11:19 pm, Sylvain Rabot <[email protected]> >> wrote: >> >>>>>> >> >>>>>> Hi, >> >>>>>> >> >>>>>> >> >>>>>> I'm wondering if we could reduce the list of GOOS/GOARCH that are >> crossbuilt for every PR by circle-ci. >> >>>>>> >> >>>>>> >> >>>>>> The building of the complete list seems like a waste of time & >> resources to me. >> >>>>>> >> >>>>>> >> >>>>>> Maybe we could select a few and only build the complete list when >> building tags ? >> >>>>>> >> >>>>>> >> >>>>>> Regards. >> >>>>>> >> >>>>>> >> >>>>>> -- >> >>>>>> Sylvain Rabot <[email protected]> >> >>>>>> >> >>>>>> -- >> >>>>>> You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> >>>>>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>>>>> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CADjtP1FJKyVj_gq-hgVgyyVbJ%3D-pECFqcPK-QviXmKB1R-oAgg%40mail.gmail.com >> . >> >>>>>> >> >>>>>> -- >> >>>>>> You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> >>>>>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>>>>> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/2508CDF1-CC2A-4AC6-B9EE-D68B53AFF166%40getmailspring.com >> . >> >>>> >> >>>> >> >>>> >> >>>> -- >> >>>> Sylvain Rabot <[email protected]> >> >>> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> >>> To unsubscribe from this group and stop receiving emails from it, >> send an email to [email protected]. >> >>> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CAFU3N5V6MiMGH32a4OB0KMfJmV7FBZbJWEe6HZ-z9%2BmiOkqusQ%40mail.gmail.com >> . >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> >> To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CANVFovU3e_avaLtSnTwYzYz5v-Aj7M_VZSi%3DA05i90T3txZ6EA%40mail.gmail.com >> . >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Prometheus Developers" group. >> > To unsubscribe from this group and stop receiving emails from it, send >> an email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/prometheus-developers/CABbyFmr98ZO%2B0Tgw%2BY4XkjsTMJ68g0J_aKDBEzPMsc7E4Un%2BJg%40mail.gmail.com >> . >> >> > > -- > Sylvain Rabot <[email protected]> > > -- > You received this message because you are subscribed to the Google Groups > "Prometheus Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GSQea1ATq6eE9crF73kWOsW_kRXB5nROPDHDhemCyvgA%40mail.gmail.com > <https://groups.google.com/d/msgid/prometheus-developers/CADjtP1GSQea1ATq6eE9crF73kWOsW_kRXB5nROPDHDhemCyvgA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Prometheus Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/prometheus-developers/CAChBsdBRgJ_a62UKkaO9iwdSMG-3FW2%2BcGYE%2BYP9hRhtEQbn0w%40mail.gmail.com.
