Hi Will! I’ve been happy with the GitHub Actions approach to Windows CI for erlfdb. The submodule-based installation of third party actions hasn’t proven to be a significant contributor to the total runtime. I think adding this workflow to the glazier repo is a great idea.
I remain interested in a simplified option for Windows builds that uses pre-built packages for Erlang/OTP instead of relying on our own glazier-based artifacts. I’m under the impression that we build Erlang ourselves mostly for historical reasons, but I could be wrong. In the CI world we could simplify a few steps by using the setup-beam action if we went that route as well. But we don’t have to address that here. I have a few questions about the workflow — what’s the thinking behind the separate “coucholddeps” and “couchdb” jobs? — but we can save that for a PR if you like. Cheers, Adam > On Dec 9, 2021, at 4:06 AM, Will Young <lostnetwork...@gmail.com> wrote: > > Hi, > > I've been looking into github workflows primarily with an interest > in being able to run ci tests on some of the supported platforms on > changes to test a potential contribution isn't going to break them. > > In the case of windows everything is unbundled so glazier's process > is providing a couchdb package with spidermonkey+icu, otp+openssl, > where most of this is built from source. > > I've put put this process into a flow as several separate jobs that > deliver intermediate results using some workarounds from discussions > on slack, and could avoid many of the older workarounds in glazier by > using newer versions of the dependencies. > > An example of the resulting release binaries are here: > https://github.com/lostnet/couchdb/releases/tag/msdeps12 > > And the workflow that is generating them is here: > https://github.com/lostnet/couchdb/blob/msdeps12/.github/workflows/mslegacy.yml > > I think it would make sense to integrate this as a flow on the > couchdb-glazier repo to create an automated reference build and > binaries. > > Issues: > > 1. versioning for reliability and supply side attacks (as discussed on > slack): > > The runners get updated to new versions of things which break builds. > It is helpful though to be easily able to attempt to reproduce older > builds again to eliminate theories of the source of new build > failures. > Versions of things should also be locked down as should uses of third > party actions like erlfdb's flows do via git submodule pinning. There > is an inherent tradeoff that the install time of anything locked down > instead of from a runner's defaults adds to the runtime. > > 2. Using the flow for release binaries: > > Aside from (1) it would not be wise to use gpg keys on a cloud > service. Releases can be modified to add binaries so a couchdb member > could add signed binaries ideally using binaries built in the flow > itself if they can verify they were correctly built (or possibly > running the flow on their own clone or manually follow the flow > actions.) > The versions of dependencies and compilers used in the build are > newer than past builds. Where possible it would be best to verify > these versions as older versions usually have more quirks for > automating the build on modern runners. > > 3. For CI: > If glazier repo had these flows and releases, I would like to add > the ability to run the couchdbolddeps job as a manual action in the > couchdb repo similar to the ubuntu ci added with esr91pr. > > 4. There are still quite a few problems in the tests to investigate as > well as whether the resulting packages are valid (and whether > foundationdb is intended to be external or a bundled deliverable on > couch4?) > > At Any rate, I would appreciate any thoughts on automating the windows builds. > > Thanks, > Will Young
