Steve George <[email protected]> writes: > One of our current challenges is that we don't want to commit changes > to the master branch that cause large rebuilds. As I understand it the > main issue is that the build farm takes a while to build all the > dependent packages and in the meantime uses that do guix pull won't > have substitutes so will have to build locally. > > Nix seems to solve this by having developers push to a different > branch from the one that users pull from [0]. This seems like a good > solution, has Guix tried this previously, am I missing something?
The way I see it at least is a balance between reducing churn, and trying to do continuous integration (working on master). Large rebuilds often don't lead to packages changing in behaviour, but the store items changing does mean that users need to build or download more things, as well as storing more items locally (deduplication does help a lot with the storage). This is what I think of as churn. Part of addressing this comes from improving the package definitions, and avoiding unnecessary changes to unrelated packages. For example, if I were making a whole bunch of Ruby related changes, and that happeend to cause a rebuild of the whole of the texlive packages, that would be something to avoid by changing the packagee definitions. Both the bordeaux build farm and ci.guix.gnu.org provide substitutes for non-master branches, although this functionality for the bordeaux build farm is relying on the guix-patches bugs, which should have been switched off at the end of last year. With QA we previously had substitutes immediately for both small patches and large branches. The automated patch testing required building things, and those built things were made available for people to download. > A rough proposal for how this would work: > > 1. Developers push to a new branch (e.g. 'main') > 2. CI builds each commit > 3a. If the build is successful the commit is updated on master > 3b. The substitute from the build is available to users > 4. User pull down from master receiving the new commit and any associated > substitute As I say, we already had this with QA, and I think that was better since the build information could be considered before merging things, and the setup was simpler since things were merged directly to master (or another branch). > One potential problem is how long it takes the build-farm to do a > world rebuild or a large rebuild. My impression is that a single world > rebuild could take two days or more, so if two commits caused that > then the whole flow of changes would fall behind. Do we agree that > build resources is a bottle-neck? I don't see it this way, as I see it as a balance between churn and trying to get things merged in quickly. Even if the build farm was infinitely fast, you'd still want to limit the churn as this would cause users to have to build or download more. Speed is relevant when trying to get quick feedback, but this is more when the build farms are used for testing changes. Things could be improved, but from my perspective at least, it's not the most important issue.
signature.asc
Description: PGP signature
