Steve George <[email protected]> writes: > On Wed, Jan 28, 2026 at 11:57:20AM +0000, Christopher Baines wrote: >> Steve George <[email protected]> writes: >> >> > One of our current challenges is that we don't want to commit changes >> > to the master branch that cause large rebuilds. As I understand it the >> > main issue is that the build farm takes a while to build all the >> > dependent packages and in the meantime uses that do guix pull won't >> > have substitutes so will have to build locally. >> > >> > Nix seems to solve this by having developers push to a different >> > branch from the one that users pull from [0]. This seems like a good >> > solution, has Guix tried this previously, am I missing something? >> >> The way I see it at least is a balance between reducing churn, and >> trying to do continuous integration (working on master). >> >> Large rebuilds often don't lead to packages changing in behaviour, but >> the store items changing does mean that users need to build or download >> more things, as well as storing more items locally (deduplication does >> help a lot with the storage). This is what I think of as churn. >> >> Part of addressing this comes from improving the package definitions, >> and avoiding unnecessary changes to unrelated packages. For example, if >> I were making a whole bunch of Ruby related changes, and that happeend >> to cause a rebuild of the whole of the texlive packages, that would be >> something to avoid by changing the packagee definitions. >> >> Both the bordeaux build farm and ci.guix.gnu.org provide substitutes for >> non-master branches, although this functionality for the bordeaux build >> farm is relying on the guix-patches bugs, which should have been >> switched off at the end of last year. With QA we previously had >> substitutes immediately for both small patches and large branches. The >> automated patch testing required building things, and those built things >> were made available for people to download. > (...) > > Right, so I think your first point is that our package definitions are > not tight enough leading to an overly complicated dependency tree > which in turn causes large numbers of rebuilds. So we could reduce > 'churn' by improving our package definitions. > > Fair enough, maybe that's a separate topic because I guess we need > tooling to help developers reduce dependencies and/or we have to split > packages more so that they tree can be more atomic?
Indeed, this is something I'd like to see the data service help with more. >> > A rough proposal for how this would work: >> > >> > 1. Developers push to a new branch (e.g. 'main') >> > 2. CI builds each commit >> > 3a. If the build is successful the commit is updated on master >> > 3b. The substitute from the build is available to users >> > 4. User pull down from master receiving the new commit and any associated >> > substitute >> >> As I say, we already had this with QA, and I think that was better since >> the build information could be considered before merging things, and the >> setup was simpler since things were merged directly to master (or >> another branch). >> >> > One potential problem is how long it takes the build-farm to do a >> > world rebuild or a large rebuild. My impression is that a single world >> > rebuild could take two days or more, so if two commits caused that >> > then the whole flow of changes would fall behind. Do we agree that >> > build resources is a bottle-neck? >> >> I don't see it this way, as I see it as a balance between churn and >> trying to get things merged in quickly. Even if the build farm was >> infinitely fast, you'd still want to limit the churn as this would cause >> users to have to build or download more. > (...) > > Exclude 'improving packages to reduce the dependency tree' (churn) for a > moment. > > Do you think our velocity is being limited by the 'build farms' speed? > What I see devs do is limit making changes to packages 'deep' in the > dependency tree because it will cause a 'world rebuild'. Especially now that we've lost QA testing patches and branches (to some extent), I don't think so, since it's now back to being more of an issue of how the build farms are used. > Second, there's the issue of the user impact of substitutes not being > available immediately on master. AFAIK every 'commit' (update to a > package) means the 'build farm' has to build it and all > dependencies. When it's done the build the binary it built > (substitute) is available to users. At the moment whenever someone > commits to 'master' there's a period of time where the binary > substitute is not available. So we continually put users in the > position where a 'guix pull' may mean they have to build something > locally. The larger the number of dependents the longer it takes and > the larger the problem for end-users. Substitutes for packages should be immediately available, assuming that ci.guix.gnu.org or the bordeaux build farm has already built them. guix pull and the substitutes needed for it are a different problem though. > All things being equal, this idea means that users won't have that > problem since 'guix pull' only has commmits which have substitutes > that are built. It also has 'quality' improvements since we would only > provide the commit to the user if the build was successful. I think if we get back to a point where patches and branches are built for testing prior to being merged, then we should have a solution to that again, it seemed to be working well before (specifically getting substitutes immediately when changes are merged).
signature.asc
Description: PGP signature
