On Wed, Jan 28, 2026 at 01:30:01PM +0000, Steve George wrote: > On Wed, Jan 28, 2026 at 11:57:20AM +0000, Christopher Baines wrote: > > Steve George <[email protected]> writes: > > > > > One of our current challenges is that we don't want to commit changes > > > to the master branch that cause large rebuilds. As I understand it the > > > main issue is that the build farm takes a while to build all the > > > dependent packages and in the meantime uses that do guix pull won't > > > have substitutes so will have to build locally. > > > > > > Nix seems to solve this by having developers push to a different > > > branch from the one that users pull from [0]. This seems like a good > > > solution, has Guix tried this previously, am I missing something? > > > > The way I see it at least is a balance between reducing churn, and > > trying to do continuous integration (working on master). > > > > Large rebuilds often don't lead to packages changing in behaviour, but > > the store items changing does mean that users need to build or download > > more things, as well as storing more items locally (deduplication does > > help a lot with the storage). This is what I think of as churn. > > > > Part of addressing this comes from improving the package definitions, > > and avoiding unnecessary changes to unrelated packages. For example, if > > I were making a whole bunch of Ruby related changes, and that happeend > > to cause a rebuild of the whole of the texlive packages, that would be > > something to avoid by changing the packagee definitions. (...)
Talked to Chris on IRC and learnt some things about this that might be obvious to others but wasn't to me :-) The way the build system works is that it looks at the state of the repository after each push. So, lets say we have 10 package updates/commits (5 are leaf packages; 5 which each cause a rebuild of Tex). The worse case scenario would be doing 10 separate pushes to the archive: each one would cause a rebuild and the last 5 would rebuild Tex 5 times. The most efficient is a single push since the everything gets built and the rebuilding of Tex only happens once. The intermediate path is that the leaf package updates can be pushed one at a time (there's no impact) and the big rebuilds are pushed once. This also impacts some of the team branches where due to the interconnected dependency tree we have branches that are constantly building large parts of the graph. I guess we have a bit of 'conways law' happening because we're organised by teams which are somewhat related to build-systems/language areas. Ideally we'd avoid those rebuilds by grouping them along the dependency graph. Teams would be responsible along the lines of the build graph. The other option being to reduce dependencies between packages and/or make the dependency lighter-weight. One option is more use of 'pinned' variables, and the other would be to split packages so you don't rebuild the whole of something massive when it's just a package dependency. Ideally we would try to find ways to reduce dependencies at key points/big packages in the graph. See the IRC log if you're interested in some other information about how CI/Cuirass handle substitutes: https://logs.guix.gnu.org/guix/2026-01-28.log Steve / Futurile
signature.asc
Description: PGP signature
