Hi, One of our current challenges is that we don't want to commit changes to the master branch that cause large rebuilds. As I understand it the main issue is that the build farm takes a while to build all the dependent packages and in the meantime uses that do guix pull won't have substitutes so will have to build locally.
Nix seems to solve this by having developers push to a different branch from the one that users pull from [0]. This seems like a good solution, has Guix tried this previously, am I missing something? A rough proposal for how this would work: 1. Developers push to a new branch (e.g. 'main') 2. CI builds each commit 3a. If the build is successful the commit is updated on master 3b. The substitute from the build is available to users 4. User pull down from master receiving the new commit and any associated substitute One potential problem is how long it takes the build-farm to do a world rebuild or a large rebuild. My impression is that a single world rebuild could take two days or more, so if two commits caused that then the whole flow of changes would fall behind. Do we agree that build resources is a bottle-neck? Nix has different sets of packages (nixos-unstable, nixos-unstable-small) with associated builds [1] - the value of the 'small' channel is that they update faster as there are fewer packages for their CI to build. Guix could implement the same pragmatic approach using the release manifests and augmenting them. The packages in this set would be built at the highest priority. The way this works in Nix is this smaller set is a branch that users can pull from. Another issue is that Nix specifically mentions that build failures can block their main branch. They try to avoid this and have some tooling so that the build status can be queried. For users they suggest that you build a local version if this happens. This seems to be a social/tooling area that we could improve generally - at the moment I think we sometimes commit things that break packages and I personally don't know of an easy way to really discover that. Yes, I know I can go check the pages but their slow and honestly very confusing. One thing I don't understand is how substitutes (3b) are made available to users in Guix? On the Nix page they talk about copying them around, do we do something similar? I'm not suggesting that we do away with team branches, there are definitely times where a 'feature branch' is useful. However, there are lots of packages where there isn't a team covering them and simply updating packages with 'large rebuilds' still seems to be a constant issue in our current flow. I recall that out of date packages was the biggest issue that users identified when we surveyed them - if we came up with a flow that let us update packages faster while also making substitutes for those packages available to our users, that would be big win. Thoughts, issues and other ideas? Steve / Futurile [0] https://nixos.wiki/wiki/Nix_channels [1] https://status.nixos.org/
