I am looking for feedback/sponsors for a new GCD proposal to set up one or more international (outside EU) binary substitute servers in order to improve download speeds.
The PR is not ready yet because the proposal needs to be a more concrete plan. I would like to collect people's feedback before updating it - if you all think this issue is important, whether colocation vs. cloud would be preferable, where the servers should be located, or any other comments/concerns. The working repository is here: https://codeberg.org/antero/guix-consensus-documents/src/branch/wip-substitutes Here is the initial markdown text: title: International substitute servers id: 6 status: draft discussion: <draft|submitted|accepted|withdrawn|deprecated> authors: Antero Mejr sponsors: <Sponsor Name> date: <date when the discussion period starts> draft-date: 2025-08-07 discussion-date: <date when the discussion period starts> deliberation-date: <date when the deliberation starts> SPDX-License-Identifier: CC-BY-SA-4.0 OR GFDL-1.3-no-invariants-or-later --- # Summary The official binary substitute servers for Guix, `ci.guix.gnu.org` and `bordeaux`, are both located in Western Europe. Users outside Western Europe may experience slower speeds when downloading substitutes: 50-200 kbps is not uncommon. Since substitutes can be quite large, this causes significant slow down when performing package operations. This RFC proposes the establishment of additional substitute servers in other regions. # Motivation In the present state, users outside Europe experience substitute download speeds on the order of kilobits per second. This comprises some of the perceived "slowness" of Guix, as mentioned in all three of the [Guix surveys](https://guix.gnu.org/en/blog/2025/guix-user-and-contributor-survey-2024-the-results-part-1/). Other package managers with distributed substitute servers achieve download speeds of tens or even hundreds of megabytes per second, many times faster than the single Guix substitute server can provide. Common dependency packages such as LLVM can be large, around 50-200 MB. Downloading substitutes for them on such a slow connection will take at least 5 minutes, and that is only for a single dependency. This is a significant impediment to package management operations and workflows. Distributing substitutes can also increase reliability. As it stands, if `ci.guix.gnu.org` is not available, the user will not be able to download substitutes at all. It also assists with reproducibility: more identical substitute builds means a higher degree of assurance of deterministic, reproducible packages. # Detailed Design Many package managers make available dozens of substitute servers (often called "mirrors") from which to download compiled packages. Usually package managers have some sort of configuration option to select the mirror with the fastest connection, typically the mirror that is geographically closest, and possibly some sort of auto-detection of connection speeds. Guix does not currently have such a configuration option, and defaults to `ci.guix.gnu.org`. The addition of a mirror selection configuration option is part of this proposal. Such an option could be a `guix-configuration` option of `guix-service-type`. The Guix installer, for both the ISO and foreign-distro forms, would have an additional prompt to select a substitute server from a list. Since many Guix users and developers are based in the United States, a US-based server would be a sensible first step beyond the European region. From there community feedback can be solicited to determine locations for future servers, so long as sufficient funds are available. ## Cost of Reverting Cost scale: 0 - No incompatibility Users of Guix System would experience no change, as substitute server authentication is handled automatically. Users of Guix on foreign distros who are installing for the first time would experience no change, so long as they choose to authorize binary substitutes at install time. For users of Guix that is already installed on a foreign distro, they may need to follow [these steps](https://guix.gnu.org/manual/en/html_node/Substitute-Server-Authorization.html) to authorize the new substitute servers if they wish to use them. Otherwise they would not see the benefits. No deprecation of existing substitute servers would be necessary. Depending on whether colocation is used, maintenance of the substitute server may require a Guix developer in the geographic area of the server, with proper vetting/access clearance to perform administrative tasks directly on the hardware. Remote maintenance of the server(s) would be possible for most operations. Should a new substitute server need to be decommissioned (for whatever reason), there would need to be a process established for doing so without breaking compatibility. This would involve having the decommissioned substitute server URL redirect to a different active and authorized substitute server. # Drawbacks and Open Issues The main drawback of maintaining additional substitute servers is cost, in both the initial investment and recurring costs. For example, Nix serves their infrastructure through [Amazon AWS](https://discourse.nixos.org/t/s3-sponsorship-extension-more-resources-to-build-a-more-sustainable-nix/50936) and their recurring costs are $15,000 per month. However, Guix could potentially mitigate some of the costs. Making use of donated hardware, and using colocation instead of the cloud could reduce the monthly spend. Also the FSF may have resources that can provide servers or colocation services at a reduced price. Guix also has an FSF [Working Together Fund](https://my.fsf.org/civicrm/contribute/transact?reset=1&id=50) that could potentially be used to fund substitute servers. The [Guix Foundation](https://foundation.guix.info/) may also be interested in providing financial support for international substitute servers.
