On 2025-10-09, jslee <[email protected]> wrote: > I’d like to know more about the complications.
1. the point at which there's knowledge of new versions which are ready to go is the signing machine for the relevant subset of files. config on these machines is intentionally simple and they don't have network access except to the origins so there's not a good way to pass the information through 2. if someone starts fetching files from one build, and another build is then made available and uploaded, cache invalidation doesn't help you anyway. Before the cdn config was done, schemes have been discussed _at length_ about possible methods. Most obvious being to use a different dir for each build, with either a client-side-cached redirect, or a single 'director' file to point at the dir, and keeping an old version) but they don't play well with mirror feed-out: it's then hard for rsync to pick up old files to delta against. (--fuzzy is normally same-dir only; in a tighter ecosystem it's possible to do something with --copy-dest --fuzzy --fuzzy but that doesn't really work out with a few dozen mirror operators using their own infrastructure and tooling - probably more than half of people asking to be listed as a mirror aren't even able to provide files at the clearly documented ftp/http subdir so asking for anything like that is really too much). > At the point of uploading a new version of a file, the process knows the > entire URL path portion of the final URL, and the ‘cdn.openbsd.org’ part is > fixed, so it should be able to (a) know if the new file is different to the > old file, and (b) request an invalidation if it is. doesn't help with 2. and that is the killer really. > When I started working with CDNs in 2014 at a media company, Akamai took > (IIRC) double-digit minutes to do a global invalidation. while related, program code distribution via CDN, especially in ~10GB chunks of ~10k URLs changing several times a week as for snapshot packages, is a bit different to usual situations around media. > how many people have access to that infrastructure? and those people are > likely busy with a lot of other more important things. something like three, and yes, and without something clever for 2 it doesn't seem worth doing anything much else. snaps already have low-ish ttl before checking the origin for changes and anything else is a much more difficult problem. -- Please keep replies on the mailing list.
