On 2020-06-17, Marc Espie <[email protected]> wrote: > The only way you end up with broken installations is when porters don't do > their jobs, that is they fail to bump a shared library or something like > that.
They do still break in some cases: libA depends on libB someapp depends on libA, libB libB has a major bump. libA gets updated but someapp is missed due to a mirror with an incomplete update. Now someapp wants old libB, but libA wants new libB, resulting in breakage. (There are also situations where some installed packages are broken for part of a pkg_add -u run, though they do sort themselves out later - I forget the details but I think it was something to do with the timing of ldconfig runs, updates to things like glib2/pango often do this). >> Even with snapshot shearing though, having this index file could provide a >> substantial speed upgrade. Instead of having to check *all* installed >> package's header for updates, you could use the index to know the subset of >> packages that you expect to have actually changed, and only download *those* >> packages' headers. If the expected "combined" sha of a given package doesn't >> match the index's version, then the mirror is clearly out of sync and we >> could abort an update as usual. > > > The problem is the multiple RTT...! if you manage to keep one single > connection open, you get a substantial speed upgrade. > > Generating a single index is more problematic than it would seem at first o > glance. This is already a problem when pkg_add fetches the directory listing (though a smaller one because the filenames don't change as often). Firstly the contents of the mirror can change during the pkg_add run so the listing becomes invalid, that can happen on any mirror. Secondly if the mirror involves a cache (CDNs) they will often cache the directory listing as an object - the other files in the directory can be out of sync with regard to the served index page. As the index is not too big and is hit on every pkg_add run against that mirror, it's highly likely to be cached, more so than probably anything else in the directory except the quirks package. > When everything goes fine, you're happy. > > If anything is out-of-synch, you're in deep shit. Sometimes things get in a state where a mirror can't get a complete sync at all, we've had times where it's been many days / week+ on fanout->L2s so nobody gets updates. One question is what to do in that situation. When it's like that, depending on what changed, updating any packages at all can cause quite bad breakage. > This would mean having several possible modes to cope with that, we don't > have enough resources to do that. > >
