On Wed, Jun 17, 2020 at 09:44:32AM -0400, Jeremy O'Brien wrote: > On Wed, Jun 17, 2020, at 08:47, Marc Espie wrote: > > On Wed, Jun 17, 2020 at 08:28:02AM -0400, Jeremy O'Brien wrote: > > > On Tue, Jun 16, 2020, at 21:02, Marc Espie wrote: > > > > > > > > The concept you need to understand is snapshot shearing. > > > > > > > > A full package snapshot is large enough that it's hard to guarantee that > > > > you will have a full snapshot on a mirror at any point in time. > > > > > > > > In fact, you will sometimes encounter a mix of two snapshots (not that > > > > often, > > > > recently, but still) > > > > > > > > Hence, the decision to not have a central index for all packages, but to > > > > keep (and trust) the actual meta-info within the packages proper. > > > > > > > > > > Sorry, I guess I should've responded to this as well. Isn't snapshot > > > shearing going to be a problem regardless of the existence of a single > > > central-index? For instance, pkg_add notices a chromium update, which > > > requires a newer version of a dependency that hasn't been propagated to > > > the mirror yet. > > > > > That's not a big problem, it just stops before updating... at least > > your installation doesn't get hosed. > > > > Sorry if I'm being dense here. I'm just trying to understand this completely. > > In the current implementation, does pkg_add recursively verify every > dependency of a package before installing *any* of them? Or does it install > dependency updates as it finds them? Because if it's the latter, I can still > envision scenarios that break a given package, such as: Package A depends on > B and C. B is sync'd, C is not yet. pkg_add installs the update to B, then > encounters the missing C update, and bails the upgrade of A. Now you have A > installed which doesn't work with the new version of B. Is this something > that can currently happen?
Nope. That's why we have shared libraries numbers. We keep the old libraries around, so packages will keep working. In the rare case where some packages have tight dependencies (see cups and cups-libs for instance), pkg_add will update both at the same time. An update is done in two steps: first extract the new package(s) to temporary files, THEN delete the old package and move the temporary files in final location. The only way you end up with broken installations is when porters don't do their jobs, that is they fail to bump a shared library or something like that. > Even with snapshot shearing though, having this index file could provide a > substantial speed upgrade. Instead of having to check *all* installed > package's header for updates, you could use the index to know the subset of > packages that you expect to have actually changed, and only download *those* > packages' headers. If the expected "combined" sha of a given package doesn't > match the index's version, then the mirror is clearly out of sync and we > could abort an update as usual. The problem is the multiple RTT...! if you manage to keep one single connection open, you get a substantial speed upgrade. Generating a single index is more problematic than it would seem at first o glance. When everything goes fine, you're happy. If anything is out-of-synch, you're in deep shit. This would mean having several possible modes to cope with that, we don't have enough resources to do that.
