Hi! On Tue, 2015-11-10 at 00:01:48 +0100, Helmut Grohne wrote: > Package: dpkg > Version: 1.18.3 > Severity: wishlist > Tags: patch > User: [email protected] > Usertags: rebootstrap
> Thank you very much for discussing the idea of DPKG_ROOT and recording > some results at https://wiki.debian.org/Teams/Dpkg/Spec/InstallBootstrap > already. By now, I am convinced that this idea is worth exploring and > have thus prepared a small patch set implementing some of the first > steps. Some of the stuff I'll cover below I think I probably covered in our conversation on IRC at the time, so it might not come as new stuff. > a) dpkg should export DPKG_ROOT. DPKG_ROOT should be a string that > should be prepended to "/" to arrive at the current installation > root (instdir). Notably, when dpkg invokes chroot(), DPKG_ROOT > becomes empty. At the moment, DPKG_ROOT is always empty, but this > should not be relied upon. This is > 0001-export-a-variable-DPKG_ROOT.patch. This part is fine in principle, and that's what I envisioned when thinking about how to merge the dpkg functional test suite into the dpkg git repo proper. But there are several reasons I've always hesitated to add this support even though it would be useful for dpkg itself. > b) Packages that do not "set -u" (nounset), can now prepend $DPKG_ROOT > to any file they operate on. With old versions $DPKG_ROOT will be > unset and with change a) $DPKG_ROOT will be empty. Thus this change > is backwards-compatible. Right, and these can always be set like ": ${DPKG_ROOT:=}". We could even recommend this as part of some doc/howto/spec for the initial deployments, before packages can assume a recent enough dpkg. > c) dpkg should gain a new force flag. I call it --force-remote-configure > for now. It is supposed to force dpkg into running maintainer scripts > without chroot even when the package in question did not declare that > its maintainer scripts support this mode of operation. Note that we > currently have no way to express whether a package supports running > maintainer scripts without chroot. The flag is being added by > 0002-add-force-remote-scripts.patch and the behavior is implemented > by 0003-inhibit-chroot-when-force-remote-scripts.patch. Packages can > only reasonably support this mode after implementing b). I don't quite like the name, as remote to me implies on some other machine. Perhaps foreign, host or extern(al), although some of these are a bit overloaded terms already. Also I'm not sure this makes sense as a force option (instead of its own proper option), as it is a behavior change that we want to always be safe to use, in contrast to a force option that just forces the behavior. Having a force option that would not chroot some times is a bit strange. We might still want to have both though, and there was a related bug report for that (#614126). > d) Once a) is accepted and b) starts getting implemented, we need to > think about a way for packages to tell that they support "remote > scripts". One way to do so would be to add a header "Remote-Scripts: > yes" to the binary package stanza. Packages thus marked would be > required to honour DPKG_ROOT in all maintainer scripts. This flag > makes no provisions yet on what programs can be assumed to be > installed outside the chroot that is operated on. Yes, ideally only packages marked as such would get a chroot-less environmemt when requested with the new option, because expecting the user to know when a package supports this mode and on what specific version is a terrible interface IMO, as it requires the user to analyze the .debs before unpacking them. One tiny concern is that if eventually we end up with the whole archive (haha? :) supporting this, then the field seems a bit of bloat. The other thing, that ISTR you brought up on IRC are triggers. I'm not sure how we'd handle those either. :/ > e) Once a) is accepted and b) starts getting implemented, we need to > think about what programs maintainer scripts can assume to be > available outside the chroot. Some ways to handle that: > * Packages may only assume "common unix functionality". Such a set > would have to be defined somehow and roughly equates what > debootstrap requires. Defining this in terms of strict POSIX compliance (or a subset of the utilities defined within) would be the easiest/best I think. > * Packages may only assume essential packages to be available. That would resitrict the external system to be a Debian system. Which might be fine I guess but is more limited than what debootstrap requires. It has the problem that even then you need to specify essential of what Debian version? > * A new set of headers Maint-{Depends,Conflicts,...} is added to > request tools to be installed. These new relations would be > checked outside the chroot (if any). > > A full Debian release or two needs to pass before such headers can > be used in the archive. This also poses the problem that a user > can remove packages required for removing other packages and thus > revoking the ability to remove certain packages. It is not clear > how the absence of Maint-Depends is supposed to be handled. It is > not clear whether dpkg needs to lock the dpkg database outside the > chroot. As I mentioned at the time I don't think this is a workable solution. These are the problems I see right away with it: * dpkg would need to load two status databases in memory, which would seriously complicate the code. * dpkg would need to lock the external database, which means multiple chroots could not be used concurrently. * As you mention, the dependencies might disappear under dpkg's feet, while it is not running in this mode. * It ties the dependency graph from one installation into another, which can seriously complicate upgrades and similar. > f) Once a), b) and d) are implemented and some version of e) is agreed > upon, debootstrap can be changed to prefer configuring packages that > support "remote scripts" to break dependency cycles. Yes. > g) At the same time as f), tools like multistrap can start making use of > this new functionality. Yes. > With the above patches, I verified that I can install a package with a > maintainer script into a chroot that has an empty database (in > particular no essential package is unpacked in the chroot). The > maintainer script is run without chroot and has DPKG_ROOT set up > properly. Thanks for the patches! I think 2 and 3 need to be merged together, but see my comments on the name and if this deserves to be a force option or something else. And they need man page updates. > bash's preinst will become a problem with this scheme. It is a binary > due to earlier breakage when it was a script, but the "remote scripts" > approach cannot work with binaries, because we cannot know whether the > cpu supports executing such binaries. I don't have a plan for > bash.preinst. I don't think the cpu support is a problem because I'd not expect actual remote execution to take place? But the linked libraries certainly are. We might perhaps need to read the maint scripts and only allow this mode if they are actual interpreted scripts (i.e. they have a shebang of any type). But that can be a safe guard added later. > I understand that what I propose herein is a steep change with wide > ranging implications. It needs sincere thought to avoid creating a > situation that is hard to fix up. Yet it seems pretty round to me > already. Despite there being open questions, we can handle at least a) > today. Yeah doing a) by itself now seems pretty innocuous to me as well, so I'd be fine with including that, but the rest seems still pretty undefined. Thanks, Guillem
