Re: Matters approaching
Hello all, sorry for the really late answer to this. This is probably better matter for an interactive discussion than email, but let's write it down anyway. Mark Shuttleworth [2014-03-18 5:59 -0700]: * configuration in /etc/ and ways that can feel comfortable but remain secure This is indeed one of the things where the phone, thanks due to its (still) relatively limited focus, could get away with the current approach of keeping most of /etc/ readonly. But this is not a solution for a converged desktop: - /etc/ is *meant* to be writable, as that's the place to put configuration in. Our own list of exceptions shows that this is already hard enough for our relatively small and confined phone stack, but it will be impractical on a desktop and absolutely not fly in the server world. So I'm fairly convinced that in the converged world we need to provide a writable /etc/. - The bind mount files hack totally breaks file system semantics, and thus system services which needs to update them, backup/restore systems, revision control, etc. and introduces race conditions. It also requires us to carry nasty hacks and hard to maintain workarounds in e. g. timedated, and probably breaks a lot of other packages which we just haven't discovered yet with the phone-only system. As Stéphane already investigated and pointed out, the currently existing file-level overlay file systems don't cut it. They still are too buggy/limited (we run into test failures with them all the time, and e. g. overlayfs doesn't provide inotify), and equally importantly, they are not available on the Android kernels. Block-based overlays don't help either, so I think we need to come up with a pure userspace solution here. For example, whenever we do a system upgrade we could check which files in /etc/ have been modified/added/removed (look at mtime, hashes, or keep the pristine /etc somewhere else and diff with that), store that delta, do the upgrade, and reapply it. That would essentially be overlay fs without kernel support. Sure, it might take a few seconds to do that computation and file system copying, but in the context of a system upgrade that should hardly matter. But there is no runtime overhead at all with this. * the meaning of /usr/local/ in this sort of environments This seems one of the easy parts to me. As the distro and /local bits have always been rather stricly separated, this should just be a writable path which we never touch. Do you see anything difficult here? * the use of LVM and other filesystem capabilities to enable user customisation In terms of general upgrade safety in the .deb world, the LVM/snapshot thing might indeed be useful; with that we could offer to undo the upgrade if it fails or the user doesn't like the result. But that's a separate (and equally complicated) matter, so we should keep that as a separate discussion. I don't think that block based snapshot/revert/overlay solutions are any help here. They might of course be nice for different problems, like dynamically extending your device's storage space with hotplugged drives, but they don't do anything to solve the install system images and deb packages side by side problem. For that I think the only viable direction is to split the dpkg database, i. e. a logical r/o underlay of /var/lib/dpkg/. The readonly part (e. g. /lib/system-image/dpkg/) would be shipped in the system image and defines the packages which you can never uninstall. dpkg needs to be modified to work on /var/lib/dpkg/ as usual, but also take the readonly part into account in terms of the status, info/*.list, etc. files. With that you have a single consistent view of packages including dependency management, with the only restriction that you can't remove/upgrade packages which are in the readonly part. With that the upgrade of the system image could in theory break/conflict with additionally installed packages. So after that we probably should run apt-get -f install etc. to clean up, or maybe upgrade/change deb packages according to the base image changes. As we already know which system image packages will be there even before we start the upgrade, we could even do this computation before, and warn/abort if it's not going to work. That shouldn't happen too often though, as we should try to not include packages which are prone to cause conflicts into the sytem image. With that combination we of course don't get the rock-solid and atomic upgrades that we have one the phone as soon as you start using deb packages. But it's not any worse than on current desktop/servers where everything is .deb, and I think that's all that we can possibly achieve here. It's probably better as all the crucial bits for booting *are* in the system image, so breaking your machine to the point of not being bootable any more should become a lot less likely than it is today. Implementing that is of course no small task, but I don't see a solution which is
Re: Matters approaching
On Fri, May 23, 2014 at 5:46 AM, Martin Pitt martin.p...@ubuntu.com wrote: Hello all, sorry for the really late answer to this. This is probably better matter for an interactive discussion than email, but let's write it down anyway. Mark Shuttleworth [2014-03-18 5:59 -0700]: * configuration in /etc/ and ways that can feel comfortable but remain secure This is indeed one of the things where the phone, thanks due to its (still) relatively limited focus, could get away with the current approach of keeping most of /etc/ readonly. But this is not a solution for a converged desktop: - /etc/ is *meant* to be writable, as that's the place to put configuration in. Our own list of exceptions shows that this is already hard enough for our relatively small and confined phone stack, but it will be impractical on a desktop and absolutely not fly in the server world. So I'm fairly convinced that in the converged world we need to provide a writable /etc/. - The bind mount files hack totally breaks file system semantics, and thus system services which needs to update them, backup/restore systems, revision control, etc. and introduces race conditions. It also requires us to carry nasty hacks and hard to maintain workarounds in e. g. timedated, and probably breaks a lot of other packages which we just haven't discovered yet with the phone-only system. As Stéphane already investigated and pointed out, the currently existing file-level overlay file systems don't cut it. They still are too buggy/limited (we run into test failures with them all the time, and e. g. overlayfs doesn't provide inotify), and equally importantly, they are not available on the Android kernels. Block-based overlays don't help either, so I think we need to come up with a pure userspace solution here. For example, whenever we do a system upgrade we could check which files in /etc/ have been modified/added/removed (look at mtime, hashes, or keep the pristine /etc somewhere else and diff with that), store that delta, do the upgrade, and reapply it. That would essentially be overlay fs without kernel support. Sure, it might take a few seconds to do that computation and file system copying, but in the context of a system upgrade that should hardly matter. But there is no runtime overhead at all with this. * the meaning of /usr/local/ in this sort of environments This seems one of the easy parts to me. As the distro and /local bits have always been rather stricly separated, this should just be a writable path which we never touch. Do you see anything difficult here? * the use of LVM and other filesystem capabilities to enable user customisation In terms of general upgrade safety in the .deb world, the LVM/snapshot thing might indeed be useful; with that we could offer to undo the upgrade if it fails or the user doesn't like the result. But that's a separate (and equally complicated) matter, so we should keep that as a separate discussion. I don't think that block based snapshot/revert/overlay solutions are any help here. They might of course be nice for different problems, like dynamically extending your device's storage space with hotplugged drives, but they don't do anything to solve the install system images and deb packages side by side problem. For that I think the only viable direction is to split the dpkg database, i. e. a logical r/o underlay of /var/lib/dpkg/. The readonly part (e. g. /lib/system-image/dpkg/) would be shipped in the system image and defines the packages which you can never uninstall. dpkg needs to be modified to work on /var/lib/dpkg/ as usual, but also take the readonly part into account in terms of the status, info/*.list, etc. files. With that you have a single consistent view of packages including dependency management, with the only restriction that you can't remove/upgrade packages which are in the readonly part. With that the upgrade of the system image could in theory break/conflict with additionally installed packages. So after that we probably should run apt-get -f install etc. to clean up, or maybe upgrade/change deb packages according to the base image changes. As we already know which system image packages will be there even before we start the upgrade, we could even do this computation before, and warn/abort if it's not going to work. That shouldn't happen too often though, as we should try to not include packages which are prone to cause conflicts into the sytem image. With that combination we of course don't get the rock-solid and atomic upgrades that we have one the phone as soon as you start using deb packages. But it's not any worse than on current desktop/servers where everything is .deb, and I think that's all that we can possibly achieve here. It's probably better as all the crucial bits for booting *are* in the system image, so breaking your machine to the point of not
