Hi, I've started working on todo/persistence [1] in live-boot upstream, so I think it's time for us to settle on exactly what we want. My intention is to update the RFC for persistence related changes in live-boot [2] with any conclusions reached in this thread, and then discuss them on the Debian live mailing list, and then (if accepted) implement them.
[1] https://tails.boum.org/todo/persistence/ [2] http://live.debian.net/devel/rfc/persistence/ Requirements ============ From the roadmap and various other places in our todo item [1] our high-level requirements seem to be these: * Persistent application-specific configurations. * Persistent user data store. * Ability to mount persistent media in read-only mode. * Protection against too much write-wearing for flash media. Current state of live-boot ========================== live-boot enables persistence in four ways: * live-rw: an overlay that is mounted on /. * live-sn: a snapshot that is copied over / on boot, and is updated with changes on / on shutdown. * home-{rw,sn}: like the above two, respectively, but only for /home. These options are too crude for us -- we need something that is more flexible, which allows to make only specific directories persistent. Only a certain type of snapshot has support for that currently, namely the cpio.gz type (i.e. a cpio archive that is gzipped). Since I've implemented read-only mode for snapshots [3], it seems we're almost there! [3] http://git.immerda.ch/?p=tails_live-boot.git;a=commit;h=c69497ad905cb09df4ced7e418fbef45e579bd3e The problems with snapshots =========================== But cpio.gz snapshots has some issues: 1. It is very unfriendly to flash based storage if we only do minor changes to our persistent data, since *all* persistent data are written back to the physical storage at *every* shutdown. I'm afraid minor changes is a more typical usage where it matters. Imagine having 100 MB of emails, fetching maybe 50 KB worth of new mails from your inbox, and then syncing *all* the 100 MB worth of old, unmodified emails back as well. That causes pretty significant write- wearing in comparison to how much data that was added to the snapshot. 2. On boot all snapshots' files are synced into the tmpfs, so they're stored in preicious RAM. Hence snapshots cannot be very large (specifically, the maximum is ~ ${RAM_SIZE}/2, and that leaves no space for other file system modifications). 3. Syncing on shutdown may take a long time since all persistent data is synced at the same time, not continuously over the whole session. 4. Crash or emergency shutdown => cartain data loss. 5. Doesn't currently support file deletion. 2 and 4 are inherent issues with snapshots, and cannot be avoided. The rest are only implementation issues: 1 may be fixed if we ditch the compression, and use "tar -u" style appending of changes, which also may improve 3 somewhat (don't know). 5 can be fixed in various ways. The case for overlays ===================== To me it seems like overlays are inherently nicer than snapshots since none of the above affects them. Sure, write wearing is also a problem for them, but I think we get the best of all worlds if we use overlays with flash friendly filesystems like yaffs/logfs/ubifs or whatever. Support for that needs to be added then, of course. At the very least the "persistent user data store" cannot use snapshots due to point 2, so an overlay is the only option and thus must support our requirements (e.g. "read-only" and "arbitrary directories", both which currently are note implemented for overlays). The overlay's only limitation is that it has static size and cannot automatically grow like a snapshot file can (snapshot partitions can't for the same reason). Trying to anticipate how much space is required for each "persistent application-specific configuration" is not a path I'd like to thread, either with us trying to do the estimate or our users. They need to share the space of an underlying overlay. Proposed solution: locally specified inclusions =============================================== We make home-{rw,sn} obsolete, only live-{rw,sn} are considered by live-boot (or more correctly, the scripts it adds to the initramfs). When a persistent media (with label/filename "live-rw") is found by live-boot, it looks for a file called .live-persistence.includes (but I'll continue calling it just ".includes") in its root. If it's not there, then it mounts the media (using aufs) on / just like it does for live-rw currently. But if .includes is present, then it doesn't mount anything on /, it instead bind-mounts the directories listed in .includes to their specified destinations. In-depth example ---------------- live-boot scans $dev and mounts it on some $mnt (current behaviour). It finds $mnt/.includes, which consists of: /home/amnesia/.gnupg /var/lib/tor This translate into live-boot doing: mount -o bind $mnt/home/amnesia/.gnupg $root/home/amnesia/.gnupg mount -o bind $mnt/var/lib/tor $root/var/lib/tor where $root is what will become the filesystem root after initramfs. Snapshots --------- .includes could also be used for all types of snapshots, makeing the rather awkward (and only available for cpio.gz type snapshots) /etc/live-snapshot.list obsolete. I say it's awkward because it's *inside* the live system, so if you want to change it you need to build a new image. Note: for snapshots the .includes file wouldn't be limited to directories but could also handle individual files. Overlays cannot handle individual files as long as we rely on bind-mounting. Backwards-compatibility ----------------------- *If* we care for backwards-compatibility, we'd have to allow for an extended syntax which allows specifying source-desination pairs. To get the old home-{rw,sn} type persistence, .live-persistence.includes should then look simply like this: . /home which translates to: mount -o bind $mnt/. $root/home If live-boot finds a home-{sn,rw} labeled/named media, it could create the above file and everything would work exactly like before. Beyond backwards-compatibility, some people may find this syntax more convenient. Not sure this is worth the effort for implementing it, though. There are other UI changes in live-boot's persistence handling that will break stuff, so people will have to be prepared for fixing their old setups any way. Conclusion ========== I think this will be pretty easy to implement, consistent between snapshots and overlays, and quite powerful in all its simplicity. Thoughts? Cheers!
signature.asc
Description: OpenPGP digital signature
_______________________________________________ tails-dev mailing list [email protected] https://mailman.boum.org/listinfo/tails-dev
