This is brilliant! I'm gonna have to add it as a feature of NORD Linux, which up to now only has the private /etc hack (with a shared RO root).
Mack, I think we've got more concerns than just sym-link follow overhead. How many people bother to set an efficient PATH command search? Coincidentally, I tried sym-links into "the system" more than five years ago. Worked fine. (Was not shared.) Binding over empty directories in the RW root obviously also works well and can be less confusing. The only downside I can see with bind mounts is that they are terribly hard to un-do ... should you ever need to. (For example, try unmounting /lib.) As of now, I have the sym-linkery working. Will try the bindage (bondage?) in a few days. Even with the private /etc hack, I have become convinced that intercepting INIT is the best way to do it. After several times through the "under control of INIT" route, I found that changing the default run-level may be impossible if /sbin/init has already read /etc/inittab (from a space that will get replaced ... or may not exist yet). So I have a pre-init for private /etc and now have a pre-init for sym-links. They seem to work well. > As far as the rest of the > init process is concerned, the effect is as if you had booted from the > writable root. Exactly! -- R; <>< On Thu, Aug 12, 2010 at 09:20, Edmund R. MacKenty <[email protected]> wrote: > On Wednesday 11 August 2010 23:12, Richard Troth wrote: >>This is an awesome idea. >>Two ways to do it: bind mount RO over existing /bin and friends, or >>let /bin and friends be sym-links into "the system", wherever it gets >>mounted. > > I'd recommend bind-mounts, because they avoid the overhead of symlinks. Even > though the inodes for those symlinks will be cached, each access to any file > in a shared system directory will have to fetch and read that inode in order > to resolve the pathname. With bind-mounts, it's all done in the mount table > which is already in kernel-space (I think?), so it's faster. > >>Need to be aware of hiding files under the RO mounts. If customers >>are PAYING for RW space, and you have content there for bootstrapping, >>but that stuff gets overlaid ... it's a drag. It is possible to boot >>an 'init' which fixes things and then does a 'pivot_root' to get the >>RW root they want. > > That's exactly what we ended up doing in the Provisioning Expert, if the boot > filesystem is shared and root is not: the kernel runs an init script that > mounts the necessary filesystems and bind-mounts the system directories > (/bin, /lib, ...) from a shared filesystem onto an instance-specific writable > root filesystem. Then it does the pivot_root to make that writable > filesystem the real root and execs the real /sbin/init to start things going. > It's sort of like having a post-initrd script. As far as the rest of the > init process is concerned, the effect is as if you had booted from the > writable root. > > I wouldn't recommend this for the faint-of-heart if you want a general-purpose > mechanism, because there's all sorts of complexities involved with LVM > filesystems, DASD activation and ordering things so you have *someplace* you > can write to when necessary. But it is very nice to have all the Linux stuff > shared and each Linux instance you create owns just its root filesystem and > whatever application-specific filesystems it might need. > > BTW: you don't have to hide any files on the R/W filesystem under R/O mounts > with this approach. You will hide some R/O files under the R/W filesystem, > but the customers won't be paying for that. This is because you're booting > with only the shared, R/O filesystems available, then adding the customer's > R/W filesystems to them. So the R/W filesystems can just have empty > directories for the mount-points: all the files you need to boot are already > on the shared filesystems. It's kind of like booting a LiveCD, but instead > of just adding tmpfs's where necessary you've got to get a hold of specific > writable devices and arrange them into the correct directory structure. > > Forgive me for going on and on about this, but this pivot_root approach is > near and dear to me because implementing it solved a lot of problems for us. > - MacK. > ----- > Edmund R. MacKenty > Software Architect > Rocket Software > 275 Grove Street · Newton, MA 02466-2272 · USA > Tel: +1.617.614.4321 > Email: [email protected] > Web: www.rocketsoftware.com > > ---------------------------------------------------------------------- > For LINUX-390 subscribe / signoff / archive access instructions, > send email to [email protected] with the message: INFO LINUX-390 or visit > http://www.marist.edu/htbin/wlvindex?LINUX-390 > ---------------------------------------------------------------------- > For more information on Linux on System z, visit > http://wiki.linuxvm.org/ > ---------------------------------------------------------------------- For LINUX-390 subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO LINUX-390 or visit http://www.marist.edu/htbin/wlvindex?LINUX-390 ---------------------------------------------------------------------- For more information on Linux on System z, visit http://wiki.linuxvm.org/
