Re: [systemd-devel] fully volatile running from ram
On Thu, 27.04.17 12:52, jr (darwinsker...@gmail.com) wrote: > On Wed, Apr 26, 2017 at 04:08:21PM +0200, Lennart Poettering wrote: > > On Tue, 25.04.17 13:13, jr (darwinsker...@gmail.com) wrote: > > > > > hello, > > > > > > in a fully-volatile boot scenario /usr from a physical disk gets mounted > > > on > > > top of an instance of a tmpfs. my first question is why is that necessary? > > > (the tmpfs part i mean) > > > > I am not sure I grok what you are trying to say? tmpfs is how on Linux > > you can easily have a volatile file system, as it lives entirely in > > memory and never hits the disk (admittedly modulo swapping). > > > > but once you mount over that tmpfs from disk the overlaying fs will hide > the underlying tmpfs, no? baring the fs-caching done in kernel > (readahead?), for anything that needs to be loaded into memory for > executing or otherwise disk has to be touched for, well reading, at least > once, no? this is the part i'm trying to understand, if the overlaying fs > is mounted from physical disk, does mounting over tmpfs causes kernel to > cache that fs entirely into memory? > > this looks a lot like initramfs where it is an instance of tmpfs but that > also gets *shadowed* once "real_root" is switched to, no? I still don't grok what you are trying to do, but do note that open files (regardless whether referenced via an open fd, or via an mmap() mapping, for example because of a ELF binary being mapped into memory) will always continue to reference the original files on the original file systems they were opened from regardless if the file system is now obstructed and the paths originally used to open them would refer to a different file or file system now. > > > my second question is, would it be possible to do the same but rather > > > than mounting the /usr *populate* the said tmpfs with OS tree from said > > > physical disk, preferably in a blocked or fs cached setup (db-cache or > > > bcachefs). i realise that this can be done easily in initrd or even > > > initramfs can hold the /usr but the problem there is when we boot > > > "developmen" and not "production" in which case we want updates to be > > > written to disk. > > > > I am not grokking this question either, but keeping the whole OS in > > memory of course means you need a lot of memory. Moreover depending on > > disk speeds it means your boot will be quite slow, as you copy stuff over > > every single time before you can boot. If you copy things around like > > that during boot the best thing would be to do it in the initrd, as > > otherwise you are running the system already, that you are about to > > prepare for running, and dropping open references the the old files is > > hard (though not entirely impossible, after all there is "systemctl > > daemon-reexec" and things). > > no, no, i'm thinking systemd as rdinit rather than init; i.e. initramfs is > the real_root. one way of doing it is to pack the /usr into a initramfs > archive and either build it into kernel or pass it via bootload (never > worked for me)? then you boot the system into ram, enjoy blazing fast > responsiveness of it but there comes along some update that one would like > to apply and it turns out be really great that update. but how do you make > it stick then? in fact if /var is not volatile and your package manager > keeps it's records there (in my case portage does) on the next boot system > is confused because it thinks that updates are going to be there. this has > a number of solutions; bcachefs or dm-cache or even --overlay. > --overlay is cool since it stages the upgrade; caching solutions are for > performance though. i just don't know if the hooks are there (kernel > and/or systmd) too boot the system this way? i.e. populate the initramfs > from current or next or the one-after gold-master which resides on disk; > start working on initramfs; associate this initramfs with original or > another block-device or subvol of an fs on the disk and let our chosen > caching system take care of mirroring our working tree with said *backup*. > next reboot we should have the option of roll-back or continuing with our > work and so on. > > please, please let me know if i'm still making no-sense :) English is not > my strong suit and on top of that i'm horrible at explaining something. Still not grokking what you are trying to do, but do note that an initrd is ultimately just a tmpfs preinitialized from a cpio archive. Also note that if you have a directory tree on disk there's little reason to copy it into tmpfs performance-wise. The kernel buffer cache will cache the data on disk into RAM anyway as it is being accessed. Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] fully volatile running from ram
On Wed, Apr 26, 2017 at 04:08:21PM +0200, Lennart Poettering wrote: > On Tue, 25.04.17 13:13, jr (darwinsker...@gmail.com) wrote: > > > hello, > > > > in a fully-volatile boot scenario /usr from a physical disk gets mounted on > > top of an instance of a tmpfs. my first question is why is that necessary? > > (the tmpfs part i mean) > > I am not sure I grok what you are trying to say? tmpfs is how on Linux > you can easily have a volatile file system, as it lives entirely in > memory and never hits the disk (admittedly modulo swapping). > but once you mount over that tmpfs from disk the overlaying fs will hide the underlying tmpfs, no? baring the fs-caching done in kernel (readahead?), for anything that needs to be loaded into memory for executing or otherwise disk has to be touched for, well reading, at least once, no? this is the part i'm trying to understand, if the overlaying fs is mounted from physical disk, does mounting over tmpfs causes kernel to cache that fs entirely into memory? this looks a lot like initramfs where it is an instance of tmpfs but that also gets *shadowed* once "real_root" is switched to, no? > > my second question is, would it be possible to do the same but rather > > than mounting the /usr *populate* the said tmpfs with OS tree from said > > physical disk, preferably in a blocked or fs cached setup (db-cache or > > bcachefs). i realise that this can be done easily in initrd or even > > initramfs can hold the /usr but the problem there is when we boot > > "developmen" and not "production" in which case we want updates to be > > written to disk. > > I am not grokking this question either, but keeping the whole OS in > memory of course means you need a lot of memory. Moreover depending on > disk speeds it means your boot will be quite slow, as you copy stuff over > every single time before you can boot. If you copy things around like > that during boot the best thing would be to do it in the initrd, as > otherwise you are running the system already, that you are about to > prepare for running, and dropping open references the the old files is > hard (though not entirely impossible, after all there is "systemctl > daemon-reexec" and things). no, no, i'm thinking systemd as rdinit rather than init; i.e. initramfs is the real_root. one way of doing it is to pack the /usr into a initramfs archive and either build it into kernel or pass it via bootload (never worked for me)? then you boot the system into ram, enjoy blazing fast responsiveness of it but there comes along some update that one would like to apply and it turns out be really great that update. but how do you make it stick then? in fact if /var is not volatile and your package manager keeps it's records there (in my case portage does) on the next boot system is confused because it thinks that updates are going to be there. this has a number of solutions; bcachefs or dm-cache or even --overlay. --overlay is cool since it stages the upgrade; caching solutions are for performance though. i just don't know if the hooks are there (kernel and/or systmd) too boot the system this way? i.e. populate the initramfs from current or next or the one-after gold-master which resides on disk; start working on initramfs; associate this initramfs with original or another block-device or subvol of an fs on the disk and let our chosen caching system take care of mirroring our working tree with said *backup*. next reboot we should have the option of roll-back or continuing with our work and so on. please, please let me know if i'm still making no-sense :) English is not my strong suit and on top of that i'm horrible at explaining something. jrun ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] fully volatile running from ram
On Tue, 25.04.17 13:13, jr (darwinsker...@gmail.com) wrote: > hello, > > in a fully-volatile boot scenario /usr from a physical disk gets mounted on > top of an instance of a tmpfs. my first question is why is that necessary? > (the tmpfs part i mean) I am not sure I grok what you are trying to say? tmpfs is how on Linux you can easily have a volatile file system, as it lives entirely in memory and never hits the disk (admittedly modulo swapping). > my second question is, would it be possible to do the same but rather than > mounting the /usr *populate* the said tmpfs with OS tree from said physical > disk, preferably in a blocked or fs cached setup (db-cache or bcachefs). i > realise that this can be done easily in initrd or even initramfs can hold > the /usr but the problem there is when we boot "developmen" and not > "production" in which case we want updates to be written to disk. I am not grokking this question either, but keeping the whole OS in memory of course means you need a lot of memory. Moreover depending on disk speeds it means your boot will be quite slow, as you copy stuff over every single time before you can boot. If you copy things around like that during boot the best thing would be to do it in the initrd, as otherwise you are running the system already, that you are about to prepare for running, and dropping open references the the old files is hard (though not entirely impossible, after all there is "systemctl daemon-reexec" and things). Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] fully volatile running from ram
hello, in a fully-volatile boot scenario /usr from a physical disk gets mounted on top of an instance of a tmpfs. my first question is why is that necessary? (the tmpfs part i mean) my second question is, would it be possible to do the same but rather than mounting the /usr *populate* the said tmpfs with OS tree from said physical disk, preferably in a blocked or fs cached setup (db-cache or bcachefs). i realise that this can be done easily in initrd or even initramfs can hold the /usr but the problem there is when we boot "developmen" and not "production" in which case we want updates to be written to disk. jrun ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel