Re: [systemd-devel] swap on zram service unit, using Conflicts=umount
On Tue, Jun 25, 2019 at 3:30 AM Zbigniew Jędrzejewski-Szmek wrote: > > On Tue, Jun 25, 2019 at 10:55:27AM +0200, Lennart Poettering wrote: > > On Mo, 24.06.19 13:16, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) > > wrote: > > > > > > So for tmpfs mounts that don't turn off DefaultDependencies= we > > > > implicit add in an After=swap.target ordering dep. The thinking was > > > > that there's no point in swapping in all data of a tmpfs because we > > > > want to detach the swap device when we are going to flush it all out > > > > right after anyway. This made quite a difference to some folks. > > > > > > But we add Conflicts=umount.target, Before=umount.target, so we do > > > swapoff on all swap devices, which means that swap in the data after all. > > > Maybe that's an error, and we should remove this, at least for > > > normal swap partitions (not files)? > > > > We never know what kind of weird storage swap might be on, I'd > > probably leave that in, as it's really hard to figure out correctly > > when leaving swap on would be safe and when not. > > > > Or to say this differently: if people want to micro-optimize that, > > they by all means should, but in that case they should probably drop > > in their manually crafted .swap unit with DefaultDependencies=no and > > all the ordering in place they need, and nothing else. i.e. I believe > > this kind of optimization is nothing we need to cater for in the > > generic case when swap is configured with /etc/fstab or through GPT > > enumeration. > > Not swapping off would make a nice optimization. Maybe we should > invert this, and "drive" this from the other side: if we get a stop > job for the storage device, then do the swapoff. Then if there are > devices which don't need to stop, we wouldn't swapoff. This would cover > the common case of swap on partition. > > I haven't really thought about the details, but in principle this > should already work, if all the dependencies are declared correctly. I like the sound of this. The gotcha with current swap on zram units (there are a few floating out there including this one), is they conflate two different things: setup and teardown of the zram device, and swapon/swapoff. What's probably better and more maintainable would be a way to setup the zram device with a service unit, and then specify it in /etc/fstab as a swap device so that the usual systemd swapon/off behavior is used. Any opinion here? (Somewhat off topic: I wish zswap was not still experimental: by enabling zswap with an ordinary swap device, it creates a memory pool (which you can define a percentage of total RAM to use) that's compressed for swap before it hits the backing device. Basically, it's like a RAM cache for swap. It'll swap to memory first, and then overflow to a swap partition or file. It also lacks all the weird interfaces of zram.) https://wiki.archlinux.org/index.php/Improving_performance#Zram_or_zswap > > zswap is different: we know exactly that the swap data is located in > > RAM, not on complex storage, hence it's entirely safe to not > > disassemble it at all, iiuc. > > Agreed. It seems that any Conflicts= (including the one I proposed) are > unnecessary/harmful. OK I'll negate the commit that inserts it. -- Chris Murphy ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Changes to dependency graph during boot
Thank you for that thorough explanation, much appreciated! Cheers, Conrad On 6/25/19 11:14 AM, Lennart Poettering wrote: > On Mo, 24.06.19 16:41, Conrad Hoffmann (c...@bitfehler.net) wrote: > >> Hi, >> >> TL;DR: I was wondering what happens if a unit executed early during the >> boot process changes the current dependency graph by either enabling or >> even starting another unit that was previously disabled. Is this defined >> behaviour, and if so what are the rules? > > So, this is usually safe, but there are some issues with this, and I > would not recommend reloading the dep tree during a regular, clean > boot. It's not designed for that. > > In some cases reloading the unit files during operation is not > safe. Specifically, consider a unit file with contains something like > that: > > ``` > [Sevice] > Type=oneshot > ExecStart=/usr/bin/something with-a-param > ExecStart=/usr/bin/something-else also-a-parm > ExecStart=/usr/bin/something-third third-param > ``` > > Now your service is already at the execution of the middle program, > and now you reload the whole thing and make a modification of the unit > file: > > ``` > [Sevice] > Type=oneshot > ExecStart=/usr/bin/something with-a-param > ExecStart=/usr/bin/something-else changed-param > ExecStart=/usr/bin/something-third third-param > ``` > > Now the question is where execution should continue: the middle line > was changed, and we can easily see that as humans, but it's not clear > from systemd's pov. It could also be that the old line was removed and > a new line inserted instead. Depending how you see that after the > reload it could mean to easier continue execution with the second line > or with the third line, and it's really not clear semantically what is > meant here. > > systemd employs some heuristics and tries to do some reasonable smart > choice, but it's black magic and it might be wrong. But this means: > reloading the whole daemon during clean code paths is > questionnable. it's fine doing that when you install or remove > packages, i.e. not in common codepaths but only in an upgrade > codepaths, but doing that on every single boot, as part of a clean > start-up is something i would advocate like that. > > Hence: changing units and issuing daemon reloads during boot-up is > ugly. Don't do it. Besides these issues pointed out above, it's also > slow. > > Noe that if you have all units in place already and just want to > enqueue them, that's entirely safe, the common case and > recommended. The suggestions above are just about of modifying units, > creating them, removing them and then reloading systemd. Don't do that. > > In systemd, we have the "generator" concept, that is our recommended > way to extend the unit dependency tree with foreign and automatically > generated units. Generators run very early during boot, before the > initial transaction is generated, which means everything they do is > already complete by the time systemd calculates the initial > transaction, and thus while that transaction and all its jobs are > executed unit files are not changed anymore. Generally I'd strongly > recommend using generators over manually modifying/creating units > during boot combined with a daemon reload. > > For more information about generators see: > > https://www.freedesktop.org/software/systemd/man/systemd.generator.html > > Lennart > > -- > Lennart Poettering, Berlin > ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] swap on zram service unit, using Conflicts=umount
On Tue, Jun 25, 2019 at 10:55:27AM +0200, Lennart Poettering wrote: > On Mo, 24.06.19 13:16, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote: > > > > So for tmpfs mounts that don't turn off DefaultDependencies= we > > > implicit add in an After=swap.target ordering dep. The thinking was > > > that there's no point in swapping in all data of a tmpfs because we > > > want to detach the swap device when we are going to flush it all out > > > right after anyway. This made quite a difference to some folks. > > > > But we add Conflicts=umount.target, Before=umount.target, so we do > > swapoff on all swap devices, which means that swap in the data after all. > > Maybe that's an error, and we should remove this, at least for > > normal swap partitions (not files)? > > We never know what kind of weird storage swap might be on, I'd > probably leave that in, as it's really hard to figure out correctly > when leaving swap on would be safe and when not. > > Or to say this differently: if people want to micro-optimize that, > they by all means should, but in that case they should probably drop > in their manually crafted .swap unit with DefaultDependencies=no and > all the ordering in place they need, and nothing else. i.e. I believe > this kind of optimization is nothing we need to cater for in the > generic case when swap is configured with /etc/fstab or through GPT > enumeration. Not swapping off would make a nice optimization. Maybe we should invert this, and "drive" this from the other side: if we get a stop job for the storage device, then do the swapoff. Then if there are devices which don't need to stop, we wouldn't swapoff. This would cover the common case of swap on partition. I haven't really thought about the details, but in principle this should already work, if all the dependencies are declared correctly. > zswap is different: we know exactly that the swap data is located in > RAM, not on complex storage, hence it's entirely safe to not > disassemble it at all, iiuc. Agreed. It seems that any Conflicts= (including the one I proposed) are unnecessary/harmful. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Changes to dependency graph during boot
On Mo, 24.06.19 16:41, Conrad Hoffmann (c...@bitfehler.net) wrote: > Hi, > > TL;DR: I was wondering what happens if a unit executed early during the > boot process changes the current dependency graph by either enabling or > even starting another unit that was previously disabled. Is this defined > behaviour, and if so what are the rules? So, this is usually safe, but there are some issues with this, and I would not recommend reloading the dep tree during a regular, clean boot. It's not designed for that. In some cases reloading the unit files during operation is not safe. Specifically, consider a unit file with contains something like that: ``` [Sevice] Type=oneshot ExecStart=/usr/bin/something with-a-param ExecStart=/usr/bin/something-else also-a-parm ExecStart=/usr/bin/something-third third-param ``` Now your service is already at the execution of the middle program, and now you reload the whole thing and make a modification of the unit file: ``` [Sevice] Type=oneshot ExecStart=/usr/bin/something with-a-param ExecStart=/usr/bin/something-else changed-param ExecStart=/usr/bin/something-third third-param ``` Now the question is where execution should continue: the middle line was changed, and we can easily see that as humans, but it's not clear from systemd's pov. It could also be that the old line was removed and a new line inserted instead. Depending how you see that after the reload it could mean to easier continue execution with the second line or with the third line, and it's really not clear semantically what is meant here. systemd employs some heuristics and tries to do some reasonable smart choice, but it's black magic and it might be wrong. But this means: reloading the whole daemon during clean code paths is questionnable. it's fine doing that when you install or remove packages, i.e. not in common codepaths but only in an upgrade codepaths, but doing that on every single boot, as part of a clean start-up is something i would advocate like that. Hence: changing units and issuing daemon reloads during boot-up is ugly. Don't do it. Besides these issues pointed out above, it's also slow. Noe that if you have all units in place already and just want to enqueue them, that's entirely safe, the common case and recommended. The suggestions above are just about of modifying units, creating them, removing them and then reloading systemd. Don't do that. In systemd, we have the "generator" concept, that is our recommended way to extend the unit dependency tree with foreign and automatically generated units. Generators run very early during boot, before the initial transaction is generated, which means everything they do is already complete by the time systemd calculates the initial transaction, and thus while that transaction and all its jobs are executed unit files are not changed anymore. Generally I'd strongly recommend using generators over manually modifying/creating units during boot combined with a daemon reload. For more information about generators see: https://www.freedesktop.org/software/systemd/man/systemd.generator.html Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] swap on zram service unit, using Conflicts=umount
On Mo, 24.06.19 21:01, Chris Murphy (li...@colorremedies.com) wrote: > On Mon, Jun 24, 2019 at 6:11 AM Lennart Poettering > wrote: > > That said, I don't really grok zram, and not sure why there's any need > > to detach it at all. I mean, if at shutdown we lose compressed RAM > > or lose uncompressed RAM shouldn't really matter. Hence from my > > perspective there's no need for Conflicts= at all, but maybe I am > > missing something? > > Huh yeah, possibly if anything there could be low memory systems just > barely getting by with swap on zram, and even swapoff at reboot time > would cause it to get stuck. It might just be better to clobber it at > reboot time? > > I'd like to allow a user to 'systemctl stop zram' which does swapoff > and removes the zram device. But is there something that could go into > the unit file that says "don't wait for swapoff, if everything else is > ready for shutdown, go ahead and reboot now?" Not sure I follow? I mean, it should be enough to say in the .swap unit for zswap that there is no Conflicts towards a unit like shutdown.target, umount.target and so on. With that in place we'll leave the swap device up until the very end, but still allow users to manually stop it with "systemctl stop" if they like. It's just that systemd won't do that on its own during shutdown. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] swap on zram service unit, using Conflicts=umount
On Mo, 24.06.19 13:16, Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) wrote: > > So for tmpfs mounts that don't turn off DefaultDependencies= we > > implicit add in an After=swap.target ordering dep. The thinking was > > that there's no point in swapping in all data of a tmpfs because we > > want to detach the swap device when we are going to flush it all out > > right after anyway. This made quite a difference to some folks. > > But we add Conflicts=umount.target, Before=umount.target, so we do > swapoff on all swap devices, which means that swap in the data after all. > Maybe that's an error, and we should remove this, at least for > normal swap partitions (not files)? We never know what kind of weird storage swap might be on, I'd probably leave that in, as it's really hard to figure out correctly when leaving swap on would be safe and when not. Or to say this differently: if people want to micro-optimize that, they by all means should, but in that case they should probably drop in their manually crafted .swap unit with DefaultDependencies=no and all the ordering in place they need, and nothing else. i.e. I believe this kind of optimization is nothing we need to cater for in the generic case when swap is configured with /etc/fstab or through GPT enumeration. zswap is different: we know exactly that the swap data is located in RAM, not on complex storage, hence it's entirely safe to not disassemble it at all, iiuc. Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] Is it possible to set a default for MemoryMax?
On Mo, 24.06.19 21:41, McKay, Sean (sean.mc...@hpe.com) wrote: > Thanks for the pointer, Lennart! > > I've done some initial review of the commit you pointed me to, and > it seems like it should be pretty straightforward to use that > understanding to implement the other functions. Might take a bit > depending on my local management's perspective on the priority of > getting this done, but I'll put together a PR as soon as I have a > chance. > > In the meantime, can anyone tell me if there's any semblance of a > 'getting started' page for systemd development? The main code > changes should be easy enough, but I see that there are a number of > tests included with the particular commit I was pointed to in > addition to the code change itself. I'd like to make sure that I > know what sort of tests are expected in a commit and how to run them > (and any other things I might need to do to ensure things are in > order before submitting the PR). https://systemd.io/HACKING.html (Btw, in addition to DefaultMemoryMin= there's now DefaultMemoryLow=. It has been added in a follow-up patch.) Lennart -- Lennart Poettering, Berlin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
