Re: [arch-general] netcl leaves network down until reenable performed - do we need note?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi list, I got hit by this, too. On Mon, 17 Aug 2020, Eli Schwartz via arch-general wrote: On 8/17/20 5:25 PM, David C. Rankin wrote: By "subservice" I think you mean ".include" stanzas? Your analysis is correct, this got removed from systemd here: https://github.com/systemd/systemd/commit/7ade8982ca1969e251a29589ae918c3b5c595afb and systemd 246 is the first release with its removal. However, netctl migrated over to *.d dropins here: https://git.archlinux.org/netctl.git/commit/?id=04d39b2573bd34d4159837afdb4793a0990bd44a So I think you should be fine if you've re-enabled the netctl profile with 1.18 or higher (released on 2018-08-07). That's 2 years. Granted, if it's been working for years, why would anyone care about manually re-enabling their netctl profile... but... There should also probably have been a warning logged in journalctl for this, if your service was still using the old method. That's true, however, when this warning first appeared, `netctl reenable my-profile` did not help - same was true several days later (probably also weeks/months). So I forgot about the warning (I'm not keen to put `for profile in $list; do netctl reenable $profile; done` into my common update routine for all my arch boxes). netctl@rlf_network\x2dstatic.service.d/ └── profile.conf However, there is no note or warning during update that any manual intervention will be needed. That will leave anyone adminning a remote arch install with netctl with a box that is unreachable and has no network. Shouldn't there be a warning about this change generated on update? Arch is always pretty good about warning when manual interaction is required -- and this is a biggie. I'm not sure it merits a news post for something that old which is only now becoming fatal. In my opinion, a news post would have been appropriate. But now it's "too late" (see below). Hopefully anyone with remotely adminned boxes caught this while monitoring journalctl logs. Logs were unhelpful at that time, see above. But, thanks for posting this to the list -- at the very least, people in the same situation will be able to figure out what happened by reading here. And hopefully they will see this *before* upgrading. Additionally, people on the list might catch this issue in advance (if they don't update too often and have not yet been hit by this issue). This makes a news post obsolete, now, I believe. Couldn't there also be a post install that does a reenable for each netctl profile found in /etc/systemd/system as another option to avoid this SNAFU? That might have been an interesting precautionary measure for netctl 1.18, at least for printing a message advising people to reenable the service. I'm not sure it makes sense to do that automatically, since disabling a profile removes customizations and the netctl manpage explicitly warns you to be careful about doing so. I think, it's not arch's way to do such things. Arch rather says "your config is broken/old, run `xyz` to fix it" than simply running `xyz`. -- Eli Schwartz Bug Wrangler and Trusted User regards, Erich -BEGIN PGP SIGNATURE- iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAl87ZzQACgkQCu7JB1Xa e1qq7w/+KHd/vyL9AloIgirCeEcYf/+JxcCOngmROoxbM/Gs3rtxynewpwqV6iCi VJGIlrH5Xl2/NasKONRQuzNyGD0qqIMMRbAiNOZq3r37RAb8vmWmRxJmY5rkymO7 yoG4ox4HSgfeTyCvQhUnMZWE8Qm7INjcI/nHvs//wIwqB5ppeLkkFLHK9LCWBpaF +dkKP5vrsvjrD9785RjjMg46iHDCJ3AEAPuZmWes7IgUR161MD7Ujf07YMDfit8u 1SrdtxsncPSR0OX/m4DKC31tBP5WjS4kGL61NjUfQm354YtkjhyULQ7ecobRNBqA xQ4i6nLpbbZJSx38S0jfOgsAqKxM5HV6pFXRUR1GTT6a0MTxXG9bERmVm6oaFQeN 9cm3b4MbeBDVXp5x7MI2VGlG2mmNaIxZWpMKtUufQwBOxSkDSp5rifqy+NUgPhLX H/WcdK8pCQfIVfiopENVA9aNrb25/ST79io0KfUNmON3HHEnCWaJGHmCFMB/yOe7 2frLccn6TP17KkSSL5cJev5tpB+epH9k3BUxln7BT9ZDGVNJwbhlksaKXWncj8Hg OEvt6vqHCuswwvcrAvTJwDeCVtQRU4dt3cqSqCHuJjKzNg6egE1gU9KOxvFkZE0T 5dNq9DrcixQoXbG59DhNsOkM8b186HvvXOxTVBJvsFwS46Nnn4g= =/I9c -END PGP SIGNATURE-
Re: [arch-general] netcl leaves network down until reenable performed - do we need note?
On 8/17/20 5:25 PM, David C. Rankin wrote: > Archdevs, > > I have two computers using netctl, one for a static and one for a dhcp > address. After the last update on May, 3, > > netctl (1.22-1 -> 1.23-1) > > all remained fine and old profiles generated as subservices in > /etc/systemd/system, such as: > > netctl\@rlf_network\\x2dstatic.service > > continued to bring the network up. However within the past week there was a > change, likely to systemd 246, that no longer respects the subservices > originally created by netctl. Now attempting at reboot causes the network to > fail completely. Not good for remotely adminned boxes. > > What is required is a "reenable" of the netctl profile. Doing so will remove > the old subservice and symlinks and replace then with a directory in > /etc/systemd/system of same name as the old subservice, but with a ".d" > appended that now contains a "profile.conf", e.g. By "subservice" I think you mean ".include" stanzas? Your analysis is correct, this got removed from systemd here: https://github.com/systemd/systemd/commit/7ade8982ca1969e251a29589ae918c3b5c595afb and systemd 246 is the first release with its removal. However, netctl migrated over to *.d dropins here: https://git.archlinux.org/netctl.git/commit/?id=04d39b2573bd34d4159837afdb4793a0990bd44a So I think you should be fine if you've re-enabled the netctl profile with 1.18 or higher (released on 2018-08-07). That's 2 years. Granted, if it's been working for years, why would anyone care about manually re-enabling their netctl profile... but... There should also probably have been a warning logged in journalctl for this, if your service was still using the old method. > netctl@rlf_network\x2dstatic.service.d/ > └── profile.conf > > However, there is no note or warning during update that any manual > intervention will be needed. That will leave anyone adminning a remote arch > install with netctl with a box that is unreachable and has no network. > > Shouldn't there be a warning about this change generated on update? Arch is > always pretty good about warning when manual interaction is required -- and > this is a biggie. I'm not sure it merits a news post for something that old which is only now becoming fatal. Hopefully anyone with remotely adminned boxes caught this while monitoring journalctl logs. But, thanks for posting this to the list -- at the very least, people in the same situation will be able to figure out what happened by reading here. And hopefully they will see this *before* upgrading. > Couldn't there also be a post install that does a reenable for each netctl > profile found in /etc/systemd/system as another option to avoid this SNAFU? That might have been an interesting precautionary measure for netctl 1.18, at least for printing a message advising people to reenable the service. I'm not sure it makes sense to do that automatically, since disabling a profile removes customizations and the netctl manpage explicitly warns you to be careful about doing so. -- Eli Schwartz Bug Wrangler and Trusted User signature.asc Description: OpenPGP digital signature
[arch-general] netcl leaves network down until reenable performed - do we need note?
Archdevs, I have two computers using netctl, one for a static and one for a dhcp address. After the last update on May, 3, netctl (1.22-1 -> 1.23-1) all remained fine and old profiles generated as subservices in /etc/systemd/system, such as: netctl\@rlf_network\\x2dstatic.service continued to bring the network up. However within the past week there was a change, likely to systemd 246, that no longer respects the subservices originally created by netctl. Now attempting at reboot causes the network to fail completely. Not good for remotely adminned boxes. What is required is a "reenable" of the netctl profile. Doing so will remove the old subservice and symlinks and replace then with a directory in /etc/systemd/system of same name as the old subservice, but with a ".d" appended that now contains a "profile.conf", e.g. netctl@rlf_network\x2dstatic.service.d/ └── profile.conf However, there is no note or warning during update that any manual intervention will be needed. That will leave anyone adminning a remote arch install with netctl with a box that is unreachable and has no network. Shouldn't there be a warning about this change generated on update? Arch is always pretty good about warning when manual interaction is required -- and this is a biggie. Couldn't there also be a post install that does a reenable for each netctl profile found in /etc/systemd/system as another option to avoid this SNAFU? -- David C. Rankin, J.D.,P.E.
Re: [arch-general] Telinit?
Op ma 17 aug. 2020 18:08 schreef David Rosenstrauch : > > > On 8/17/20 12:01 PM, Giancarlo Razzolini via arch-general wrote: > > systemctl rescue > > Thanks! > > > > But this begs the question, why are you doing this before running pacman? > > ??? I always shut down all running daemons when I'm about to update my > system - seems like standard operating procedure to me: For (older) Windows systems: yes, something like this was sometimes neccesary. Unix behaves different: 1) I'd expect > that it would be completely unpredictable what would happen if you > updated/replaced an application's files while it was running, Actually, not much. When $process has a file open and that file gets deleted or replaced, $process can hapily continue using the old data. Until it reopens the file (data or application code), it'll keep using the old version. 2) there's > often config file changes that occur; again, I don't want to try to > update or resolve those while an application is running. > Same as #1: config files are usually read when starting and ignored while running. That said, it's still a good idea to restart the running services (or the whole server when the kernel is updated), but in principle you can just continue working while updating and reboot sometime later. Mvg, Guus Snijders >
Re: [arch-general] Telinit?
On 8/17/20 12:19 PM, Giancarlo Razzolini via arch-general wrote: Em agosto 17, 2020 13:08 David Rosenstrauch escreveu: ??? I always shut down all running daemons when I'm about to update my system - seems like standard operating procedure to me: 1) I'd expect that it would be completely unpredictable what would happen if you updated/replaced an application's files while it was running, 2) there's often config file changes that occur; again, I don't want to try to update or resolve those while an application is running. It's not necessary to do so in most cases, but, you might be interested in checkservices [0]. We use it when we update Arch servers, but we don't bring the system to rescue because, you know, that would interrupt all the services. And almost all of them (all?), can run just fine until they are restarted. So, my suggestion to you is, run pacman, and optionally run checkservices afterwards, to restart things that might be needed, if you want. No need to put the system in rescue mode. Thanks for the pointer to checkservices. Will read up on it. That said, I think I'm still likely to stick with updating in rescue mode: For the specific server in question (my home server) some brief downtime isn't really an issue. And at work, where nearly everything is cloud (and downtime would be an issue), we always just replace an instance with one running a newer version, rather than update-in-place. Thanks, DR
Re: [arch-general] Telinit?
Em agosto 17, 2020 13:08 David Rosenstrauch escreveu: ??? I always shut down all running daemons when I'm about to update my system - seems like standard operating procedure to me: 1) I'd expect that it would be completely unpredictable what would happen if you updated/replaced an application's files while it was running, 2) there's often config file changes that occur; again, I don't want to try to update or resolve those while an application is running. It's not necessary to do so in most cases, but, you might be interested in checkservices [0]. We use it when we update Arch servers, but we don't bring the system to rescue because, you know, that would interrupt all the services. And almost all of them (all?), can run just fine until they are restarted. So, my suggestion to you is, run pacman, and optionally run checkservices afterwards, to restart things that might be needed, if you want. No need to put the system in rescue mode. Regards, Giancarlo Razzolini [0] https://github.com/archlinux/contrib/blob/master/admin/checkservices pgpHbUxGhNrAG.pgp Description: PGP signature
Re: [arch-general] Telinit?
On 8/17/20 12:01 PM, Giancarlo Razzolini via arch-general wrote: systemctl rescue Thanks! But this begs the question, why are you doing this before running pacman? ??? I always shut down all running daemons when I'm about to update my system - seems like standard operating procedure to me: 1) I'd expect that it would be completely unpredictable what would happen if you updated/replaced an application's files while it was running, 2) there's often config file changes that occur; again, I don't want to try to update or resolve those while an application is running. DR
Re: [arch-general] Telinit?
Em agosto 17, 2020 12:56 David Rosenstrauch escreveu: Hmmm ... OK. I wasn't aware that telinit was now being considered legacy/deprecated. I've habitually used it for years to drop to single user mode, which I do every time before I perform a system update with pacman. I guess I'll have to find some other command to do the same using systemd. (I think they call it rescue mode, rather than single user mode.) Thanks, systemctl rescue But this begs the question, why are you doing this before running pacman? Regards, Giancarlo Razzolini pgp58HiUaCd7S.pgp Description: PGP signature
Re: [arch-general] Telinit?
Thanks much for the detailed explanation. Response below. On 8/16/20 10:38 PM, Eli Schwartz via arch-general wrote: The sysvcompat symlinks still installed are: - halt - reboot - poweroff - shutdown - init These programs are generically useful on fully systemd systems, and systemd documents them as such: "These commands are implemented in a way that preserves basic compatibility with the original SysV commands. systemctl(1) verbs halt, poweroff, reboot provide the same functionality with some additional features." When it comes to telinit, the description is, rather: "This is a legacy command available for compatibility only. It should not be used anymore, as the concept of runlevels is obsolete." Hmmm ... OK. I wasn't aware that telinit was now being considered legacy/deprecated. I've habitually used it for years to drop to single user mode, which I do every time before I perform a system update with pacman. I guess I'll have to find some other command to do the same using systemd. (I think they call it rescue mode, rather than single user mode.) Thanks, DR It seems clear to me why it's no longer installed except when systemd is configured with the necessary code to interpret and convert old /etc/init.d and /etc/rc.d infrastructure into stubbed systemd units. As for whether systemd should provide a split sysvcompat package that provides symlinks for generally useful programs styled after sysvinit, or, alternatively, provide the full-blown HAVE_SYSV_COMPAT initscript parser etc? Personally, I don't believe the split is useful anymore, as I believe it was originally meant for the sysvinit -> systemd migration period to allow having both installed at the same time and easily switching between the two. I'd rather remove it entirely, and fold it into "systemd". It's required by "base" anyway, lol. I don't believe the intention is to provide runtime generation of systemd units, and I think the pkgdesc is misleadingly simple in that regard. ... Anyway, if you want to have dialogue about whether it's useful to have a telinit program, regardless of upstream's defaults, by all means, feel free to have that discussion. But can it please not include rationalizations like "why are we deviating from upstream by not including it"?
