[systemd-devel] systemd 252 released
A new, official systemd release has just been tagged . Please download the tarball here: https://github.com/systemd/systemd/archive/v252.tar.gz Changes since the previous release: Announcements of Future Feature Removals: * We intend to remove cgroup v1 support from systemd release after the end of 2023. If you run services that make explicit use of cgroup v1 features (i.e. the "legacy hierarchy" with separate hierarchies for each controller), please implement compatibility with cgroup v2 (i.e. the "unified hierarchy") sooner rather than later. Most of Linux userspace has been ported over already. * We intend to remove support for split-usr (/usr mounted separately during boot) and unmerged-usr (parallel directories /bin and /usr/bin, /lib and /usr/lib, etc). This will happen in the second half of 2023, in the first release that falls into that time window. For more details, see: https://lists.freedesktop.org/archives/systemd-devel/2022-September/048352.html Compatibility Breaks: * ConditionKernelVersion= checks that use the '=' or '!=' operators will now do simple string comparisons (instead of version comparisons á la stverscmp()). Version comparisons are still done for the ordering operators '<', '>', '<=', '>='. Moreover, if no operator is specified, a shell-style glob match is now done. This creates a minor incompatibility compared to older systemd versions when the '*', '?', '[', ']' characters are used, as these will now match as shell globs instead of literally. Given that kernel version strings typically do not include these characters we expect little breakage through this change. * The service manager will now read the SELinux label used for SELinux access checks from the unit file at the time it loads the file. Previously, the label would be read at the moment of the access check, which was problematic since at that time the unit file might already have been updated or removed. New Features: * systemd-measure is a new tool for calculating and signing expected TPM2 PCR values for a given unified kernel image (UKI) booted via sd-stub. The public key used for the signature and the signed expected PCR information can be embedded inside the UKI. This information can be extracted from the UKI by external tools and code in the image itself and is made available to userspace in the booted kernel. systemd-cryptsetup, systemd-cryptenroll, and systemd-creds have been updated to make use of this information if available in the booted kernel: when locking an encrypted volume/credential to the TPM systemd-cryptenroll/systemd-creds will use the public key to bind the volume/credential to any kernel that carries PCR information signed by the same key pair. When unlocking such volumes/credentials systemd-cryptsetup/systemd-creds will use the signature embedded in the booted UKI to gain access. Binding TPM-based disk encryption to public keys/signatures of PCR values — instead of literal PCR values — addresses the inherent "brittleness" of traditional PCR-bound TPM disk encryption schemes: disks remain accessible even if the UKI is updated, without any TPM specific preparation during the OS update — as long as each UKI carries the necessary PCR signature information. Net effect: if you boot a properly prepared kernel, TPM-bound disk encryption now defaults to be locked to kernels which carry PCR signatures from the same key pair. Example: if a hypothetical distro FooOS prepares its UKIs like this, TPM-based disk encryption is now – by default – bound to only FooOS kernels, and encrypted volumes bound to the TPM cannot be unlocked on kernels from other sources. (But do note this behaviour requires preparation/enabling in the UKI, and of course users can always enroll non-TPM ways to unlock the volume.) * systemd-pcrphase is a new tool that is invoked at six places during system runtime, and measures additional words into TPM2 PCR 11, to mark milestones of the boot process. This allows binding access to specific TPM2-encrypted secrets to specific phases of the boot process. (Example: LUKS2 disk encryption key only accessible in the initrd, but not later.) Changes in systemd itself, i.e. the manager and units * The cpu controller is delegated to user manager units by default, and CPUWeight= settings are applied to the top-level user
Re: [systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent
I had my focus on the child (main daemon process), therefore the term went backward parent and grand-parent. Sorry, if that caused confusion, but let's go with your definitions. Thank you for the explanation and pointing out the steps! Especially 1. makes it much clearer than the list described here https://www.freedesktop.org/software/systemd/man/daemon.html, which only mention that P' shall call exit(). chrony is probably not making sure of 1., were P should wait until P' has exited. Now that you mentioned sysv semantics is old I come across an option in chrony which will not fork. I shall probably try using that instead. Best Regards, Christopher Wong From: Lennart Poettering Sent: Monday, October 31, 2022 11:43:17 AM To: Christopher Wong Cc: systemd-devel@lists.freedesktop.org Subject: Re: [systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent On Mo, 31.10.22 11:40, Lennart Poettering (lenn...@poettering.net) wrote: > This is almost certainly a bug in chrony. If you use Type=forking, > then the process that systemd forks off (let's call it "P") should > wait until all of the below holds: > > 1. The middle child P' has exited > 2. The grandchild (and main daemon process) P'' is running > 3. The PID file has been successfully written to contain the PID of P''. BTW, let me add an explanation, *why* this is needed: if they leave P'' running for a bit longer, then there's a race: if for some reason the deamon ends up failing shortly after starting up there is a race if P' or P'' die first. If P'' dies first, then the service manager will never see its SIGCHLD and cannot determine there was a failure. If P' dies first then all is good, as the P'' SIGCHLD will be properly collected by the service manager. But anyway, it's 2022, chrony being stuck in sysv semantics is sad. Use sd_notify(). Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] systemd-resolved/NetworkManager resolv.conf handling
On 10/26/22 20:44, Thomas HUMMEL wrote: Hello, I'm not sure if this is a systemd-resolved or NetworkManager question but it involves both (I know Thomas HALLER is a member of this list too) on Fedora release 36 (Thirty Six) using the following kernel and packages 5.19.16-200.fc36.x86_64 #1 SMP PREEMPT_DYNAMIC systemd-250.8-1.fc36.x86_64 systemd-resolved-250.8-1.fc36.x86_64 NetworkManager-1.38.4-1.fc36.x86_64 I'm using a proprietary vpn client which does not seem to work very well with systemd-resolved. As a matter of fact it seems to create a manual NM profile which does not include dns properties and it seems to (try to) set /etc/resolv.conf aside (F5 vpn linux client f5fpc for the record) Making it work is not the question here. I'm trying to understand how the 2 nameservers it configures may end up in /run/systemd/resolve/resolv.conf (and global systemd-resolved config as shown by resolvectl status) ONLY when I switch from a non systemd-resolved config then back to a systemd-resolved config /etc/resolv.conf is usually symlink to either /run/systemd/resolve/resolv.conf or /run/systemd/resolve/stub-resolv.conf. These nameservers ends there, because the f5fpc client just rewritten /etc/resolv.conf with a content it thought is appropriate. I think you should raise and issue to f5 support and request correct integration with at least Network Manager. If it had been told the dns servers it should use, it could propagate them to systemd-resolved. If it has already NM profile, I don't see a reason why DNS servers are not configured by it. It should allow at least by some configuration change to propagate those servers to NM. It should not overwrite /etc/resolv.conf, especially if it is just symlink to other place. I would suggest using strace to find what exactly it does and what it tries to modify. I expect sources for that client are not available. Here's exactly what I'm doing/experiencing: Starting from a) default NetworkManager config: # grep -iE 'dns|rc\.manager' NetworkManager.conf # ls -l conf.d/ total 0 b) systemd-resolved stub-resolv.conf mode: # ls -l /etc/resolv.conf lrwxrwxrwx 1 root root 37 Oct 26 19:15 /etc/resolv.conf -> /run/systemd/resolve/stub-resolv.conf and with (not linked from /etc/resolv.conf) : /run/systemd/resolve/resolve.conf following content: nameserver 192.168.1.1 nameserver 2a01:cb00:7e1:3300:aa6a:bbff:fe6e:190 search home matching my auto wireless NM profile 1) I start the vpn client obviously it does not work very well with systemd-resolved as I don't get corresponding nameserver (10.33.1.2,10.33.1.3) anywhere and name resolution does not work for corresponding zones /run/systemd/resolve/resolve.conf content has not changed 2) I stop the vpn client, and switch to the following setup # rm /etc/resolv.conf rm: remove symbolic link '/etc/resolv.conf'? y # cat < /etc/NetworkManager/conf.d/foo.conf > [main] > dns=default > rc.manager=file > EOF # reboot -> after the reboot the /etc/resolv.conf link as been recreated : why ? (/run/systemd/resolve/resolv.conf hasn't changed, which seems normal to me) 3) I remove it again and reboot # rm /etc/resolv.conf rm: remove symbolic link '/etc/resolv.conf'? y # reboot The systemd guys believe the systemd-resolved should always create /etc/resolv.conf if it does not exist already. Create empty /etc/resolv.conf file as a workaround. -> this time /etc/resolv.conf is as expected a regular file which content is handled by NM: $ ls -l /etc/resolv.conf -rw-r--r-- 1 root root 114 Oct 26 20:22 /etc/resolv.conf $ cat /etc/resolv.conf # Generated by NetworkManager search home nameserver 192.168.1.1 nameserver 2a01:cb00:7e1:3300:aa6a:bbff:fe6e:190 4) I start the vpn client it wrote to /etc/resolv.conf (which seems wrong to me but is out of scope here) $ cat /etc/resolv.conf #F5 Networks Inc. :File modified by VPN process search pasteur.fr home nameserver 10.33.1.2 nameserver 10.33.1.3 the 2 nameservers it provided do not appear in /run/systemd/resolve/resolv.conf 6) I stop the vpn client switch back to my orgininal config, and reboot # rm /etc/NetworkManager/conf.d/foo.conf rm: remove regular file '/etc/NetworkManager/conf.d/foo.conf'? y # rm /etc/resolv.conf rm: remove regular file '/etc/resolv.conf'? y # ln -s /run/systemd/resolve/stub-resolv.conf /etc/resolv.conf # reboot -> everything looks as expected 7) I start the vpn client -> its provided nameserver appear in /run/systemd/resolv/resolv.conf (and resolution of related zones work) -> why ? Where does the info come from ? nameserver 10.33.1.2 nameserver 10.33.1.3 nameserver 192.168.1.1 # Too many DNS servers configured, the following entries may be ignored. nameserver 2a01:cb00:7e1:3300:aa6a:bbff:fe6e:190 search pasteur.fr home Can you help me figure out what's happening or at least how can the behavior seem to change with what seem a rollback to the initial state ? I just guess
Re: [systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent
On Mo, 31.10.22 11:40, Lennart Poettering (lenn...@poettering.net) wrote: > This is almost certainly a bug in chrony. If you use Type=forking, > then the process that systemd forks off (let's call it "P") should > wait until all of the below holds: > > 1. The middle child P' has exited > 2. The grandchild (and main daemon process) P'' is running > 3. The PID file has been successfully written to contain the PID of P''. BTW, let me add an explanation, *why* this is needed: if they leave P'' running for a bit longer, then there's a race: if for some reason the deamon ends up failing shortly after starting up there is a race if P' or P'' die first. If P'' dies first, then the service manager will never see its SIGCHLD and cannot determine there was a failure. If P' dies first then all is good, as the P'' SIGCHLD will be properly collected by the service manager. But anyway, it's 2022, chrony being stuck in sysv semantics is sad. Use sd_notify(). Lennart -- Lennart Poettering, Berlin
Re: [systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent
On Mo, 31.10.22 08:04, Christopher Wong (christopher.w...@axis.com) wrote: > Hi, > > > We have during boot received the "Supervising process..." warning > from systemd related to chronyd.service. This is not always > happening, but when it happens systemd receives SIGCHLD from > grand-parent (22955) before the parent (22956). See logs below: grand-parent? do you mean grand-child? I am a bit confusing what you are trying to say? > Oct 25 10:34:55.104980 axis-accc8ed1c728 systemd[22955]: chronyd.service: > Executing: /usr/sbin/chronyd -u chronyd -f /run/chrony/chronyd.conf > Oct 25 10:34:55.117999 axis-accc8ed1c728 chronyd[22957]: chronyd version 4.2 > starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP -SCFILTER -SIGND +ASYNCDNS > +NTS -SECHASH +IPV6 +DEBUG) > Oct 25 10:34:55.120781 axis-accc8ed1c728 chronyd[22957]: Frequency -8.172 +/- > 1.366 ppm read from /var/lib/chrony/drift > Oct 25 10:34:55.124304 axis-accc8ed1c728 systemd[1]: > systemd-journald.service: Received EPOLLHUP on stored fd 82 (stored), closing. > Oct 25 10:34:55.126460 axis-accc8ed1c728 systemd[1]: Received SIGCHLD from > PID 22955 (chronyd). > Oct 25 10:34:55.126708 axis-accc8ed1c728 systemd[1]: Child 22955 (chronyd) > died (code=exited, status=0/SUCCESS) > Oct 25 10:34:55.126920 axis-accc8ed1c728 systemd[1]: chronyd.service: Child > 22955 belongs to chronyd.service. > Oct 25 10:34:55.127000 axis-accc8ed1c728 systemd[1]: chronyd.service: Control > process exited, code=exited, status=0/SUCCESS (success) > Oct 25 10:34:55.127027 axis-accc8ed1c728 systemd[1]: chronyd.service: Got > final SIGCHLD for state start. > Oct 25 10:34:55.127160 axis-accc8ed1c728 systemd[1]: chronyd.service: > Potentially unsafe symlink chain, will now retry with relaxed checks: > /run/chrony/chronyd.pid > Oct 25 10:34:55.127571 axis-accc8ed1c728 systemd[1]: chronyd.service: New > main PID 22957 belongs to service, we are happy. > Oct 25 10:34:55.127598 axis-accc8ed1c728 systemd[1]: chronyd.service: Main > PID loaded: 22957 > Oct 25 10:34:55.127759 axis-accc8ed1c728 systemd[1]: Custom log in > process-util.c fnc pid_is_my_child(): pid: 22957, ppid: 22956, cached_pid: 1. > Oct 25 10:34:55.127785 axis-accc8ed1c728 systemd[1]: chronyd.service: > Supervising process 22957 which is not our child. We'll most likely not > notice when it exits. > Oct 25 10:34:55.127964 axis-accc8ed1c728 systemd[1]: chronyd.service: Changed > start -> running > Oct 25 10:34:55.128006 axis-accc8ed1c728 systemd[1]: chronyd.service: Job > 117032 chronyd.service/start finished, result=done > Oct 25 10:34:55.128053 axis-accc8ed1c728 systemd[1]: Started NTP > client/server. > ... > Oct 25 10:34:55.158173 axis-accc8ed1c728 systemd[1]: Received SIGCHLD from > PID 22956 (chronyd). > Oct 25 10:34:55.158436 axis-accc8ed1c728 systemd[1]: Child 22956 (chronyd) > died (code=exited, status=0/SUCCESS) > Oct 25 10:34:55.158679 axis-accc8ed1c728 systemd[1]: chronyd.service: Child > 22956 belongs to chronyd.service. > > > The chronyd does two forks. In the normal case the parent will die > first and then the grand-parent will die. This behavior is according > to the SysV Daemons implementation > https://www.freedesktop.org/software/systemd/man/daemon.html > However, it seems scheduling for parent and grand-parent can vary > and result in a different behavior. This is almost certainly a bug in chrony. If you use Type=forking, then the process that systemd forks off (let's call it "P") should wait until all of the below holds: 1. The middle child P' has exited 2. The grandchild (and main daemon process) P'' is running 3. The PID file has been successfully written to contain the PID of P''. That all said, it's 2022, maybe chrony should just use Type=notify and sd_notify() like any modern code? Lennart -- Lennart Poettering, Berlin
[systemd-devel] Warning "Supervising process..." due to SIGCHLD from grand-parent
Hi, We have during boot received the "Supervising process..." warning from systemd related to chronyd.service. This is not always happening, but when it happens systemd receives SIGCHLD from grand-parent (22955) before the parent (22956). See logs below: Oct 25 10:34:55.104980 axis-accc8ed1c728 systemd[22955]: chronyd.service: Executing: /usr/sbin/chronyd -u chronyd -f /run/chrony/chronyd.conf Oct 25 10:34:55.117999 axis-accc8ed1c728 chronyd[22957]: chronyd version 4.2 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP -SCFILTER -SIGND +ASYNCDNS +NTS -SECHASH +IPV6 +DEBUG) Oct 25 10:34:55.120781 axis-accc8ed1c728 chronyd[22957]: Frequency -8.172 +/- 1.366 ppm read from /var/lib/chrony/drift Oct 25 10:34:55.124304 axis-accc8ed1c728 systemd[1]: systemd-journald.service: Received EPOLLHUP on stored fd 82 (stored), closing. Oct 25 10:34:55.126460 axis-accc8ed1c728 systemd[1]: Received SIGCHLD from PID 22955 (chronyd). Oct 25 10:34:55.126708 axis-accc8ed1c728 systemd[1]: Child 22955 (chronyd) died (code=exited, status=0/SUCCESS) Oct 25 10:34:55.126920 axis-accc8ed1c728 systemd[1]: chronyd.service: Child 22955 belongs to chronyd.service. Oct 25 10:34:55.127000 axis-accc8ed1c728 systemd[1]: chronyd.service: Control process exited, code=exited, status=0/SUCCESS (success) Oct 25 10:34:55.127027 axis-accc8ed1c728 systemd[1]: chronyd.service: Got final SIGCHLD for state start. Oct 25 10:34:55.127160 axis-accc8ed1c728 systemd[1]: chronyd.service: Potentially unsafe symlink chain, will now retry with relaxed checks: /run/chrony/chronyd.pid Oct 25 10:34:55.127571 axis-accc8ed1c728 systemd[1]: chronyd.service: New main PID 22957 belongs to service, we are happy. Oct 25 10:34:55.127598 axis-accc8ed1c728 systemd[1]: chronyd.service: Main PID loaded: 22957 Oct 25 10:34:55.127759 axis-accc8ed1c728 systemd[1]: Custom log in process-util.c fnc pid_is_my_child(): pid: 22957, ppid: 22956, cached_pid: 1. Oct 25 10:34:55.127785 axis-accc8ed1c728 systemd[1]: chronyd.service: Supervising process 22957 which is not our child. We'll most likely not notice when it exits. Oct 25 10:34:55.127964 axis-accc8ed1c728 systemd[1]: chronyd.service: Changed start -> running Oct 25 10:34:55.128006 axis-accc8ed1c728 systemd[1]: chronyd.service: Job 117032 chronyd.service/start finished, result=done Oct 25 10:34:55.128053 axis-accc8ed1c728 systemd[1]: Started NTP client/server. ... Oct 25 10:34:55.158173 axis-accc8ed1c728 systemd[1]: Received SIGCHLD from PID 22956 (chronyd). Oct 25 10:34:55.158436 axis-accc8ed1c728 systemd[1]: Child 22956 (chronyd) died (code=exited, status=0/SUCCESS) Oct 25 10:34:55.158679 axis-accc8ed1c728 systemd[1]: chronyd.service: Child 22956 belongs to chronyd.service. The chronyd does two forks. In the normal case the parent will die first and then the grand-parent will die. This behavior is according to the SysV Daemons implementation https://www.freedesktop.org/software/systemd/man/daemon.html However, it seems scheduling for parent and grand-parent can vary and result in a different behavior. When systemd receives the SIGCHLD from the grand-parent (22955) it start to run service_load_pid_file() and set the new main PID, which also later triggers the warning "Supervising process...", but shouldn't systemd wait for the parent (22956) to the main PID to die before considering the service as started? Now chronyd is started before the parent (22956) has died, indicated by the log entry "Started NTP client/server." above. In https://www.freedesktop.org/software/systemd/man/systemd.service.html it is only mentioning that when the parent process exits then the unit is started. It doesn't mention grand-parent when type=forking and the service binary is doing double fork(). Testing with systemd 251.3. Best Regards, Christopher Wong
