** Description changed:
[ Impact ]
During unattended-upgrades or needrestart on systems with unreachable or
recovering remote network shares (NFS, CIFS) in /etc/fstab, `systemctl
daemon-reexec` triggers `systemd-fstab-generator`.
`fstab-generator` calls `canonicalize_mount_path()` which runs `chase()` into
the unreachable network mount, blocking indefinitely in uninterruptible sleep
(D-state).
After 90s, the generator process is killed by SIGALRM, returning -EPROTO.
During daemon-reexec, `manager_startup()` treats this error as fatal and
freezes PID 1 ("Freezing execution."), halting the entire OS and requiring a
hard reboot.
[ Fix ]
- `fstab-generator`: Avoid traversing into already mounted remote network
shares. Check `/proc/self/mountinfo` for the target path or resolved symlink
without path cache. If present as an active mount point, skip chasing the last
component to avoid entering the mounted remote filesystem and triggering
D-state hangs. Unmounted paths and parent directory symlinks continue to be
chased normally.
+ `fstab-generator`: Add `canonicalize_network_mount_path()` to avoid calling
`chase()` into already mounted remote network shares. Parse
`/proc/self/mountinfo` without a path cache and check whether the simplified
target path or its 1-level symlink target (`readlink_malloc()`) is already an
active mount point. If mounted, return the canonical path directly from
`/proc/self/mountinfo` without touching the remote VFS. Unmounted paths fall
back to `chase()` normally.
[ Test Plan ]
1. Mount a remote NFS share in /etc/fstab (e.g. /mnt/nfs).
2. Block traffic to the NFS server (e.g. `iptables -A OUTPUT -p tcp --dport
2049 -j DROP`).
3. Execute `systemctl daemon-reexec`.
- Without fix: `fstab-generator` blocks in D-state, times out after 90s, PID
1 prints "Freezing execution." and the system halts.
- - With fix: `fstab-generator` recognizes the active mount in mountinfo
without VFS lookup, skips traversal, and `daemon-reexec` completes in <1s.
+ - With fix: `fstab-generator` recognizes the active mount in
`/proc/self/mountinfo` without VFS lookup, skips `chase()`, and `daemon-reexec`
completes in <1s.
[ Where problems could occur ]
- - `fstab-generator`: Symlinked mount points whose target is already mounted
on a network share skip last-component chasing; unmounted symlinks continue to
be chased normally. Multi-level symlink chains and nested mounts under
unreachable network shares remain out of scope for generator canonicalization.
+ - `fstab-generator`: Network mount points that are already active in
`/proc/self/mountinfo` (directly or via a 1-level symlink) skip `chase()`;
unmounted paths continue to be chased normally. Multi-level symlink chains or
intermediate directory symlinks pointing to an active network mount will fall
back to `chase()`.
[ Other Info ]
- Upstream PR: https://github.com/systemd/systemd/pull/44369
- Upstream issue: https://github.com/systemd/systemd/issues/39354
- NixOS issue: https://github.com/NixOS/nixpkgs/issues/375376
- - Target Ubuntu series: noble (24.04 LTS), resolute (26.04 LTS), stonking
(26.10).
- - Development series status: The fix lands first in devel (stonking) via
upstream sync/cherry-pick; stable SRUs target noble and resolute.
- - Test suites: 0 regressions in systemd test suites.
+ - Target Ubuntu series: tenacious (27.04), stonking (26.10), resolute (26.04
LTS), noble (24.04 LTS).
+ - Development series status: The fix targets devel (tenacious) first via
upstream merge/cherry-pick; stable SRUs target stonking, resolute, and noble.
+ - Test suites: 0 regressions in systemd test suites
(TEST-81-GENERATORS.fstab-generator.sh extended with 7 test cases).
--- [ Original Report ]
I am running Ubuntu 24.04 on a Hyper-V virtual machine.
The system became unusable after an unattended-upgrade triggered a
systemd daemon-reexec via needrestart.
Environment:
- Ubuntu 24.04
- systemd 255.4-1ubuntu8.17
- SSSD 2.9.4-1.1ubuntu6.7
- Hyper-V virtual machine
- unattended-upgrades enabled
- needrestart installed
Timeline from the journal:
Aug 27 06:16:00 systemd[1]: Starting apt-daily-upgrade.service - Daily apt
upgrade and clean activities...
Aug 27 06:16:38 unattended-upgrade started
Aug 27 06:16:42 unattended-upgrade finished upgrading:
vim, vim-common, vim-runtime, vim-tiny, xxd,
libssl3t64, openssl
Aug 27 06:16:44 systemd[1]: Reexecuting requested from client PID 769685
('systemctl') (unit apt-daily-upgrade.service)...
Aug 27 06:16:44 systemd[1]: Reexecuting.
Aug 27 06:16:44 systemd[1]: systemd 255.4-1ubuntu8.17 running in system mode
(...)
About 90 seconds later:
Aug 27 06:18:14 systemd[1]: Failed to fork off sandboxing environment for
executing generators: Protocol error
Aug 27 06:18:14 systemd[1]: Freezing execution.
After that, PID 1 became effectively unresponsive.
Examples of subsequent failures:
dbus-daemon:
Failed to activate service 'org.freedesktop.systemd1': timed out
systemd-journald:
Failed to send WATCHDOG=1 notification message
SSSD socket activation also stopped working, so NSS lookups for domain
users failed.
Even a normal reboot was no longer possible:
Call to Reboot failed: Connection timed out
Failed to open /run/initctl: No such device or address
Failed to talk to init daemon: Connection timed out
The VM had to be hard-powered off from Hyper-V.
A SysRq task dump while the system was in the failed state showed PID 1
sleeping in waitid():
task:systemd state:S pid:1
Call Trace:
__schedule
schedule
do_wait
kernel_waitid
__do_sys_waitid
__x64_sys_waitid
...
The system itself was otherwise not under resource pressure:
- low load
- sufficient free memory
- no relevant D-state processes
- system D-Bus daemon still alive
The daemon-reexec appears to have been triggered by needrestart.
The installed needrestart hook contains:
/etc/needrestart/restart.d/systemd-manager:
exec systemctl daemon-reexec
needrestart internally identifies the target as "systemd-manager".
As a workaround I added:
/etc/needrestart/conf.d/99-no-systemd-manager-reexec.conf
with:
$nrconf{override_rc}{qr(^systemd-manager$)} = 0;
This prevents automatic daemon-reexec of PID 1 while keeping unattended-
upgrades enabled.
A similar "Failed to fork off sandboxing environment for executing
generators: Protocol error" event had already occurred once on the
previous day after a systemd reexec, but the system recovered that time.
This looks like a systemd daemon-reexec / generator sandboxing failure
rather than an SSSD or Hyper-V failure. SSSD only became a downstream
casualty after PID 1 entered "Freezing execution".
Please let me know if further logs, coredumps, or a full SysRq task dump
would be useful.
ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: systemd 255.4-1ubuntu8.17
ProcVersionSignature: Ubuntu 6.8.0-138.138-generic 6.8.12
Uname: Linux 6.8.0-138-generic x86_64
ApportVersion: 2.28.3-0ubuntu0.1
Architecture: amd64
CasperMD5CheckResult: pass
CloudArchitecture: x86_64
CloudID: none
CloudName: none
CloudPlatform: none
CloudSubPlatform: config
Date: Thu Aug 27 11:00:24 2026
InstallationDate: Installed on 2025-03-28 (517 days ago)
InstallationMedia: Ubuntu-Server 22.04.4 LTS "Jammy Jellyfish" - Release
amd64 (20240216.1)
Lspci:
Lspci-vt:
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Lsusb-t:
Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
MachineType: Microsoft Corporation Virtual Machine
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm
XDG_RUNTIME_DIR=<set>
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-138-generic
root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
SourcePackage: systemd
UpgradeStatus: Upgraded to noble on 2025-05-26 (458 days ago)
dmi.bios.date: 10/23/2025
dmi.bios.release: 4.1
dmi.bios.vendor: Microsoft Corporation
dmi.bios.version: Hyper-V UEFI Release v4.1
dmi.board.asset.tag: None
dmi.board.name: Virtual Machine
dmi.board.vendor: Microsoft Corporation
dmi.board.version: Hyper-V UEFI Release v4.1
dmi.chassis.asset.tag: 8801-4483-8005-5833-2836-0338-95
dmi.chassis.type: 3
dmi.chassis.vendor: Microsoft Corporation
dmi.chassis.version: Hyper-V UEFI Release v4.1
dmi.modalias:
dmi:bvnMicrosoftCorporation:bvrHyper-VUEFIReleasev4.1:bd10/23/2025:br4.1:svnMicrosoftCorporation:pnVirtualMachine:pvrHyper-VUEFIReleasev4.1:rvnMicrosoftCorporation:rnVirtualMachine:rvrHyper-VUEFIReleasev4.1:cvnMicrosoftCorporation:ct3:cvrHyper-VUEFIReleasev4.1:skuNone:
dmi.product.family: Virtual Machine
dmi.product.name: Virtual Machine
dmi.product.sku: None
dmi.product.version: Hyper-V UEFI Release v4.1
dmi.sys.vendor: Microsoft Corporation
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2165295
Title:
systemd freezes after daemon-reexec triggered by unattended-upgrades /
needrestart on Ubuntu 24.04
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2165295/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
