** Description changed:
+ [ Impact ]
+ During unattended-upgrades or needrestart on systems with unreachable or
recovering remote network shares (NFS, CIFS) in /etc/fstab, `systemctl
daemon-reexec` triggers `systemd-fstab-generator`.
+ `fstab-generator` calls `canonicalize_mount_path()` which runs `chase()` into
the unreachable network mount, blocking indefinitely in uninterruptible sleep
(D-state).
+ After 90s, the generator process is killed by SIGALRM, returning -EPROTO.
During daemon-reexec, `manager_startup()` treats this error as fatal and
freezes PID 1 ("Freezing execution."), halting the entire OS and requiring a
hard reboot.
+
+ [ Fix ]
+ Two-part systemic fix:
+ 1. `core/manager`: During daemon-reexec on an already running system
(`serialization && !switching_root`), do not freeze PID 1 on generator
failures. Log an error acknowledging that on-disk generated units may be
missing, and proceed with deserializing in-memory units, matching
`daemon-reload` resilience. Boot-time, switch-root, and soft-reboot handling is
untouched.
+ 2. `fstab-generator`: Avoid traversing into already mounted remote network
shares. Check `/proc/self/mountinfo` for the target path or resolved symlink
without path cache; if present as an active mount point, skip chasing the last
component to avoid entering the mounted remote filesystem.
+
+ [ Test Plan ]
+ 1. Mount a remote NFS share in /etc/fstab (e.g. /mnt/nfs).
+ 2. Block traffic to the NFS server (e.g. `iptables -A OUTPUT -p tcp --dport
2049 -j DROP`).
+ 3. Execute `systemctl daemon-reexec`.
+ - Without fix: PID 1 blocks in D-state, generator times out after 90s, PID
1 prints "Freezing execution." and the system halts.
+ - With fix: `fstab-generator` recognizes the active mount in mountinfo
without VFS lookup, skips traversal, and `daemon-reexec` completes in <1s. If
any generator times out, PID 1 logs an error and continues running without
freezing.
+
+ [ Where problems could occur ]
+ - `core/manager`: If a generator fails during reexec, units generated on disk
will be missing, relying on deserialized units from the previous manager
instance. Cold boot, switch-root, and soft-reboot error handling is untouched.
+ - `fstab-generator`: Symlinked mount points whose target is already mounted
on a network share skip last-component chasing; unmounted symlinks continue to
be chased normally.
+
+ [ Other Info ]
+ - Upstream PR: https://github.com/systemd/systemd/pull/44369
+ - Upstream issue: https://github.com/systemd/systemd/issues/39354
+ - NixOS issue: https://github.com/NixOS/nixpkgs/issues/375376
+ - Target Ubuntu series: noble (24.04 LTS), resolute (26.04 LTS), stonking
(26.10).
+
+
+
I am running Ubuntu 24.04 on a Hyper-V virtual machine.
The system became unusable after an unattended-upgrade triggered a
systemd daemon-reexec via needrestart.
Environment:
- Ubuntu 24.04
- systemd 255.4-1ubuntu8.17
- SSSD 2.9.4-1.1ubuntu6.7
- Hyper-V virtual machine
- unattended-upgrades enabled
- needrestart installed
Timeline from the journal:
Aug 27 06:16:00 systemd[1]: Starting apt-daily-upgrade.service - Daily apt
upgrade and clean activities...
Aug 27 06:16:38 unattended-upgrade started
Aug 27 06:16:42 unattended-upgrade finished upgrading:
- vim, vim-common, vim-runtime, vim-tiny, xxd,
- libssl3t64, openssl
+ vim, vim-common, vim-runtime, vim-tiny, xxd,
+ libssl3t64, openssl
Aug 27 06:16:44 systemd[1]: Reexecuting requested from client PID 769685
('systemctl') (unit apt-daily-upgrade.service)...
Aug 27 06:16:44 systemd[1]: Reexecuting.
Aug 27 06:16:44 systemd[1]: systemd 255.4-1ubuntu8.17 running in system mode
(...)
About 90 seconds later:
Aug 27 06:18:14 systemd[1]: Failed to fork off sandboxing environment for
executing generators: Protocol error
Aug 27 06:18:14 systemd[1]: Freezing execution.
After that, PID 1 became effectively unresponsive.
Examples of subsequent failures:
dbus-daemon:
- Failed to activate service 'org.freedesktop.systemd1': timed out
+ Failed to activate service 'org.freedesktop.systemd1': timed out
systemd-journald:
- Failed to send WATCHDOG=1 notification message
+ Failed to send WATCHDOG=1 notification message
SSSD socket activation also stopped working, so NSS lookups for domain
users failed.
Even a normal reboot was no longer possible:
Call to Reboot failed: Connection timed out
Failed to open /run/initctl: No such device or address
Failed to talk to init daemon: Connection timed out
The VM had to be hard-powered off from Hyper-V.
A SysRq task dump while the system was in the failed state showed PID 1
sleeping in waitid():
task:systemd state:S pid:1
Call Trace:
- __schedule
- schedule
- do_wait
- kernel_waitid
- __do_sys_waitid
- __x64_sys_waitid
- ...
+ __schedule
+ schedule
+ do_wait
+ kernel_waitid
+ __do_sys_waitid
+ __x64_sys_waitid
+ ...
The system itself was otherwise not under resource pressure:
- low load
- sufficient free memory
- no relevant D-state processes
- system D-Bus daemon still alive
The daemon-reexec appears to have been triggered by needrestart.
The installed needrestart hook contains:
/etc/needrestart/restart.d/systemd-manager:
- exec systemctl daemon-reexec
+ exec systemctl daemon-reexec
needrestart internally identifies the target as "systemd-manager".
As a workaround I added:
/etc/needrestart/conf.d/99-no-systemd-manager-reexec.conf
with:
$nrconf{override_rc}{qr(^systemd-manager$)} = 0;
This prevents automatic daemon-reexec of PID 1 while keeping unattended-
upgrades enabled.
A similar "Failed to fork off sandboxing environment for executing
generators: Protocol error" event had already occurred once on the
previous day after a systemd reexec, but the system recovered that time.
This looks like a systemd daemon-reexec / generator sandboxing failure
rather than an SSSD or Hyper-V failure. SSSD only became a downstream
casualty after PID 1 entered "Freezing execution".
Please let me know if further logs, coredumps, or a full SysRq task dump
would be useful.
ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: systemd 255.4-1ubuntu8.17
ProcVersionSignature: Ubuntu 6.8.0-138.138-generic 6.8.12
Uname: Linux 6.8.0-138-generic x86_64
ApportVersion: 2.28.3-0ubuntu0.1
Architecture: amd64
CasperMD5CheckResult: pass
CloudArchitecture: x86_64
CloudID: none
CloudName: none
CloudPlatform: none
CloudSubPlatform: config
Date: Thu Aug 27 11:00:24 2026
InstallationDate: Installed on 2025-03-28 (517 days ago)
InstallationMedia: Ubuntu-Server 22.04.4 LTS "Jammy Jellyfish" - Release
amd64 (20240216.1)
Lspci:
-
+
Lspci-vt:
-
+
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Lsusb-t:
-
+
Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
MachineType: Microsoft Corporation Virtual Machine
ProcEnviron:
- LANG=en_US.UTF-8
- PATH=(custom, no user)
- SHELL=/bin/bash
- TERM=xterm
- XDG_RUNTIME_DIR=<set>
+ LANG=en_US.UTF-8
+ PATH=(custom, no user)
+ SHELL=/bin/bash
+ TERM=xterm
+ XDG_RUNTIME_DIR=<set>
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-138-generic
root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
SourcePackage: systemd
UpgradeStatus: Upgraded to noble on 2025-05-26 (458 days ago)
dmi.bios.date: 10/23/2025
dmi.bios.release: 4.1
dmi.bios.vendor: Microsoft Corporation
dmi.bios.version: Hyper-V UEFI Release v4.1
dmi.board.asset.tag: None
dmi.board.name: Virtual Machine
dmi.board.vendor: Microsoft Corporation
dmi.board.version: Hyper-V UEFI Release v4.1
dmi.chassis.asset.tag: 8801-4483-8005-5833-2836-0338-95
dmi.chassis.type: 3
dmi.chassis.vendor: Microsoft Corporation
dmi.chassis.version: Hyper-V UEFI Release v4.1
dmi.modalias:
dmi:bvnMicrosoftCorporation:bvrHyper-VUEFIReleasev4.1:bd10/23/2025:br4.1:svnMicrosoftCorporation:pnVirtualMachine:pvrHyper-VUEFIReleasev4.1:rvnMicrosoftCorporation:rnVirtualMachine:rvrHyper-VUEFIReleasev4.1:cvnMicrosoftCorporation:ct3:cvrHyper-VUEFIReleasev4.1:skuNone:
dmi.product.family: Virtual Machine
dmi.product.name: Virtual Machine
dmi.product.sku: None
dmi.product.version: Hyper-V UEFI Release v4.1
dmi.sys.vendor: Microsoft Corporation
** Description changed:
[ Impact ]
During unattended-upgrades or needrestart on systems with unreachable or
recovering remote network shares (NFS, CIFS) in /etc/fstab, `systemctl
daemon-reexec` triggers `systemd-fstab-generator`.
`fstab-generator` calls `canonicalize_mount_path()` which runs `chase()` into
the unreachable network mount, blocking indefinitely in uninterruptible sleep
(D-state).
After 90s, the generator process is killed by SIGALRM, returning -EPROTO.
During daemon-reexec, `manager_startup()` treats this error as fatal and
freezes PID 1 ("Freezing execution."), halting the entire OS and requiring a
hard reboot.
[ Fix ]
Two-part systemic fix:
1. `core/manager`: During daemon-reexec on an already running system
(`serialization && !switching_root`), do not freeze PID 1 on generator
failures. Log an error acknowledging that on-disk generated units may be
missing, and proceed with deserializing in-memory units, matching
`daemon-reload` resilience. Boot-time, switch-root, and soft-reboot handling is
untouched.
2. `fstab-generator`: Avoid traversing into already mounted remote network
shares. Check `/proc/self/mountinfo` for the target path or resolved symlink
without path cache; if present as an active mount point, skip chasing the last
component to avoid entering the mounted remote filesystem.
[ Test Plan ]
1. Mount a remote NFS share in /etc/fstab (e.g. /mnt/nfs).
2. Block traffic to the NFS server (e.g. `iptables -A OUTPUT -p tcp --dport
2049 -j DROP`).
3. Execute `systemctl daemon-reexec`.
- - Without fix: PID 1 blocks in D-state, generator times out after 90s, PID
1 prints "Freezing execution." and the system halts.
- - With fix: `fstab-generator` recognizes the active mount in mountinfo
without VFS lookup, skips traversal, and `daemon-reexec` completes in <1s. If
any generator times out, PID 1 logs an error and continues running without
freezing.
+ - Without fix: PID 1 blocks in D-state, generator times out after 90s, PID
1 prints "Freezing execution." and the system halts.
+ - With fix: `fstab-generator` recognizes the active mount in mountinfo
without VFS lookup, skips traversal, and `daemon-reexec` completes in <1s. If
any generator times out, PID 1 logs an error and continues running without
freezing.
[ Where problems could occur ]
- `core/manager`: If a generator fails during reexec, units generated on disk
will be missing, relying on deserialized units from the previous manager
instance. Cold boot, switch-root, and soft-reboot error handling is untouched.
- `fstab-generator`: Symlinked mount points whose target is already mounted
on a network share skip last-component chasing; unmounted symlinks continue to
be chased normally.
[ Other Info ]
- Upstream PR: https://github.com/systemd/systemd/pull/44369
- Upstream issue: https://github.com/systemd/systemd/issues/39354
- NixOS issue: https://github.com/NixOS/nixpkgs/issues/375376
- Target Ubuntu series: noble (24.04 LTS), resolute (26.04 LTS), stonking
(26.10).
+ --- [ Original Report ]
I am running Ubuntu 24.04 on a Hyper-V virtual machine.
The system became unusable after an unattended-upgrade triggered a
systemd daemon-reexec via needrestart.
Environment:
- Ubuntu 24.04
- systemd 255.4-1ubuntu8.17
- SSSD 2.9.4-1.1ubuntu6.7
- Hyper-V virtual machine
- unattended-upgrades enabled
- needrestart installed
Timeline from the journal:
Aug 27 06:16:00 systemd[1]: Starting apt-daily-upgrade.service - Daily apt
upgrade and clean activities...
Aug 27 06:16:38 unattended-upgrade started
Aug 27 06:16:42 unattended-upgrade finished upgrading:
vim, vim-common, vim-runtime, vim-tiny, xxd,
libssl3t64, openssl
Aug 27 06:16:44 systemd[1]: Reexecuting requested from client PID 769685
('systemctl') (unit apt-daily-upgrade.service)...
Aug 27 06:16:44 systemd[1]: Reexecuting.
Aug 27 06:16:44 systemd[1]: systemd 255.4-1ubuntu8.17 running in system mode
(...)
About 90 seconds later:
Aug 27 06:18:14 systemd[1]: Failed to fork off sandboxing environment for
executing generators: Protocol error
Aug 27 06:18:14 systemd[1]: Freezing execution.
After that, PID 1 became effectively unresponsive.
Examples of subsequent failures:
dbus-daemon:
Failed to activate service 'org.freedesktop.systemd1': timed out
systemd-journald:
Failed to send WATCHDOG=1 notification message
SSSD socket activation also stopped working, so NSS lookups for domain
users failed.
Even a normal reboot was no longer possible:
Call to Reboot failed: Connection timed out
Failed to open /run/initctl: No such device or address
Failed to talk to init daemon: Connection timed out
The VM had to be hard-powered off from Hyper-V.
A SysRq task dump while the system was in the failed state showed PID 1
sleeping in waitid():
task:systemd state:S pid:1
Call Trace:
__schedule
schedule
do_wait
kernel_waitid
__do_sys_waitid
__x64_sys_waitid
...
The system itself was otherwise not under resource pressure:
- low load
- sufficient free memory
- no relevant D-state processes
- system D-Bus daemon still alive
The daemon-reexec appears to have been triggered by needrestart.
The installed needrestart hook contains:
/etc/needrestart/restart.d/systemd-manager:
exec systemctl daemon-reexec
needrestart internally identifies the target as "systemd-manager".
As a workaround I added:
/etc/needrestart/conf.d/99-no-systemd-manager-reexec.conf
with:
$nrconf{override_rc}{qr(^systemd-manager$)} = 0;
This prevents automatic daemon-reexec of PID 1 while keeping unattended-
upgrades enabled.
A similar "Failed to fork off sandboxing environment for executing
generators: Protocol error" event had already occurred once on the
previous day after a systemd reexec, but the system recovered that time.
This looks like a systemd daemon-reexec / generator sandboxing failure
rather than an SSSD or Hyper-V failure. SSSD only became a downstream
casualty after PID 1 entered "Freezing execution".
Please let me know if further logs, coredumps, or a full SysRq task dump
would be useful.
ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: systemd 255.4-1ubuntu8.17
ProcVersionSignature: Ubuntu 6.8.0-138.138-generic 6.8.12
Uname: Linux 6.8.0-138-generic x86_64
ApportVersion: 2.28.3-0ubuntu0.1
Architecture: amd64
CasperMD5CheckResult: pass
CloudArchitecture: x86_64
CloudID: none
CloudName: none
CloudPlatform: none
CloudSubPlatform: config
Date: Thu Aug 27 11:00:24 2026
InstallationDate: Installed on 2025-03-28 (517 days ago)
InstallationMedia: Ubuntu-Server 22.04.4 LTS "Jammy Jellyfish" - Release
amd64 (20240216.1)
Lspci:
Lspci-vt:
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Lsusb-t:
Lsusb-v: Error: command ['lsusb', '-v'] failed with exit code 1:
MachineType: Microsoft Corporation Virtual Machine
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm
XDG_RUNTIME_DIR=<set>
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-138-generic
root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
SourcePackage: systemd
UpgradeStatus: Upgraded to noble on 2025-05-26 (458 days ago)
dmi.bios.date: 10/23/2025
dmi.bios.release: 4.1
dmi.bios.vendor: Microsoft Corporation
dmi.bios.version: Hyper-V UEFI Release v4.1
dmi.board.asset.tag: None
dmi.board.name: Virtual Machine
dmi.board.vendor: Microsoft Corporation
dmi.board.version: Hyper-V UEFI Release v4.1
dmi.chassis.asset.tag: 8801-4483-8005-5833-2836-0338-95
dmi.chassis.type: 3
dmi.chassis.vendor: Microsoft Corporation
dmi.chassis.version: Hyper-V UEFI Release v4.1
dmi.modalias:
dmi:bvnMicrosoftCorporation:bvrHyper-VUEFIReleasev4.1:bd10/23/2025:br4.1:svnMicrosoftCorporation:pnVirtualMachine:pvrHyper-VUEFIReleasev4.1:rvnMicrosoftCorporation:rnVirtualMachine:rvrHyper-VUEFIReleasev4.1:cvnMicrosoftCorporation:ct3:cvrHyper-VUEFIReleasev4.1:skuNone:
dmi.product.family: Virtual Machine
dmi.product.name: Virtual Machine
dmi.product.sku: None
dmi.product.version: Hyper-V UEFI Release v4.1
dmi.sys.vendor: Microsoft Corporation
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2165295
Title:
systemd freezes after daemon-reexec triggered by unattended-upgrades /
needrestart on Ubuntu 24.04
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/2165295/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
