Public bug reported:

On nearly every shutdown, plymouthd --mode=shutdown aborts on an assertion just 
as PID 1 hands over
to systemd-shutdown. Because systemd-coredump.socket has already been stopped 
at that point, the
crash is never recorded, and the only visible trace is this line on the console:

    systemd-coredump[3927]: Failed to connect to coredump service:
Connection refused

Users keep seeing that message without any way to tell what is behind it (see 
bug #2058265 and
many Linux Mint / Ubuntu forum threads).

This is fixed upstream by commit 4f53b52f549b62676ccd025190b80246b2c23649, 
"drm: Fix crash when
terminal fd is still -1 after reconnect" (Hans de Goede, 2025-07-01, merge 
request !354:
https://gitlab.freedesktop.org/plymouth/plymouth/-/merge_requests/354). Fedora 
shipped it in
plymouth-24.004.60-19 (https://bugzilla.redhat.com/show_bug.cgi?id=2370979).

The backtrace in that commit message follows the same path as the one below:
  1. the terminal fd's disconnect handler calls open_input_source() again;
  2. the terminal could not be reopened;
  3. ply_event_loop_watch_fd() is called with fd -1.

Noble's current 24.004.60-1ubuntu7.2 does not include the fix.

Request: please backport commit 4f53b52f to noble as an SRU. It is small and 
self-contained
(8 insertions, 12 deletions): it adds an fd >= 0 check before 
ply_event_loop_watch_fd() in
open_input_source() of the drm and frame-buffer renderers.


SYSTEM

  Distribution:     Linux Mint 22.3, Ubuntu 24.04 (noble) base. Mint uses 
Ubuntu's plymouth
                    packages unmodified; these are installed from noble-updates.
  plymouth:         24.004.60-1ubuntu7.2 (also libplymouth5, 
plymouth-theme-spinner,
                    plymouth-theme-ubuntu-text)
  Kernel:           6.8.0-139-generic
  Machine:          Dell Inspiron 5566
  Graphics:         Intel HD 620 [8086:5916] + AMD Topaz XT [1002:6900], hybrid.
                    DRM nodes are card1/card2 (simpledrm has card0 early, then 
i915 takes over).
  Kernel cmdline:   ro quiet splash pcie_aspm=off
  Frequency:        the message appears at the end of every logged shutdown 
since 2026-07-25
                    except three


WHAT HAPPENS

Journal from one shutdown (the coredump-tattle lines come from the capture 
wrapper described
further down):

    systemd[1]: Reached target poweroff.target - System Power Off.
    systemd[1]: Shutting down.
    coredump-tattle: pid=335103 comm=plymouthd exe=/usr/sbin/plymouthd signal=6
                     cmdline=/usr/sbin/plymouthd --mode=shutdown 
--attach-to-session
    kernel: fbcon: Taking over console
    kernel: Console: switching to colour frame buffer device 170x48
    systemd-coredump[335226]: Failed to connect to coredump service: Connection 
refused
    systemd-shutdown[1]: Syncing filesystems and block devices.

The crash lands 131 ms after "Shutting down.", i.e. as PID 1 execs 
systemd-shutdown. Across
other shutdowns the gap was 3-57 ms. fbcon taking over right afterwards is a 
consequence of
plymouthd dying and dropping DRM master.


ASSERTION AND BACKTRACE

    plymouthd: ../src/libply/ply-event-loop.c:732: ply_event_loop_watch_fd:
               Assertion `fd >= 0' failed.

    #3  __GI_raise (sig=6)
    #4  __GI_abort ()
    #5  __assert_fail_base (assertion="fd >= 0", 
file="../src/libply/ply-event-loop.c",
                            line=732, function="ply_event_loop_watch_fd")
    #6  __assert_fail (...)
    #7  ply_event_loop_watch_fd () at /lib/x86_64-linux-gnu/libply.so.5
    #8  ??? () at /usr/lib/x86_64-linux-gnu/plymouth/renderers/drm.so
    #9  ply_event_loop_process_pending_events () at 
/lib/x86_64-linux-gnu/libply.so.5
    #10 ply_event_loop_run () at /lib/x86_64-linux-gnu/libply.so.5
    #11 main ()

No debug symbols were available for drm.so, so frame 8 was identified from its 
disassembly. The
code matches open_input_source() in the DRM renderer:
  - the has_input_source() assertion (input_source compared with 
&backend->input_source,
    offset 0x28);
  - a test of the input_source_is_open bit;
  - the backend->terminal NULL check;
  - then ply_terminal_get_fd() feeding ply_event_loop_watch_fd():

    mov    0x8(%r12),%rdi            ; backend->terminal
    test   %rdi,%rdi
    je     ...                       ; terminal == NULL: skip (the guard from 
bug #2103533)
    call   ply_terminal_get_fd@plt   ; returns -1 here, the terminal is already 
closed
    mov    %eax,%esi                 ; passed straight through as the fd
    mov    $0x1,%edx                 ; PLY_EVENT_LOOP_FD_STATUS_HAS_DATA
    call   ply_event_loop_watch_fd   ; assert (fd >= 0) -> abort

Frame 9 is the event loop dispatching a disconnect. The upstream commit message 
describes exactly
this path: on_input_source_disconnected() calls open_input_source() after
ply_terminal_reopen_device() has failed to reopen the terminal. Here the 
terminal presumably can't
be reopened because the session VT is being torn down at shutdown. The 
intermediate frames are
tail calls or inlined in this build.


WHY THE EXISTING NOBLE FIX DOES NOT COVER THIS

24.004.60-1ubuntu7.2 carries the two NULL-guard commits from bug #2103533 
(63597f92d1...,
5c10072a97...). Those guard backend->terminal == NULL. Here the terminal 
pointer is not NULL, so
the guard passes, but the terminal has already been closed. 
ply_terminal_get_fd() therefore
returns -1, and that value is handed to ply_event_loop_watch_fd(). Commit 
4f53b52f adds exactly
that missing check. The trigger differs from
rhbz#2370979 (pressing Escape during boot there; the VT hanging up at shutdown 
here), but the
assertion and line are identical.


HOW THE CRASH WAS CAPTURED

systemd-coredump.socket is stopped about half a second before the crash 
(Conflicts=
shutdown.target), so coredumpctl never sees it and the core is lost. To capture 
it, I pointed
kernel.core_pattern at a small wrapper that saves the dump itself when 
systemd-coredump is
refused. I set the pattern at runtime only, so it reverts at the next boot:

    sysctl -w kernel.core_pattern='|/usr/local/sbin/coredump-tattle %P
%u %g %s %t 9223372036854775808 %h %d'

/usr/local/sbin/coredump-tattle:

    #!/bin/sh
    comm=$(cat /proc/$1/comm 2>/dev/null)
    printf '<4>coredump-tattle: pid=%s comm=%s exe=%s signal=%s cmdline=%s\n' 
"$1" "$comm" \
      "$(readlink /proc/$1/exe 2>/dev/null)" "$4" \
      "$(tr '\0' ' ' < /proc/$1/cmdline 2>/dev/null | head -c 200)" > /dev/kmsg
    /usr/lib/systemd/systemd-coredump "$@" && exit 0
    umask 027
    out=/var/lib/coredump-tattle/core.$(printf %s "$comm" | tr -c 
'A-Za-z0-9._-' '_').$1.sig$4
    head -c 512M > "$out"
    printf '<4>coredump-tattle: saved %s (%s bytes)\n' "$out" "$(wc -c < 
"$out")" > /dev/kmsg

When its socket is refused, systemd-coredump exits before reading the dump, so 
the complete core
is still on stdin for the wrapper. / is still read-write at that stage. I no 
longer have the core
file itself, but the backtrace above came from it, and the wrapper reproduces 
it on any shutdown.


WORKAROUND (CONFIRMED)

Dropping --attach-to-session from the shutdown splash units stops the crash. 
plymouthd is then
left without a terminal, so the existing terminal == NULL guard skips the 
failing block.
  - The splash still shows as usual.
  - plymouth-poweroff.service still starts in about the same time (~8.5 s here).
  - The journal goes straight from "Shutting down." to "Syncing filesystems and 
block devices."
    with no coredump line.

/etc/systemd/system/plymouth-{poweroff,reboot,halt}.service.d/no-attach-
to-session.conf:

    [Service]
    ExecStart=
    ExecStart=/usr/sbin/plymouthd --mode=shutdown
    TimeoutStartSec=20

This is only a local workaround. Without --attach-to-session, plymouthd no 
longer redirects console
messages away from the screen, although no extra text was visible in practice.


IMPACT

Cosmetic but persistent. The splash dies ~0.1 s before power-off, uncovering 
the console, and every
shutdown ends with an error message that looks alarming and can't be explained 
without capturing
the core by hand. The visible symptom has been reported since at least March 
2024 (bug #2058265)
without the cause being identified.

** Affects: plymouth (Ubuntu)
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2167885

Title:
  plymouthd --mode=shutdown aborts with "Assertion `fd >= 0' failed" in
  ply_event_loop_watch_fd at shutdown (upstream MR !354 not in noble)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/plymouth/+bug/2167885/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to