Public bug reported:
With gnome-remote-desktop in remote-login (headless) mode, gnome-shell
crashes with SIGSEGV when the RDP client changes the virtual monitor to
a size that is not an exact multiple of the scale factor. The whole user
session is terminated, including all running applications.
The core dump shows that clutter_stage_paint_to_buffer() reads back a
5099×2033 texture into a PipeWire buffer that is only 5098×2032 (stride
20392 = 5098 × 4). The read-back writes one column and one row past the
end of the buffer; the actual fault happens inside Mesa's llvmpipe
(libgallium), but Mesa only executes the oversized read-back it is
given.
Environment
Ubuntu 26.04.1 LTS, kernel 7.0.0-43-generic, x86_64
Hyper-V guest, no GPU: hyperv_drm, rendering via llvmpipe
gnome-shell 50.1-0ubuntu1.5
mutter 50.1-0ubuntu2.5
gnome-remote-desktop 50.2-0ubuntu0.1 (system service, remote login, headless
session)
FreeRDP 3.32.2+dfsg-0ubuntu0.26.04.1
Mesa 26.0.8-1ubuntu0.3 (crash dump below); the crash also reproduces with Mesa
26.2.4-1ubuntu1~26.04.1 from resolute-proposed (no dump captured for that run)
RDP client:
Microsoft Windows App 2.0.1489.0 and 2.0.1601.0 (both reproduce), Windows 11
Enterprise build 26300
Windows desktop: 5120×2160 at 150 % scaling (144 DPI)
RDPGFX capability set 10.7 accepted, H.264 AVC444/AVC420 available
Steps to reproduce
Enable remote login in gnome-remote-desktop (system mode).
Connect with Windows App in full-screen mode from a Windows desktop of
5120×2160 at 150 %. The session gets virtual monitor Meta-0 with mode
5120×2160, scale 1.5. This is stable.
Leave full-screen mode in Windows App. The client sends a new monitor layout
and the virtual monitor changes to 5098×2032, scale 1.5.
gnome-shell crashes 1–15 s later with SIGSEGV; the session is logged out and
all applications are closed.
Observed monitor states (from org.gnome.Mutter.DisplayConfig.GetCurrentState
during the test):
Time Mode Logical monitor scale
before 5120×2160@60 1.5
after leaving full screen 5098×2032@60 1.5
~1 s later (none, gnome-shell crashed)
Expected result
The virtual monitor is resized and the session keeps running.
Analysis (from the core dump, symbols via debuginfod.ubuntu.com)
Frame 15, meta_screen_cast_stream_src_record_to_buffer: width=5098,
height=2032, stride=20392
Frame 14, meta_screen_cast_virtual_stream_src_record_to_buffer:
view_rect = {x = 0, y = 0, width = 3399, height = 1355}, scale = 1.5
Frame 13, clutter_stage_paint_to_buffer: rect = {0, 0, 3399, 1355},
scale = 1.5, texture_width = 5099, texture_height = 2033, destination
bitmap COGL_PIXEL_FORMAT_BGRA_8888_PRE, width 5099, height 2033
So the logical view size is rounded up (5098 / 1.5 = 3398.67 → 3399,
2032 / 1.5 = 1354.67 → 1355), and scaling it back gives a texture of
ceil(3399 × 1.5) × ceil(1355 × 1.5) = 5099 × 2033, which is larger than
the 5098 × 2032 buffer it is read into.
With 5120×2160 the round trip fits (3413 × 1.5 = 5119.5 → 5120; 1440 ×
1.5 = 2160), which is why full screen works.
This looks like the same class of problem that was fixed for area
screencasts in 2020 ("clutter/stage: Make paint_to_buffer use the
correct texture/buffer size"), but for the virtual stream source used by
gnome-remote-desktop.
Kernel message for one of the crashes (same Mesa offset in all dumps):
gnome-shell[…]: segfault at … error 6 in
libgallium-26.0.8-1ubuntu0.3.so[3b47e1,…]
Full symbolized backtrace: see attached backtrace.txt.
Workaround
Keep the RDP client in full-screen mode, use a fixed resolution, or use a
client scale factor that divides the window size exactly (100 % or 200 %).
** Affects: mutter (Ubuntu)
Importance: Undecided
Status: New
** Attachment added: "backtrace.txt"
https://bugs.launchpad.net/bugs/2175114/+attachment/6019572/+files/backtrace.txt
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2175114
Title:
gnome-shell SIGSEGV in
meta_screen_cast_virtual_stream_src_record_to_buffer when an RDP
client resizes a remote-login session with fractional scale 1.5
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/mutter/+bug/2175114/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs