https://bugs.kde.org/show_bug.cgi?id=523787
Bug ID: 523787
Summary: krdpserver SIGSEGV in FFmpeg vf_scale from KPipeWire
Encoder::encodeFrame during RDP reconnect
Classification: Frameworks and Libraries
Product: KPipeWire
Version First 6.7.3
Reported In:
Platform: Fedora RPMs
OS: Linux
Status: REPORTED
Severity: crash
Priority: NOR
Component: general
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected]
Target Milestone: ---
STEPS TO REPRODUCE
- Start KDE Plasma on Wayland and enable Remote Desktop/KRDP.
- Connect to the machine using an RDP client.
- Disconnect and reconnect several times in quick succession.
In my reproduction:
- The first two connections displayed the desktop correctly.
- A subsequent connection was accepted, and mouse movement/input worked, but
the client displayed a black window.
- krdpserver later crashed with SIGSEGV.
The problem is intermittent, but repeated disconnect/reconnect attempts trigger
it.
RDP client: Windows App Version 11.3.8 (3054) - macOS
Connection resolution: native 4480×2520 display
OBSERVED RESULT
On a failed reconnect, the RDP connection remains partly functional: the
pointer moves and input reaches the remote Plasma session, but no desktop image
is displayed.
krdpserver subsequently crashed with SIGSEGV.
coredumpctl recorded:
Signal: 11 (SEGV)
Executable: /usr/bin/krdpserver
User Unit: app-org.kde.krdpserver.service
A fully symbolized core dump shows the following call path:
scale_eval_dimensions() at libavfilter/vf_scale.c:560
config_props() at libavfilter/vf_scale.c:633
scale_frame() at libavfilter/vf_scale.c:812
do_scale() at libavfilter/vf_scale.c:950
ff_framesync_activate()
ff_filter_graph_run_once()
get_frame_internal()
Encoder::encodeFrame() at kpipewire/src/encoder.cpp:91
PipeWireProduce::setupStream() encoder thread
at kpipewire/src/pipewireproduce.cpp:179
At the fault site, GDB shows:
desc = 0x0
inlink = 0xfffefc046460
out_desc = 0xffff420e0818
The faulting AArch64 instruction is:
ldr b30, [x26, #9]
with:
x26 = 0x0
This is a NULL dereference of the input AVPixFmtDescriptor while evaluating the
scale-filter dimensions. The instruction reads the descriptor field at offset
9, corresponding to log2_chroma_w.
The input frame itself is non-NULL in scale_frame():
in = 0xfffeb01dbc70
The NULL in visible in the caller do_scale() is expected after scale_frame()
takes ownership of the caller's frame pointer; it is not evidence that the
graph was activated without an input frame.
The crash occurs in FFmpeg's dynamic frame-change/reconfiguration path.
Immediately before config_props() is called, scale_frame() updates the filter
link using the incoming frame's format, dimensions and colour metadata.
The trace therefore suggests that the frame or link contained an invalid, unset
or corrupted pixel-format value at that moment. It does not establish where
that value originated. Possible areas for investigation include PipeWire stream
renegotiation, encoder/filter-graph lifetime during reconnect, and validation
of AVFrame::format before it is submitted to the FFmpeg graph.
EXPECTED RESULT
Repeated RDP disconnects and reconnects should either:
- successfully restart the desktop capture and encoding pipeline; or
fail cleanly with an error.
- krdpserver should not display a permanently black stream or crash because the
scale filter received invalid or inconsistent pixel-format metadata.
SOFTWARE/OS VERSIONS
Operating system: Fedora Asahi Remix 44
Architecture: aarch64
Hardware: Apple M1 iMac, 24-inch
Display server: Wayland
kpipewire-6.7.3-1.fc44.aarch64
krdp-6.7.3-1.fc44.aarch64
plasma-workspace-6.7.3-1.fc44.aarch64
ffmpeg-libs-8.1.2-2.fc44.aarch64 (RPM Fusion)
x264-libs-0.165-5.20250608gitb35605ac.fc44.aarch64
kernel 7.1.5-400.asahi.fc44.aarch64+16k
The FFmpeg runtime and debuginfo packages were verified to be from the exact
same build:
ffmpeg-libs-8.1.2-2.fc44.aarch64
ffmpeg-libs-debuginfo-8.1.2-2.fc44.aarch64
ffmpeg-debuginfo-8.1.2-2.fc44.aarch64
ffmpeg-debugsource-8.1.2-2.fc44.aarch64
The libavfilter runtime build ID also resolves to the matching RPM Fusion debug
file:
Build ID: ac05e83612b1bc7c8675b64ad7a887e5200143f5
GDB reports:
Syms Read: Yes
/lib64/libavfilter.so.11
RELATED ENCODER OBSERVATION
This is included as related environment information and is not claimed to be
the direct cause of the NULL-descriptor crash.
An attempted OpenH264 encoder path logged:
[libopenh264] Error:ParamValidationExt(), width > 0, height > 0,
width * height <= 9437184, invalid 4480 x 2528 in dependency
layer settings!
The x264 path also logged:
MB rate (44240000) > level limit (16711680)
At the configured 1/1000 codec time base, this corresponds to a 280×158 coded
macroblock grid. That grid is consistent with a 4480×2520 source encoded in a
macroblock-aligned 4480×2528 coded area. The x264 warning alone does not
distinguish an explicitly configured height of 2528 from normal macroblock
padding of a 2520 source.
WORKAROUND
Restarting the KRDP user service restores operation temporarily:
systemctl --user restart app-org.kde.krdpserver.service
Fully closing the RDP client before reconnecting also appears to reduce the
likelihood of the failure.
ADDITIONAL NOTES
The immediate segmentation fault occurs in FFmpeg's scale filter, but the
reproducing pipeline is constructed and driven through KPipeWire's
Encoder::encodeFrame(). I am filing this under KPipeWire so the incoming
frame-format and reconnect/renegotiation lifecycle can be investigated.
The core proves:
-A real, non-NULL frame reached scale_frame().
- Dynamic filter reconfiguration occurred.
- av_pix_fmt_desc_get(inlink->format) produced a NULL descriptor.
- FFmpeg dereferenced that descriptor and crashed.
The core strongly suggests invalid or corrupted pixel-format metadata, but it
does not yet prove which component created or damaged that value, nor whether
every black-screen-only occurrence has exactly the same root cause.
I can provide the exact pixel-format enum extracted from the core in a
follow-up comment.
ATTACHMENTS
I will attach:
- krdpserver-2363-ffmpeg-symbolized.txt
- krdpserver-2363-coredump-info.txt
- KRDP/KPipeWire journal around the failure
- process/memory timeline from the repeated connection attempts
I have removed or will remove the machine ID, boot ID, hostname and other
private information from publicly uploaded logs.
--
You are receiving this mail because:
You are watching all bug changes.
