Hi Zack,
I'm reporting a memory issue in vmwgfx where graphics memory is not
returned to the system even after every DRM client has released its GEM
objects. Only a reboot frees it. This comes out of KDE bug 523812, which
was closed as RESOLVED UPSTREAM with the assessment that this looks like
a driver-side leak rather than a compositor one.
- Environment:
Guest: CachyOS (Arch-based), kernel 7.1.5 and 7.1.6
Host: VMware Workstation on Windows, AMD Ryzen 9 7950X
Driver: vmwgfx 2.21.0.0, VMware SVGA II
Renderer: llvmpipe (LIBGL_ALWAYS_SOFTWARE=1, see note below)
Compositor: KWin 6.7.3, Plasma Wayland, 3 x 1920x1080 @ 60 Hz
RAM: 31 GiB, zram swap
3D accel: disabled in the VM settings
- What happens:
During normal desktop use, buffers of exactly 8355840 bytes accumulate.
That is 1920 x 1088 x 4 — one screen-sized 32-bit framebuffer with the
height padded to a multiple of 64. Growth is tied to rendering activity,
not to uptime: an unattended session leaked exactly zero bytes over 7.3
hours, while active use adds roughly 1.4 GB/hour.
At 9 hours of uptime, kwin_wayland held 1846 GEM objects, 1553 of them
that exact size. All of them had pin_count = 0. At 34.5 hours it was
8057 objects.
- The measurement that points at the driver:
I logged out of the Plasma session and logged back in, without
rebooting. That restarts the compositor (pid 2612 -> 3156650) and closes
its DRM fd, so the kernel drops all its GEM references.
before logout after re-login
GEM objects, all clients 8139 19
system_ttm usage 49.47 GB 45.80 GB
GPUActive (/proc/meminfo) 10.86 GB 11.06 GB
MemAvailable 11.06 GB 13.32 GB
So: no DRM client held more than 19 GEM objects in total, yet roughly 11
GB remained accounted as GPUActive and was not returned. The ~2.3 GB
gained in MemAvailable is very likely just the browsers and editors from
the old session exiting, not graphics memory.
A reboot immediately afterwards did return it:
GPUActive 0.11 GB
RAM in use 4.5 GB of 31 GB (was 18 GB before the reboot)
- Caveats I want to be upfront avout:
- system_ttm reporting 46-49 GB on a machine with 31 GiB of RAM is
something I cannot explain. My assumption is that TTM accounts pages it
has evicted to shmem, while GPUActive counts only resident ones. If that
assumption is wrong, the volume figures need reinterpreting. The object
counts and the fact that a reboot is required stand either way.
- Summing the reported sizes of the GEM objects gives ~46 GB, which also
exceeds physical RAM, so that sum over-counts — presumably shared or
multiply-mapped objects. I would trust the counts, not that total.
- This is a CachyOS kernel, not vanilla. I have not yet reproduced on
mainline. I am happy to test a vanilla or drm-tip kernel if that would
help; the machine reproduces this within a working day.
- LIBGL_ALWAYS_SOFTWARE=1 is set here to work around an older, separate
vmwgfx surface-handle problem that crashed Chromium/Electron compositor
threads with SIGILL. The behaviour described above occurs with that
workaround in place.
- Possibly related earlier work:
I am aware of the dumb-buffer leak fixes in vmwgfx from the 6.10 era
("drm/vmwgfx: Fix handling of dumb buffers", "drm/vmwgfx: Fix dumb
buffer leak") and the rework that made GEM handles own the dumb buffer.
The buffers accumulating here are exactly screen-sized dumb buffers, so
this may well be in the same area, but on a much newer kernel, so either
something remains or it is a regression. I did not find an open report
matching this.
- How to reproduce the measurements:
# object count and per-client totals (root)
grep -c bytes /sys/kernel/debug/dri/0/vmwgfx_gem_info
awk '/^ *pid/{p=$2" "$4} /bytes/{n[p]++; s[p]+=$2} \
END{for(k in n) printf "%-26s %6d objects %9.1f MB\n", k,
n[k], s[k]/1048576}' \
/sys/kernel/debug/dri/0/vmwgfx_gem_info | sort -k3 -rn
# pool usage (root)
awk '/usage:/{print $2}' /sys/kernel/debug/dri/0/system_ttm
# resident graphics memory
grep -E 'GPUActive|GPUReclaim' /proc/meminfo
Use the desktop normally for a few hours; a browser window with a
looping CSS animation makes it measurable within minutes. Opening and
closing 32 windows produced no growth at all, so it is the continuous
rendering that drives it.
Happy to run anything specific, apply a debug patch, or test a different
kernel.
Regards,
Jellis Onsea
AccessWeb
