Thank you for the patch! I applied it to my 7.0-rc3 tree. To be absolutely certain that the new module was loaded and the modified code path was being executed, I added a custom DRM_INFO printk right next to your if (surface->res.dirty == NULL) check. I can confirm the code is definitely being hit during Wayland usage.
Unfortunately, the severe slab leak still persists. When moving windows, radix_tree_node and the 1KB kmalloc caches (kmalloc-rnd-*-1k) continue to grow rapidly per frame, eventually leading to OOM. Just to double-check, setting KWIN_DRM_NO_AMS=1 still completely stops the leak, so the missing reference drop (perhaps a missing drm_framebuffer_put or GEM unreference) seems to be hiding somewhere else in the atomic commit/cleanup path of vmwgfx. I am ready to test any further patches you might have. Let me know if you need me to enable specific drm debug logs or use ftrace to track down the exact leaking objects. Best regards, Yuma 2026年3月13日(金) 21:56 Ian Forbes <[email protected]>: > > Can you retry with > https://patchwork.kernel.org/project/dri-devel/patch/[email protected]/ > appliecd to your kernel? > > Thank you, > Ian > > On Thu, Mar 12, 2026 at 11:48 AM Yuma Kakei <[email protected]> wrote: >> >> Hi vmwgfx maintainers and DRM developers, >> >> I have encountered a severe Slab memory leak in the vmwgfx driver when using >> a Wayland compositor (KWin) with Atomic KMS enabled. This issue eventually >> leads to complete memory exhaustion and OOM within a few minutes of normal >> desktop usage. >> >> [Environment] >> Kernel: 7.0-rc3 (Mainline) and 6.17.0-14 (HWE kernel) >> Distro: KDE neon Unstable (Ubuntu 24.04 LTS base) >> Compositor: KWin Wayland (Latest master branch) >> Guest Graphics: VMware SVGA II Adapter (vmwgfx) >> >> [Symptoms & Evidence] >> When moving windows or triggering screen updates on the Wayland session, >> system memory is rapidly consumed. >> Monitoring with slabtop reveals that radix_tree_node and kmalloc-rnd-07-1k >> objects are leaking aggressively per frame. >> >> It appears that DRM objects (likely GEM handles/IDR or framebuffers) created >> during the Atomic KMS commit are not being properly freed by the driver >> after the frame is rendered, leaving orphaned objects in the kernel Slab. >> >> Earlier, before disabling 3D acceleration, dmesg also showed: >> vmwgfx: mob memory overflow. Consider increasing guest RAM and >> graphicsMemory. >> However, even with 3D acceleration disabled (llvmpipe fallback), the Slab >> memory leak persists as long as Atomic KMS is used. >> >> [Steps to Reproduce] >> 1. Boot a VMware guest with vmwgfx and launch a modern Wayland compositor >> (KWin) that strictly uses Atomic KMS. >> 2. Move windows around to force screen redraws. >> 3. Run sudo slabtop -s c and observe radix_tree_node and kmalloc-rnd-07-1k >> growing indefinitely. >> >> [Workaround] >> Disabling Atomic KMS in the compositor completely stops the memory leak. >> Setting the environment variable KWIN_DRM_NO_AMS=1 forces KWin to use legacy >> DRM API, which works perfectly without any Slab growth. This strongly >> suggests the bug resides in the vmwgfx atomic commit/cleanup path. >> >> Any assistance or guidance on testing potential patches would be greatly >> appreciated. I am happy to provide further logs or test patches if needed. >> >> Best regards, >> Yuma Kakei
