Public bug reported:

Summary
Playing a multi-molecule trajectory movie in VMD (Visual Molecular
Dynamics) with the default iris driver makes system Shmem grow from
~1 GB to ~28-29 GB within minutes until the kernel OOM-killer fires.
The memory is not in VMD's heap: VMD RSS stays ~600 MB, and no process
shows RssShmem. With LIBGL_ALWAYS_SOFTWARE=1 or
MESA_LOADER_DRIVER_OVERRIDE=zink the same movie plays to the end
(500 ns, 5 trajectories) with Shmem flat at ~0.6-1.1 GB.

System
- Ubuntu 26.04.1 LTS, GNOME on Wayland (VMD runs via XWayland/GLX)
- Mesa 26.0.8-1ubuntu0.3 (libgl1-mesa-dri, mesa-libgallium)
- CPU/GPU: Alder Lake-S GT1 [UHD Graphics 770, 8086:4690], i915
  (a second ASPEED AST2600 VGA device is only the BMC display)
- 32 GB RAM, 61 GB swap, glxinfo: "Unified memory: yes"
- Kernel 7.0.0-42-generic; an older kernel [version: 7.0.0-43-generic] crashed 
the
  same way, so this does not look kernel-specific
- VMD [version: 1.9.4a57], [binary from UIUC website download, Linux 64-bit 
OpenGL build]

Steps to reproduce
1. Launch vmd with no overrides.
2. Load 5 trajectories [system size ~2300 atoms, ~10000 frames each].
3. Play the animation.

Observed
- 23:24:30  Shmem 1.19 GB, VMD RSS 0.62 GB
- 23:29:58  Shmem 29.2 GB (OOM report: shmem:29190736kB,
  shmem_thp 5-12 GB); GNOME Shell, Chrome and VMD OOM-killed.
- The kernel log shows "Purging GPU memory, 0 pages freed" just
  before the kill, i.e. the i915 shrinker found nothing to reclaim.
- Same pattern on 2 separate runs (and 1 earlier crash).

Not affected
- LIBGL_ALWAYS_SOFTWARE=1: full playback, Shmem ~0.6 GB
- MESA_LOADER_DRIVER_OVERRIDE=zink: full 500 ns playback,
  Shmem ~1.1 GB
- Idle desktop: Shmem 0.64 GB after 12 h uptime with VMD closed

Possibly related
- Fedora 44 report of the same symptom with VMD on an Intel UHD 630
  (CML GT2), Mesa iris, Wayland. zink stops the growth there too
  (with pixelated points). Fedora 42 on the same machine was fine:
  https://discussion.fedoraproject.org/t/198655
- Launchpad bug 2143073 (Intel shmem growth; Mesa fix regarding
  throttling of uploads for Gallium drivers without a memory budget)
- xemu issue #3083 (Intel iGPU, ~36 GB unmapped shmem)

Hypothesis (unverified)
VMD re-uploads geometry every frame. iris appears to queue these
buffers without limit, so they accumulate as shmem-backed GEM objects.

ProblemType: Bug
DistroRelease: Ubuntu 26.04
Package: libgl1-mesa-dri 26.0.8-1ubuntu0.3
ProcVersionSignature: Ubuntu 7.0.0-42.42-generic 7.0.14
Uname: Linux 7.0.0-42-generic x86_64
ApportVersion: 2.34.1-0ubuntu0.1
Architecture: amd64
BootLog: Error: [Errno 13] Permission denied: '/var/log/boot.log'
CasperMD5CheckResult: pass
CompositorRunning: None
CurrentDesktop: ubuntu:GNOME
Date: Tue Nov 17 10:20:24 2026
DistUpgraded: 2026-10-31 22:22:14,602 WARNING No regular users with home 
directories found
DistroCodename: resolute
DistroVariant: ubuntu
ExtraDebuggingInterest: Yes, if not too technical
GraphicsCard:
 Intel Corporation Alder Lake-S GT1 [UHD Graphics 770] [8086:4690] (rev 0c) 
(prog-if 00 [VGA controller])
   Subsystem: Super Micro Computer Inc Device [15d9:1c48]
 ASPEED Technology, Inc. ASPEED Graphics Family [1a03:2000] (rev 52) (prog-if 
00 [VGA controller])
   Subsystem: ASPEED Technology, Inc. ASPEED Graphics Family [1a03:2000]
InstallationDate: Installed on 2024-03-27 (965 days ago)
InstallationMedia: Ubuntu 22.04.2 LTS "Jammy Jellyfish" - Release amd64 
(20230223)
MachineType: Supermicro Super Server
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-7.0.0-42-generic 
root=UUID=10d126ef-88b7-41be-a4de-ea0f24300189 ro quiet splash 
crashkernel=2G-4G:320M,4G-32G:512M,32G-64G:1024M,64G-128G:2048M,128G-:4096M
SourcePackage: mesa
UpgradeStatus: Upgraded to resolute on 2026-10-31 (16 days ago)
dmi.bios.date: 02/17/2023
dmi.bios.release: 5.27
dmi.bios.vendor: American Megatrends International, LLC.
dmi.bios.version: 2.0a
dmi.board.asset.tag: Default string
dmi.board.name: X13SAE-F
dmi.board.vendor: Supermicro
dmi.board.version: 1.02A
dmi.chassis.asset.tag: Chassis Asset Tag
dmi.chassis.type: 3
dmi.chassis.vendor: Supermicro
dmi.chassis.version: 0123456789
dmi.modalias: 
dmi:bvnAmericanMegatrendsInternational,LLC.:bvr2.0a:bd02/17/2023:br5.27:svnSupermicro:pnSuperServer:pvr0123456789:rvnSupermicro:rnX13SAE-F:rvr1.02A:cvnSupermicro:ct3:cvr0123456789:skuDefaultstring:pfaFamily:
dmi.product.family: Family
dmi.product.name: Super Server
dmi.product.sku: Default string
dmi.product.version: 0123456789
dmi.sys.vendor: Supermicro
version.compiz: compiz N/A
version.libdrm2: libdrm2 2.4.131-1
version.libgl1-mesa-dri: libgl1-mesa-dri 26.0.8-1ubuntu0.3
version.libgl1-mesa-glx: libgl1-mesa-glx N/A
version.xserver-xorg-core: xserver-xorg-core 2:21.1.22-1ubuntu1.3
version.xserver-xorg-input-evdev: xserver-xorg-input-evdev N/A
version.xserver-xorg-video-ati: xserver-xorg-video-ati 1:22.0.0-1build2
version.xserver-xorg-video-intel: xserver-xorg-video-intel 
2:2.99.917+git20210115-1build2
version.xserver-xorg-video-nouveau: xserver-xorg-video-nouveau 1:1.0.18-1build1

** Affects: mesa (Ubuntu)
     Importance: Undecided
         Status: New


** Tags: amd64 apport-bug resolute ubuntu wayland-session

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2174691

Title:
  VMD crashing with RAM overflow

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/mesa/+bug/2174691/+subscriptions


-- 
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Reply via email to