On Tue, Aug 11, 2026 at 07:09:33PM -0700, Matthew Brost wrote: > On Sat, Aug 08, 2026 at 05:34:04PM +0800, Neil Zhong wrote: > > On Fri, Jul 31, 2026 at 11:00:49PM -0700, Matthew Brost wrote: > > > This actually roughly what downstream customers are carrying :). > > > > > > It is basically these 3 patches [9] [10] [11] implemented directly in > > > the Xe shrinker code to avoid touching the MM or TTM... > > > > > > Alas I got nack'd by someone outside my subsystem on this approach. > > > > > > I think if you use the reference patches above to implement a heuristic > > > in Xe, or come up with a similar one, it will likely solve this issue. > > > > > > Since you're on 6.18, you may also be missing some Xe/TTM changes > > > related to this problem that have already been merged into drm-tip. > > > There are a couple of one-line fixes that should help somewhat, but the > > > heuristic is what I think will actually address the root cause. > > > > > > As heads up, I've started looking at this again and pushing to get > > > something upstream as this at least 5th time someone or org has flagged > > > this as a problem. Any data you can provide will help us push towards a > > > solution. > > > > Hi Matt, > > > > Thanks for all the details here, very helpful. > > > Thanks. I tested [9]-[13] on the same machine and the fragmentation > > heuristic does reduce the frequency of the problem. However, after these > > tests I would like to clarify my actual requirement, since my previous > > watermark-based proposal did not express it correctly. > > > > For BOs that belong to a latency-critical visual processing working set, > > I think userspace should be able to mark them as non-shrinkable, and Xe > > should not back them up under any memory-reclaim condition while that mark > > is held. This would be a hard residency contract, not another reclaim > > priority or a fragmentation hint. > > This is roughly what customers have indicated to us for laptop-type > products: anything displayed on the screen should avoid eviction or > shrinking at all costs. This series came out of that discussion: > > https://patchwork.freedesktop.org/series/170454/ > > This customer, in particular, utilizes priority bands to express this > heuristic (e.g., the compositor is the highest priority, any > non-privileged UI-related content is normal priority, and everything > else is low priority). I'm not sure if stock distros do anything like > this. > > Pinning would take this even further, allowing the compositor (or anyone > really) to effectively say, "Don't shrink this". > > > > > Why a hard contract is useful for visual workloads > > -------------------------------------------------- > > > > The reproducer is continuous 4K60 HDR playback. Every decoded frame is > > Can you give me instructions on how to recreate this on our end and your > machine, memory details? I have a bunch of various reproducers which I > have been using for shrinker work and the more the better. > > > imported through DMA-BUF and processed by libplacebo/OpenGL for HDR tone > > mapping before presentation. The active BO set contains decoded video > > surfaces, intermediate render targets and presentation-related surfaces. > > > > DMA-BUF in priority series moves to the prior band that is least likely > to be shrunk. > > > At 60 Hz the complete frame interval is 16.667 ms. If glFlush() is blocked > > for longer than this, the application misses at least one presentation > > deadline. Repeated missed deadlines are perceived directly as dropped > > frames or visible stutter. A 100-500 ms reclaim/restore storm is an > > obvious freeze, even if the system eventually recovers and its average > > throughput looks normal. > > > > Fence-idle is not equivalent to cold for this workload. A video surface > > can have no active fence in the small gap between two frames and still be > > part of the application's current visual working set. The next frame can > > need the same BO immediately. > > > > Ok, I think I see a potential problem here with priorities. If, for > example, a buffer is assigned a priority indicating that it is unlikely > to be evicted but has no active fences, it could be chosen for shrinking > before buffers whose priorities indicate "shrink this first" if those > buffers have active fences. > > > The trace demonstrates exactly this case. In one sequence, kswapd > > completed backup of an 8,208-page BO and the rendering thread started > > restoring the exact same ttm_tt about 33 microseconds later. The kernel > > copied about 32 MiB to shmem, dropped the WC pages, then immediately had > > to allocate pages, copy the data back and reapply WC. > > > > In the ten-minute default-watermark capture: > > > > successful ttm_tt_backup: 4,853 > > ttm_tt_restore: 4,810 > > minimum backup+restore copy traffic: 50,436.105 MiB > > Flush > 16.667 ms: 209 > > Flush > 100 ms: 65 > > maximum trace-aligned Flush: 223.195 ms > > Also a quick write up how you extracted these numbers from reproducer so > I can recreate on my end. > > > > > Of 4,791 restores matched to the same preceding backup, 4,279 happened > > within 100 ms and 4,753 within one second. kswapd0 performed 4,839 of the > > 4,853 backups, while the player's rendering thread performed most of the > > restores. Of the 209 Flush calls over one frame interval, 205 contained > > ttm_tt_restore() and all 209 contained set_pages_array_wc(). > > > > This is not useful recovery of cold memory. It is destruction and > > immediate reconstruction of the visible working set. > > > > Yes, indeed. We really don't want to destroy a working set unless the > core system genuinely needs memory and doing so is the only option. > Even then, there may be parts of the working set that simply cannot be > shrunk, as you are suggesting. > > > What the heuristic test showed > > ------------------------------ > > > > I backported [9]-[13], extended the fragmentation check to direct reclaim, > > and tested a `high + min` watermark threshold. In a follow-up run with the > > same kernel, video and playback configuration, I set vm.min_free_kbytes to > > 50000. This lowered the Normal-zone `high + min` threshold from about > > 379.8 MiB to 255.1 MiB. > > > > With the lower threshold, the fragmentation helper returned true more > > often and Xe working-set churn fell by about 90%: > > > > successful ttm_tt_backup: 4,853 -> 443 > > ttm_tt_restore: 4,810 -> 435 > > Flush > 16.667 ms: 209 -> 19 > > Flush > 100 ms: 65 -> 3 > > > > This confirms that preventing working-set backup prevents the visual > > stalls. It does not make the individual restore path cheaper. When the > > heuristic still allowed a storm, the maximum Flush was 191.655 ms and > > contained 25 restores and 23 WC conversions. > > > > I do not think tuning global watermarks is the right solution. More > > Nor do I. [9]-[13] were Xe replacement for what is IMO a proper solution > in the core MM: https://patchwork.freedesktop.org/series/168651/ I'm > pushing on this patch a bit more with the core MM maintainers and have > another shrinker locally that is semi-related to this as well. > > I guess I'd like numbers with the patch above + priority bands to see if > that is enough prevent working set shrinking of valuable buffers + > spikes in flush times. > > > importantly, I no longer think that protection for explicitly identified > > visual BOs should depend on whether reclaim was caused by fragmentation > > or genuine low memory. Once userspace has declared a bounded set as > > presentation-critical, violating that residency guarantee produces an > > immediate and user-visible failure. > > > > To be clear - this would be an addition to fixes discussed above, right? > > > Possible explicit marking mechanism > > ----------------------------------- > > > > Could Xe provide an opt-in, mlock-like mechanism for this purpose? > > > > Yes, we could implement something like this, but we'd need buy-in across > the entire stack (i.e., from user space as well). I'll run this by the > internal team too to see if anyone can immediately poke holes in it, > because I don't currently see any obvious issues. > > > One possible interface would be a new DRM_IOCTL_XE_MADVISE VMA attribute, > > for example DRM_XE_VMA_ATTR_RECLAIM_POLICY, with states similar to: > > > > DRM_XE_VMA_RECLAIM_DEFAULT > > DRM_XE_VMA_RECLAIM_NO_SHRINK > > This seems like a reasonable API. > > > > > NO_SHRINK would mean that the backing BO is excluded from the Xe shrinker > > while at least one protected VMA holds the attribute. Userspace would set > > it when a video/render surface enters the active visual pipeline and clear > > it after the surface leaves that working set. Unbind, VM destruction or > > file close would also release the holder automatically. > > > > For a BO shared by multiple VMAs, Xe could maintain a BO-level > > no_shrink_count, similar to the holder accounting already used for > > purgeable state. The shrinker would skip a BO with a non-zero count. I > > would prefer a separate shrinker-specific count rather than exposing TTM > > pin_count, because pinning also affects placement and migration, which is > > broader than the requested guarantee. > > > > The existing WILLNEED state does not provide this contract: it prevents > > purging of the contents, but the non-purge shrinker may still back up and > > unpopulate the BO. It also cannot simply be redefined because WILLNEED is > > the default state for all VMAs. SCANOUT is not sufficient either, since > > many HDR intermediate and imported video surfaces are not scanout BOs. > > > > I understand that an unprivileged client must not be allowed to make an > > unbounded amount of memory unreclaimable. Like mlock, this could be > > controlled by an explicit per-file, per-client or cgroup byte limit, and > > I think we could just hook into mlock accounting. There is an exported > function for exactly this purpose: > > https://elixir.bootlin.com/linux/v7.1.7/source/mm/util.c#L549 >
I guess using mlock accounting for pinning has been discussed in the past and was ultimately rejected because it is susceptible to fork-bomb attacks, which can result in all SRAM being pinned. Thomas has a write-up with more details that he can perhaps share, but I think the community direction of a pinning uAPI is reasonable. However, we likely need cgroup-based pinning limits. Dave has a series implementing cgroups for SRAM here [1], and we'd likely need to extend this to support pinning limits as well. Likewise, the VRAM controller would also need pinning limits. Matt [1] https://patchwork.freedesktop.org/series/169824/ > You'd have to deal with multiple VMAs (from the same or different MMs in > a dma-buf) aliasing the same BO and ensure that accounting remains > consistent everywhere, but it shouldn't be too difficult. We already > have this problem WILLNEED/WONTNEED and solved it. > > Ofc, this only works for system memory buffers so we'd some VRAM type > accounting too. iirc Thomas was working on cgroups for that part in a > slightly different context though. > > > possibly by a privilege check. If the requested protected set exceeds the > > configured limit, the madvise should fail rather than silently accepting > > the mark and later violating it under pressure. The application or system > > service would then decide which visual surfaces to protect or release. > > > > Within that bounded contract, however, I think NO_SHRINK should remain a > > hard guarantee even in genuine low-memory reclaim. Under pressure the > > kernel may reclaim unmarked BOs and other memory, reject additional > > NO_SHRINK requests, or require the application/service to release part of > > its protected set. Backing up an already accepted presentation-critical > > BO and stalling a frame by hundreds of milliseconds defeats the purpose > > of the interface. > > > > For comparison, my original workaround approximated such a hard contract > > by excluding VM-bound WC BOs from non-purge shrinking. In a 21-minute > > capture, no XE_EXEC, VM_BIND or DMA-BUF ioctl exceeded the 16.7 ms frame > > interval; their maxima were 348 us, 136 us and 20 us. The call rate of > > set_pages_array_wc fell to 0.179/s. That automatic VM-bound WC rule is too > > broad, but an explicit and bounded userspace mark could provide the same > > latency guarantee only for the BOs that actually need it. > > > > Would an explicit, bounded NO_SHRINK/latency-critical VMA attribute be a > > reasonable Xe UAPI direction? If so, I can prototype the BO holder > > accounting and shrinker exclusion, then modify the video/Mesa path to mark > > only the active visual working set and collect another strict A/B trace. > > No issue if you want to prototype this, but as mentioned above, this > would require buy-in from user space (which is not under my control) and > at least one other person on the KMD team (most likely Thomas). So I > can't guarantee that it won't be rejected by someone. > > Matt > > > [9] https://patchwork.freedesktop.org/patch/720030/?series=165329&rev=1 > > [10] https://patchwork.freedesktop.org/patch/720036/?series=165329&rev=1 > > [11] https://patchwork.freedesktop.org/patch/720031/?series=165329&rev=1 > > [12] https://patchwork.freedesktop.org/patch/732698/?series=168466&rev=1 > > [13] https://patchwork.freedesktop.org/patch/732420/?series=168389&rev=1 > > > > Thanks, > > Neil
