AMD General > -----Original Message----- > From: Natalie Vock <[email protected]> > Sent: Thursday, August 20, 2026 3:29 PM > To: SHANMUGAM, SRINIVASAN <[email protected]>; > Koenig, Christian <[email protected]>; Deucher, Alexander > <[email protected]> > Cc: [email protected]; Kuehling, Felix <[email protected]>; > Zhu, James <[email protected]>; Lazar, Lijo <[email protected]>; Six, > Lancelot <[email protected]>; Pelloux-Prayer, Pierre-Eric <Pierre- > [email protected]>; Timur Kristóf <[email protected]>; Samuel > Pitoiset <[email protected]> > Subject: Re: [RFC PATCH 1/3] drm/amdgpu/uapi: Add second-level trap handler > ops to VM ioctl > > Hi, > > first of all: Thanks for working on this! It's great seeing trap handler > support come > together. > > On 8/20/26 09:01, Srinivasan Shanmugam wrote: > > When a GPU shader hits an exception, memory fault, or debug > > breakpoint, the hardware jumps to the first-level trap handler. The > > first-level handler (managed by the kernel via CWSR) checks the TMA > > buffer for a second-level handler address. If one is installed, it > > forwards the trap to that userspace handler, allowing the runtime or > > debugger to handle shader exceptions without modifying the kernel trap > > handler. > > > > KFD already supports this for compute workloads. Render-node user > > queues had no equivalent mechanism. Add it. > > > > The second-level handler is a per-VM setting — it applies to all > > shader waves executing under that VMID regardless of queue type. GFX > > and compute queues from the same process share the same VMID, so one > > SET_L2_TRAP call covers all queue types for that process. This > > configuration is not CWSR-specific; CWSR is only the first-level > > handler mechanism. The correct home for this setting is the VM ioctl > > (DRM_AMDGPU_VM), following the same pattern as > > AMDGPU_VM_OP_RESERVE_VMID. > > > > Add two new VM ioctl operations: > > AMDGPU_VM_OP_SET_L2_TRAP (op = 3) — install second-level handler > > AMDGPU_VM_OP_CLEAR_L2_TRAP (op = 4) — remove second-level > handler > > > > Extend drm_amdgpu_vm_in with a 32-byte union for op-specific data. The > > l2trap member carries the GPU virtual addresses and sizes of the TBA > > (handler code) and TMA (handler scratch memory). > > This should be a BO handle and offset+size, instead. The BOs associated with > the > TBA/TMA must be tracked as used by every submission from the VM that has this > trap handler installed, otherwise you introduce a ton of race conditions. Off > the top > of my head, here are a few: > 1. The GEM VA ioctl can spuriously fail to actually update page tables. > This is okay and intentional, and BOs with outdated page tables will > be updated on the next submit if they're used by the submission. If > the TBA/TMA BOs aren't marked in the set of used buffers, the PTs may > end up never being updated and subsequent accesses will fault. > 2. The TBA/TMA may be evicted/moved around concurrently with executing > submissions if these submissions didn't add their fences to the > TBA/TMA resv, which would likely randomly corrupt things or hang. > > A simpler solution could be requiring the TBA/TMA buffers to be > VM_ALWAYS_VALID, in which case synchronization to all submissions in the VM > is taken care of automagically. This prevents exporting the TBA/TMA to an fd, > but I > don't expect anyone would want to do this.
Hi Natalie, Thanks for the feedback. We will update the UAPI in v2 to pass BO handles alongside the GPU VA for TBA/TMA. This ensures the kernel can properly track and pin the buffers. Thanks, Srini > > Regards, > Natalie
