Background
----------
When a GPU shader crashes or hits an exception, the hardware
automatically jumps to a trap handler. There are two levels:
First-level — always active, managed by the kernel (CWSR)
Second-level — optional, installed by userspace (RADV, ROCm)
The second-level handler lets userspace debuggers catch GPU crashes
and inspect what went wrong.
Problem
-------
The existing second-level trap handler support (merged earlier) only
works for user queues. RADV on GFX9 and GFX10 hardware (Vega, Navi,
Steam Deck) uses kernel queues — not user queues — because user queues
are not supported on those GPUs. This means RADV has no way to install
a second-level trap handler for shader debugging on those platforms.
Root cause: the SQ_SHADER_TMA register for kernel queue VMIDs was
never programmed by the driver, so there was nowhere to store the
second-level handler address.
Solution
--------
This series fixes the problem in three steps:
Patch 1 — Infrastructure
- Allocate a small per-VM TMA buffer (kq_tma_bo) at VM creation time,
the same way page tables are allocated.
- Map it read-only into the GPU VM at a fixed address
(AMDGPU_VA_RESERVED_TRAP_START).
- Program SQ_SHADER_TMA for all kernel queue VMIDs to point to this
fixed address. Because each VM has its own buffer at the same
address, per-process isolation comes from the page tables — not
from reprogramming the register per job.
Patch 2 — Hardware register programming (GFX10/11/12)
- Implement the actual register writes (SQ_SHADER_TBA/TMA) for all
supported GPU generations via SRBM select.
- GFX10 uses mm-prefixed registers, GFX11/12 use reg-prefixed.
- GFX12.1 has multiple XCC instances and iterates over all of them.
- Called at boot and on GPU resume.
Patch 3 — Wire up SET_L2_TRAP and CLEAR_L2_TRAP
- When userspace calls SET_L2_TRAP, write the second-level handler
address into both the user queue TMA and the new kernel queue TMA.
- When userspace calls CLEAR_L2_TRAP, zero both.
- No new ioctl needed — RADV uses the same SET_L2_TRAP ioctl that
user queues already use.
Design decisions
----------------
- kq_tma_bo is per-VM, not device-level. One BO per process,
same model as page tables.
- Fixed VA (AMDGPU_VA_RESERVED_TRAP_START) in every VM's address
space. Same register value for all VMIDs — isolation via page tables.
- GPU sees it as read-only. Kernel CPU updates it via CPU mapping
when SET_L2_TRAP is called.
- Lifecycle matches page tables: allocated at VM init, freed at
VM fini.
Only compilation tested.
Srinivasan Shanmugam (3):
drm/amdgpu: Add per-VM kernel queue first-level trap handler
infrastructure
drm/amdgpu: Implement kernel VMID SQ_SHADER_TBA/TMA programming for
GFX10/11/12
drm/amdgpu: Extend second-level trap handler to kernel queues
drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_trap.c | 107 ++++++++++++++++++++
drivers/gpu/drm/amd/amdgpu/amdgpu_trap.h | 7 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 9 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 13 +++
drivers/gpu/drm/amd/amdgpu/gfxhub_v11_5_0.c | 38 +++++++
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_0.c | 38 +++++++
drivers/gpu/drm/amd/amdgpu/gfxhub_v12_1.c | 36 +++++++
drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c | 38 +++++++
drivers/gpu/drm/amd/amdgpu/gfxhub_v3_0.c | 33 ++++++
10 files changed, 320 insertions(+)
--
2.34.1
