This series adds support for a userspace second-level trap handler on
render-node user queues (UQs), equivalent to what KFD already provides
for compute workloads via the HSA trap handler ABI.
When a GPU shader hits an exception, the hardware jumps to the
first-level CWSR trap handler. The first-level handler checks two slots
in the per-VM TMA buffer for a second-level handler address and
forwards the trap to userspace if one is installed.
The second-level handler is a per-VM (per-process) setting. All shader
waves executing under this VMID share the same handler regardless of
queue type. GFX and compute queues from the same process share the same
VMID, so one SET_L2_TRAP covers all queue types for that DRM file
descriptor. This is confirmed to work for GFX9 and later hardware.
Two new VM ioctl operations are added:
AMDGPU_VM_OP_SET_L2_TRAP (op = 3)
AMDGPU_VM_OP_CLEAR_L2_TRAP (op = 4)
Both TBA and TMA buffers MUST be allocated with VM_ALWAYS_VALID. The
GPU jumps to TBA on hardware exception without any software submission,
so the kernel cannot track these BOs in a job's used-buffer list. An
evicted BO leaves a stale page table entry, causing a GPU page fault on
the next shader exception.
Changes in v2:
- Move SET_L2_TRAP from the old TRAP ioctl (never released to
upstream) into the VM ioctl as a per-VM property. The TRAP ioctl
and its associated structures are removed entirely (Patch 1).
- Add VM_ALWAYS_VALID validation in amdgpu_trap_validate_user_addr().
SET_L2_TRAP returns -EINVAL if TBA or TMA BO does not have this
flag. Without it, eviction leaves a stale page table entry
(Natalie Vock).
- Clarify in kdoc: SET_L2_TRAP is per-VM (per-process). No conflict
between processes on user queue VMIDs. kq_tma_bo is CWSR scratch
for the first-level handler only. SET_L2_TRAP for kernel queue
VMIDs is not supported in this series (Lijo Lazar).
- Replace void *tma_cpu_addr with struct iosys_map tma_map. On dGPUs
the TMA BO may be placed in VRAM and accessed via MMIO (PCI BAR);
raw pointer dereferences are unsafe there.
- Add evict-flush-write-restore sequence in SET_L2_TRAP. Writing TBA
and TMA as two separate stores creates a race window where a
trapping wave reads a mismatched pair.
- Evict user queues in amdgpu_trap_free() at file close to prevent
waves from trapping into a freed TMA BO.
- Intercept GEM_VA UNMAP and CLEAR to disable the handler when TBA
or TMA is unmapped without calling CLEAR_L2_TRAP first (Patch 3).
RFC v1: https://patchwork.freedesktop.org/series/172489/
Srinivasan Shanmugam (3):
drm/amdgpu/uapi: Add second-level trap handler ops to VM ioctl
drm/amdgpu: Add VM ioctl handlers for second-level trap handler
drm/amdgpu: Disable L2 trap handler when its VA range is unmapped
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 1 -
.../drm/amd/amdgpu/amdgpu_eviction_fence.c | 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 22 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 10 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_trap.c | 298 ++++++++++++------
drivers/gpu/drm/amd/amdgpu/amdgpu_trap.h | 63 +++-
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c | 16 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_userq.h | 2 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 25 ++
include/uapi/drm/amdgpu_drm.h | 98 ++++--
10 files changed, 402 insertions(+), 134 deletions(-)
--
2.34.1
