Currently kfd manages kfd_process in a one context (kfd_process) per program manner, thus each user space program only onws one kfd context (kfd_process).
This model works fine for most of the programs, but imperfect
for a hypervisor like QEMU. Because all programs in the guest
user space share the same only one kfd context, which is
problematic, including but not limited to:
As illustrated in Figure 1, all guest user space programs share the same fd of
/dev/kfd
and the same kfd_process, and the same PASID leading to the same
GPU_VM address space. Therefore the IOVA range of each
guest user space programs are not isolated,
they can attack each other through GPU DMA.
+----------------------------------------------------------------------------------+
|
|
| +-----------+ +-----------+ +------------+ +------------+
|
| | | | | | | | |
|
| | Program 1 | | Program 2 | | Program 3 | | Program N |
|
| | | | | | | | |
|
| +----+------+ +--------+--+ +--+---------+ +-----+------+
|
| | | | |
|
| | | | |
Guest |
| | | | |
|
+-------+----------------------+------------+----------------------+---------------+
| | | |
| | | |
| | | |
| | | |
| +--+------------+---+ |
| | file descriptor | |
+-------------------+ of /dev/kfd +------------------+
| opened by QEMU |
| |
+---------+---------+ User Space
| QEMU
|
---------------------------------------+-----------------------------------------------------
| Kernel
Space
| KFD Module
|
+--------+--------+
| |
| kfd_process |<------------------The only
one KFD context
| |
+--------+--------+
|
+--------+--------+
| PASID |
+--------+--------+
|
+--------+--------+
| GPU_VM |
+-----------------+
Fiture 1
This series implements a multiple contexts solution:
- Allow each program to create and hold multiple contexts (kfd processes)
- Each context has its own fd of /dev/kfd and an exclusive kfd_process,
which is a secondary kfd context. So that PASID/GPU VM isolates their IOVA
address spaces.
Therefore, they can not attack each other through GPU DMA.
The design is illustrated in Figure 2 below:
+---------------------------------------------------------------------------------------------------------+
|
|
|
|
|
|
|
+----------------------------------------------------------------------------------+
|
| |
| |
| | +-----------+ +-----------+ +-----------+
+-----------+ | |
| | | | | | | | |
| | |
| | | Program 1 | | Program 2 | | Program 3 | | Program N
| | |
| | | | | | | | |
| | |
| | +-----+-----+ +-----+-----+ +-----+-----+
+-----+-----+ | |
| | | | | |
| |
| | | | | |
Guest | |
| | | | | |
| |
|
+-------+------------------+-----------------+----------------+--------------------+
|
| | | | |
QEMU |
| | | | |
|
+---------------+------------------+-----------------+----------------+--------------------------+--------+
| | | |
|
| | | |
|
| | | |
|
+---+----+ +---+----+ +---+----+ +---+----+
+---+-----+
| | | | | | | |
| Primary |
| FD 1 | | FD 2 | | FD 3 | | FD 4 |
| FD |
| | | | | | | |
| |
+---+----+ +---+----+ +---+----+ +----+---+
+----+----+
| | | |
| User Space
| | | |
|
-------------------+------------------+-----------------+-----------------+--------------------------+----------------------------
| | | |
| Kernel SPace
| | | |
|
| | | |
|
+--------------------------------------------------------------------------------------------------------------------------+
| +------+------+ +------+------+ +------+------+
+------+------+ +------+------+ |
| | Secondary | | Secondary | | Secondary | | Secondary
| | Primary | KFD Module |
| |kfd_process 1| |kfd_process 2| |kfd_process 3| |kfd_process
4| | kfd_process | |
| | | | | | | |
| | | |
| +------+------+ +------+------+ +------+------+
+------+------+ +------+------+ |
| | | | |
| |
| +------+------+ +------+------+ +------+------+
+------+------+ +------+------+ |
| | PASID | | PASID | | PASID | | PASID
| | PASID | |
| +------+------+ +------+------+ +------+------+
+------+------+ +------+------+ |
| | | | |
| |
| | | | |
| |
| +------+------+ +------+------+ +------+------+
+------+------+ +------+------+ |
| | GPU_VM | | GPU_VM | | GPU_VM | | GPU_VM
| | GPU_VM | |
| +-------------+ +-------------+ +-------------+
+-------------+ +-------------+ |
|
|
+--------------------------------------------------------------------------------------------------------------------------+
Figure 2
Zhu Lingshan (9):
amdkfd: enlarge the hashtable of kfd_process
amdkfd: mark the first kfd_process as the primary one
amdkfd: find_process_by_mm always return the primary context
amdkfd: Introduce kfd_create_process_sysfs as a separate function
amdkfd: destroy kfd secondary contexts through fd close
amdkfd: process svm ioctl only on the primary kfd process
amdkfd: process USERPTR allocation only on the primary kfd process
amdkfd: identify a secondary kfd process by its id
amdkfd: introduce new ioctl AMDKFD_IOC_CREATE_PROCESS
drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 62 ++++++-
drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 14 +-
drivers/gpu/drm/amd/amdkfd/kfd_process.c | 204 +++++++++++++++++------
include/uapi/linux/kfd_ioctl.h | 8 +-
4 files changed, 231 insertions(+), 57 deletions(-)
--
2.47.1
