This patch series introduces the dxgkrnl driver, which provides Linux
support for virtual compute devices (vGPUs) paravirtualized by a Windows
Hyper-V host. The primary use case is Windows Subsystem for Linux (WSL2),
where the driver enables GPU-accelerated workloads inside Linux containers
running on Windows.
The driver creates /dev/dxg, a miscdevice that user-space API libraries
(such as the open-source libdxg) use to communicate with virtual GPU
adapters via IOCTLs implementing a subset of the WDDM/MCDM D3DKMT
interface. Physical GPU access is performed entirely on the Windows host;
the guest driver communicates over Hyper-V VMBus channels.
Key characteristics:
- Self-contained under drivers/hv/dxgkrnl/
- Depends only on CONFIG_HYPERV, DMA_SHARED_BUFFER, SYNC_FILE
- Supports multiple vGPU adapters per VM
- DMA fence integration via dxgsyncfile (SYNC_FILE)
- Supports compute-only accelerators (AI/ML workloads) as well as
full graphics adapters
Changes since v3 (posted March 2022):
- Replace deprecated one-element arrays [1] with C99 flexible arrays []
- Replace %px with %p in trace macros
- Remove unnecessary braces from single-statement if blocks
- Remove LINUX_VERSION_CODE guard for max_pkt_size (added in 5.15,
well before any target kernel for this submission)
- Remove linux/version.h include (no longer needed)
- Fix whitespace issues flagged by checkpatch
- Replace non-debug DXG_ERR do{}while(0) macro with direct dev_err call
- Change -EBADE to -ENODEV for global channel duplicate detection
(as requested by Greg KH in v3 review)
- Remove MODULE_VERSION (not recommended for in-tree drivers)
- Add explanatory comment to guid_to_luid() cast
- Additional features and fixes developed in the WSL2 fork:
* dxgsyncfile: DMA fence / sync file integration
* D3DKMTEnumProcesses, D3DDKMTIsFeatureEnabled, D3DKMTInvalidateCache
* Compute-only adapter support
* pin_user_pages for DMA-accessible memory
* Retry logic for VMBus ring buffer full condition
* Various synchronization and memory safety fixes
Regarding the dxgglobal singleton raised in v3 review:
The design reflects a host architecture constraint: each Hyper-V VM has
exactly one global VMBus channel offered by the host, regardless of how
many vGPU adapters are present. The dxgglobal structure encapsulates this
VM-level state (global channel, adapter list, process list, host event
tracking). Per-adapter state is separately managed in dxgadapter objects.
This design was previously explained in the v3 thread; the architecture
matches the Hyper-V GPU-PV protocol which is fixed by the host side.
The patches apply on top of v6.6-lts. The user-space library (libdxg)
that communicates with this driver is available at:
https://github.com/microsoft/libdxg
The full WDDM compute stack (OpenCL, oneAPI, OpenVINO) is available
open-source via Intel's compute-runtime project.
Iouri Tarassov ([email protected]) is the primary author and
maintainer of this driver.
Eric Curtin (1):
drivers: hv: dxgkrnl: Fix checkpatch issues and address reviewer
feedback
Hideyuki Nagase (1):
drivers: hv: dxgkrnl: Fix crash at hmgrtable_free_handle
Iouri Tarassov (53):
drivers: hv: dxgkrnl: Driver initialization and loading
drivers: hv: dxgkrnl: Add VMBus message support, initialize VMBus
channels.
drivers: hv: dxgkrnl: Creation of dxgadapter object
drivers: hv: dxgkrnl: Opening of /dev/dxg device and dxgprocess
creation
drivers: hv: dxgkrnl: Enumerate and open dxgadapter objects
drivers: hv: dxgkrnl: Creation of dxgdevice objects
drivers: hv: dxgkrnl: Creation of dxgcontext objects
drivers: hv: dxgkrnl: Creation of compute device allocations and
resources
drivers: hv: dxgkrnl: Creation of compute device sync objects
drivers: hv: dxgkrnl: Operations using sync objects
drivers: hv: dxgkrnl: Sharing of dxgresource objects
drivers: hv: dxgkrnl: Sharing of sync objects
drivers: hv: dxgkrnl: Creation of paging queue objects.
drivers: hv: dxgkrnl: Submit execution commands to the compute device
drivers: hv: dxgkrnl: Share objects with the host
drivers: hv: dxgkrnl: Query the dxgdevice state
drivers: hv: dxgkrnl: Map(unmap) CPU address to device allocation
drivers: hv: dxgkrnl: Manage device allocation properties
drivers: hv: dxgkrnl: Flush heap transitions
drivers: hv: dxgkrnl: Query video memory information
drivers: hv: dxgkrnl: The escape ioctl
drivers: hv: dxgkrnl: Ioctl to put device to error state
drivers: hv: dxgkrnl: Ioctls to query statistics and clock calibration
drivers: hv: dxgkrnl: Offer and reclaim allocations
drivers: hv: dxgkrnl: Ioctls to manage scheduling priority
drivers: hv: dxgkrnl: Manage residency of allocations
drivers: hv: dxgkrnl: Manage compute device virtual addresses
drivers: hv: dxgkrnl: Add support to map guest pages by host
drivers: hv: dxgkrnl: Removed struct vmbus_gpadl, which was defined in
the main linux branch
drivers: hv: dxgkrnl: Remove dxgk_init_ioctls
drivers: hv: dxgkrnl: Creation of dxgsyncfile objects
drivers: hv: dxgkrnl: Use tracing instead of dev_dbg
drivers: hv: dxgkrnl: Implement D3DKMTWaitSyncFile
drivers: hv: dxgkrnl: Improve tracing and return values from copy from
user
drivers: hv: dxgkrnl: Fix synchronization locks
drivers: hv: dxgkrnl: Close shared file objects in case of a failure
drivers: hv: dxgkrnl: Added missed NULL check for resource object
drivers: hv: dxgkrnl: Fixed dxgkrnl to build for the 6.1 kernel
drivers: hv: dxgkrnl: Added support for compute only adapters
drivers: hv: dxgkrnl: Added implementation for D3DKMTInvalidateCache
drivers: hv: dxgkrnl: Handle process ID in D3DKMTQueryStatistics
drivers: hv: dxgkrnl: Implement the D3DKMTEnumProcesses API
drivers: hv: dxgkrnl: Implement D3DDKMTIsFeatureEnabled API
drivers: hv: dxgkrnl: Implement known escapes
drivers: hv: dxgkrnl: Fixed coding style issues
drivers: hv: dxgkrnl: Fixed the implementation of
D3DKMTQueryClockCalibration
drivers: hv: dxgkrnl: Retry sending a VM bus packet when there is no
place in the ring buffer
drivers: hv: dxgkrnl: Add support for locking a shared allocation by
not the owner
drivers: hv: dxgkrnl: Fix build breaks when switching to 6.6 kernel
due to hv_driver remove callback change.
drivers: hv: dxgkrnl: Fix build breaks when switching to 6.6 kernel
due to removed uuid_le_cmp
drivers: hv: dxgkrnl: Implement D3DKMTEnumProcesses to match the
Windows implementation
drivers: hv: dxgkrnl: Use pin_user_pages instead of get_user_pages for
DMA accessible memory
drivers: hv: dxgkrnl: Do not print error messages when virtual GPU is
not present
MAINTAINERS | 7 +
drivers/hv/Kconfig | 2 +
drivers/hv/Makefile | 1 +
drivers/hv/dxgkrnl/Kconfig | 28 +
drivers/hv/dxgkrnl/Makefile | 5 +
drivers/hv/dxgkrnl/dxgadapter.c | 1367 ++++++++
drivers/hv/dxgkrnl/dxgkrnl.h | 1042 ++++++
drivers/hv/dxgkrnl/dxgmodule.c | 971 +++++
drivers/hv/dxgkrnl/dxgprocess.c | 348 ++
drivers/hv/dxgkrnl/dxgsyncfile.c | 481 +++
drivers/hv/dxgkrnl/dxgsyncfile.h | 33 +
drivers/hv/dxgkrnl/dxgvmbus.c | 3992 +++++++++++++++++++++
drivers/hv/dxgkrnl/dxgvmbus.h | 910 +++++
drivers/hv/dxgkrnl/hmgr.c | 567 +++
drivers/hv/dxgkrnl/hmgr.h | 112 +
drivers/hv/dxgkrnl/ioctl.c | 5648 ++++++++++++++++++++++++++++++
drivers/hv/dxgkrnl/misc.c | 38 +
drivers/hv/dxgkrnl/misc.h | 96 +
include/uapi/misc/d3dkmthk.h | 1794 ++++++++++
19 files changed, 17442 insertions(+)
create mode 100644 drivers/hv/dxgkrnl/Kconfig
create mode 100644 drivers/hv/dxgkrnl/Makefile
create mode 100644 drivers/hv/dxgkrnl/dxgadapter.c
create mode 100644 drivers/hv/dxgkrnl/dxgkrnl.h
create mode 100644 drivers/hv/dxgkrnl/dxgmodule.c
create mode 100644 drivers/hv/dxgkrnl/dxgprocess.c
create mode 100644 drivers/hv/dxgkrnl/dxgsyncfile.c
create mode 100644 drivers/hv/dxgkrnl/dxgsyncfile.h
create mode 100644 drivers/hv/dxgkrnl/dxgvmbus.c
create mode 100644 drivers/hv/dxgkrnl/dxgvmbus.h
create mode 100644 drivers/hv/dxgkrnl/hmgr.c
create mode 100644 drivers/hv/dxgkrnl/hmgr.h
create mode 100644 drivers/hv/dxgkrnl/ioctl.c
create mode 100644 drivers/hv/dxgkrnl/misc.c
create mode 100644 drivers/hv/dxgkrnl/misc.h
create mode 100644 include/uapi/misc/d3dkmthk.h
