https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=296663
Bug ID: 296663
Summary: bhyve: gvt-d probe claims discrete Intel GPUs
(Arc/DG2), VM startup fails with "Unknown IGD device"
/ GSM errors
Product: Base System
Version: 15.1-RELEASE
Hardware: amd64
OS: Any
Status: New
Severity: Affects Some People
Priority: ---
Component: bhyve
Assignee: [email protected]
Reporter: [email protected]
Created attachment 272689
--> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=272689&action=edit
Proof of concept, not for commit: skip GVT-d quirks for devices not in the IGD
table
### Environment
Same system as bug #296662: FreeBSD 15.1-RELEASE amd64, AMD Ryzen
(Cezanne) host with AMD-Vi, Intel Arc A380 (8086:56a5) on ppt(4),
Ubuntu 24.04 guest via vm-bhyve/grub-bhyve.
### Overview
usr.sbin/bhyve/amd64/pci_gvt-d.c registers quirks for passing through
Intel integrated GPUs (Graphics Stolen Memory, OpRegion). Its probe
matches on vendor == Intel && class == display only:
gvt_d_probe():
vendor == 0x8086 ?
class == PCIC_DISPLAY ?
-> claim the device
This also matches discrete Intel GPUs (Arc/DG2, e.g. 8086:56a5), which
have no GSM and no OpRegion. gvt_d_init() then fails — depending on the
host, either at the hw.intel_graphics_stolen_* sysctls / E820
allocation, at get_igd_ops() ("Unknown IGD device. It's not supported
yet!"), or at gvt_d_setup_opregion() ("Invalid OpRegion signature") —
and because a pe_init failure is fatal in init_pci(), the VM does not
start at all.
This failure mode was previously reported on freebsd-virtualization for
an Arc A380 on an EPYC host ("Unable to setup Graphics Stolen Memory" /
"Invalid OpRegion signature", January 2025), where the suggested
direction was to skip the GVT-d quirks for such devices.
Note: on my system this failure mode was masked in an unfortunate way —
because of the FLR recovery bug (bug #296662), gvt_d_probe() read
0xffff as the vendor ID from the still-unresponsive device and silently
declined it, so the VM started but with an invisible GPU. Once the FLR
issue is fixed, every Arc passthrough attempt would instead die in
gvt_d_init(). The two bugs together make Arc passthrough impossible in
different ways, whichever one you hit first.
### Steps to Reproduce
1. Ensure the GPU responds to config space accesses at probe time (fix
or work around the FLR-timing issue from bug #296662).
2. Start a bhyve VM with an Arc GPU function as a passthru device.
3. bhyve exits during initialization with GSM/IGD/OpRegion errors.
### Expected Results
Discrete Intel GPUs are passed through as plain passthru devices; the
GVT-d quirks apply only to the integrated GPUs they were written for.
### Root cause confirmation (proof of concept)
To confirm the analysis, a proof-of-concept change was tested on the
affected system (see attachment): in gvt_d_probe(), return ENXIO when
get_igd_ops() has no entry for the device, so unknown/discrete devices
fall through to the generic passthru path instead of failing VM
startup. Tested together with the PoC from bug #296662: the Arc A380
passes through cleanly to an Ubuntu 24.04 guest, i915 initializes fully
(GuC 70.36.0 / HuC authenticated), VA-API hardware transcoding works.
The one-line direction (restrict the GVT-d probe to devices the code
actually knows how to handle) matches what was suggested on
freebsd-virtualization; the exact criterion (device table vs. host bus
0/2/0 vs. something else) is left to the maintainers.
usr.sbin/bhyve/amd64/pci_gvt-d.c is currently identical in main and
releng/15.1, so the PoC applies unmodified to both; all testing was
done on 15.1-RELEASE.
### Provenance disclosure
The analysis was performed interactively on the affected hardware. The
attached proof-of-concept diff was developed with the assistance of an
LLM (Anthropic Claude). I am not a C developer and have not performed a
full code review, so I am explicitly NOT proposing it for commit; it is
attached to document how the root cause was confirmed and as a
temporary mitigation for affected users. I am happy to test any
proposed fix on the affected hardware.
See also bug #296662 (bhyve passthru caches the PCI config header while
the device is still recovering from FLR) — both bugs must be addressed
for Intel Arc passthrough to work.
--
You are receiving this mail because:
You are the assignee for the bug.
