On Wed, Sep 05, 2018 at 02:26:51PM +0800, Daniel Drake wrote:
> On Tue, Aug 28, 2018 at 5:57 PM, Peter Wu <[email protected]> wrote:
> > Only non-bridge devices can be passed to a guest, but perhaps logging
> > access to the emulated bridge is already sufficient. The Prefetchable
> > Base Upper 32 Bits register is at offset 0x28.
> >
> > In a trace where the Nvidia device is disabled/enabled via Device
> > Manager, I see writes on the enable path:
> >
> > [email protected]:rp_write_config (ioh3420, @0x28, 0x0, len=0x4)
>
> Did you do anything special to get an emulated bridge included in this setup?
Yes, I followed instructions in QEMU's docs/pcie.txt and ended up with:
-device ioh3420,id=rp1,bus=pcie.0,addr=1c.0,port=1
-device
vfio-pci,bus=rp1,host=01:00.0,rombar=0,x-pci-sub-vendor-id=0x1028,x-pci-sub-device-id=0x07be
(Subvendor/device IDs are from lspci -nnv).
> Folllowing the instructions at
> https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF I can
> successfully pass through devices to windows running under
> virt-manager. In the nvidia GPU case I haven't got passed the driver
> installation failure, but I can pass through other devices OK and
> install their drivers.
After installing drivers, it would still not start. For that to work I
had to pass the VBIOS via an ACPI _ROM method:
-acpitable file=fakedev.aml
-fw_cfg name=opt/nl.lekensteyn/vfio-vbios,file=vbios.rom
These options were taken from:
https://github.com/Lekensteyn/acpi-stuff/blob/master/d3test/XPS9560/boot-vm
fakedev.asl source file and instructions to extract the VBIOS:
https://github.com/Lekensteyn/acpi-stuff/tree/master/d3test
> However I do not end up with any PCI-to-PCI bridges in this setup. The
> passed through device sits at address 00:08.0, parent is the PCI host
> bridge 00:00.0.
>
> (I'm trying to spy if Windows appears to restore or reset the PCI
> bridge prefetch registers upon resume)
If you want to suspend the guest, note that Windows refuses suspend
with the default VGA adapter (see "devicequery /a"). Try the QXL adapter
with https://gitlab.freedesktop.org/spice/win32/qxl-wddm-dod
-vga qxl -device qemu-xhci -device usb-tablet
Not sure how well tested this is, I had to patch Linux to avoid an oops.
If I try this on Windows, it successfully suspends ("info status" in
QEMU monitor says "paused (suspended)"), but resume ends up with a black
screen...
Luckily, the important information is already logged. Windows 10 indeed
seems to write to "Prefetchable Base Upper 32 Bits" on resume[1].
--
Kind regards,
Peter Wu
https://lekensteyn.nl
[1]: QEMU output (annotated with register names) for
./run-vm.sh -device usb-tablet -vga qxl /tmp/w10.qcow2 -trace
rp_read_config,file=/dev/stdout -trace rp_write_config,file=/dev/stdout
<suspend>
NET._PS3
[email protected]:rp_write_config (ioh3420, @0x12c, 0x0, len=0x4)
AER: Root Error Command
[email protected]:rp_write_config (ioh3420, @0xac, 0x0, len=0x2)
PCIE: Root Control
[email protected]:rp_read_config (ioh3420, @0xac, len=0x2) 0x0
PCIE: Root Control
[email protected]:rp_read_config (ioh3420, @0xa0, len=0x2) 0x0
PCIE: Link Control
[email protected]:rp_write_config (ioh3420, @0xb0, 0x10000, len=0x4)
PCIE: Root Status
[email protected]:rp_read_config (ioh3420, @0x4, len=0x2) 0x506
Command
[email protected]:rp_read_config (ioh3420, @0xc, len=0x1) 0x0
Cacheline Size
[email protected]:rp_read_config (ioh3420, @0xd, len=0x1) 0x0
Latency Timer
[email protected]:rp_read_config (ioh3420, @0x3c, len=0x1) 0x0
Interrupt Line
[email protected]:rp_read_config (ioh3420, @0x19, len=0x1) 0x1
Secondary Bus Number
[email protected]:rp_read_config (ioh3420, @0x1a, len=0x1) 0x1
Subordiante Bus Number
[email protected]:rp_read_config (ioh3420, @0x3e, len=0x2) 0x2
Bridge Control
[email protected]:rp_read_config (ioh3420, @0x0, len=0x2) 0x8086
Device ID
[email protected]:rp_read_config (ioh3420, @0x2, len=0x2) 0x3420
Vendor ID
[email protected]:rp_read_config (ioh3420, @0x8, len=0x1) 0x2
Revision
[email protected]:rp_read_config (ioh3420, @0x9, len=0x1) 0x0
Class Code
[email protected]:rp_read_config (ioh3420, @0xa, len=0x1) 0x4
+1 Class Code
[email protected]:rp_read_config (ioh3420, @0xb, len=0x1) 0x6
+2 Class Code
[email protected]:rp_read_config (ioh3420, @0xe, len=0x1) 0x1
Header Type
[email protected]:rp_read_config (ioh3420, @0x6, len=0x2) 0x10
Status
[email protected]:rp_read_config (ioh3420, @0x34, len=0x1) 0xe0
Capabilities Pointer
[email protected]:rp_read_config (ioh3420, @0xe0, len=0x2) 0x9001
[email protected]:rp_read_config (ioh3420, @0x90, len=0x2) 0x6010
PCI Express
[email protected]:rp_read_config (ioh3420, @0x60, len=0x2) 0x4005
Message Signaled Interrupts
[email protected]:rp_read_config (ioh3420, @0x40, len=0x2) 0xd
Bridge subsystem vendor/device ID
[email protected]:rp_read_config (ioh3420, @0x44, len=0x2) 0x8086
[email protected]:rp_read_config (ioh3420, @0x46, len=0x2) 0x0
[email protected]:rp_read_config (ioh3420, @0x98, len=0x2) 0x7
PCIE: Device Control
[email protected]:rp_read_config (ioh3420, @0xb8, len=0x2) 0x0
PCIE: Device Control 2
[email protected]:rp_read_config (ioh3420, @0x0, len=0x2) 0x8086
Device ID
[email protected]:rp_read_config (ioh3420, @0x2, len=0x2) 0x3420
Vendor ID
[email protected]:rp_read_config (ioh3420, @0x8, len=0x1) 0x2
Revision
[email protected]:rp_read_config (ioh3420, @0x9, len=0x1) 0x0
Class Code
[email protected]:rp_read_config (ioh3420, @0xa, len=0x1) 0x4
+1 Class Code
[email protected]:rp_read_config (ioh3420, @0xb, len=0x1) 0x6
+2 Class Code
[email protected]:rp_read_config (ioh3420, @0xe, len=0x1) 0x1
Header Type
[email protected]:rp_read_config (ioh3420, @0x6, len=0x2) 0x10
Status
[email protected]:rp_read_config (ioh3420, @0x34, len=0x1) 0xe0
Capabilities Pointer
[email protected]:rp_read_config (ioh3420, @0xe0, len=0x2) 0x9001
[email protected]:rp_read_config (ioh3420, @0x90, len=0x2) 0x6010
PCI Express
[email protected]:rp_read_config (ioh3420, @0x60, len=0x2) 0x4005
Message Signaled Interrupts
[email protected]:rp_read_config (ioh3420, @0x40, len=0x2) 0xd
Bridge subsystem vendor/device ID
[email protected]:rp_read_config (ioh3420, @0x44, len=0x2) 0x8086
[email protected]:rp_read_config (ioh3420, @0x46, len=0x2) 0x0
[email protected]:rp_read_config (ioh3420, @0x4, len=0x2) 0x506
Command
[email protected]:rp_write_config (ioh3420, @0x4, 0x506, len=0x2)
Command
[email protected]:rp_read_config (ioh3420, @0x62, len=0x2) 0x103
MSI: Message Control
[email protected]:rp_write_config (ioh3420, @0x62, 0x102, len=0x2)
MSI: Message Control
[email protected]:rp_read_config (ioh3420, @0x4, len=0x2) 0x506
Command
[email protected]:rp_write_config (ioh3420, @0x4, 0x500, len=0x2)
Command
[email protected]:rp_read_config (ioh3420, @0x9a, len=0x2) 0x0
PCIE: Device Status
[email protected]:rp_read_config (ioh3420, @0xe0, len=0x4) 0xc8039001
[email protected]:rp_read_config (ioh3420, @0xe4, len=0x4) 0x8
[email protected]:rp_write_config (ioh3420, @0xe4, 0xb, len=0x2)
[email protected]:rp_read_config (ioh3420, @0xe4, len=0x2) 0xb
_PS3
PG00._ON
PG00._OFF
<resume>
PG00._ON
PG00._ON
_PS0
[email protected]:rp_read_config (ioh3420, @0x0, len=0x2) 0x8086
Device ID
[email protected]:rp_read_config (ioh3420, @0x2, len=0x2) 0x3420
Vendor ID
[email protected]:rp_read_config (ioh3420, @0x8, len=0x1) 0x2
Revision
[email protected]:rp_read_config (ioh3420, @0x9, len=0x1) 0x0
Class Code
[email protected]:rp_read_config (ioh3420, @0xa, len=0x1) 0x4
+1 Class Code
[email protected]:rp_read_config (ioh3420, @0xb, len=0x1) 0x6
+2 Class Code
[email protected]:rp_read_config (ioh3420, @0xe, len=0x1) 0x1
Header Type
[email protected]:rp_read_config (ioh3420, @0x6, len=0x2) 0x10
Status
[email protected]:rp_read_config (ioh3420, @0x34, len=0x1) 0xe0
Capabilities Pointer
[email protected]:rp_read_config (ioh3420, @0xe0, len=0x2) 0x9001
[email protected]:rp_read_config (ioh3420, @0x90, len=0x2) 0x6010
PCI Express
[email protected]:rp_read_config (ioh3420, @0x60, len=0x2) 0x4005
Message Signaled Interrupts
[email protected]:rp_read_config (ioh3420, @0x40, len=0x2) 0xd
Bridge subsystem vendor/device ID
[email protected]:rp_read_config (ioh3420, @0x44, len=0x2) 0x8086
[email protected]:rp_read_config (ioh3420, @0x46, len=0x2) 0x0
[email protected]:rp_read_config (ioh3420, @0xe0, len=0x4) 0xc8039001
[email protected]:rp_read_config (ioh3420, @0xe4, len=0x4) 0xb
[email protected]:rp_write_config (ioh3420, @0xe4, 0x8, len=0x2)
[email protected]:rp_read_config (ioh3420, @0x0, len=0x2) 0x8086
Device ID
[email protected]:rp_read_config (ioh3420, @0x2, len=0x2) 0x3420
Vendor ID
[email protected]:rp_read_config (ioh3420, @0x8, len=0x1) 0x2
Revision
[email protected]:rp_read_config (ioh3420, @0x9, len=0x1) 0x0
Class Code
[email protected]:rp_read_config (ioh3420, @0xa, len=0x1) 0x4
+1 Class Code
[email protected]:rp_read_config (ioh3420, @0xb, len=0x1) 0x6
+2 Class Code
[email protected]:rp_read_config (ioh3420, @0xe, len=0x1) 0x1
Header Type
[email protected]:rp_read_config (ioh3420, @0x6, len=0x2) 0x10
Status
[email protected]:rp_read_config (ioh3420, @0x34, len=0x1) 0xe0
Capabilities Pointer
[email protected]:rp_read_config (ioh3420, @0xe0, len=0x2) 0x9001
[email protected]:rp_read_config (ioh3420, @0x90, len=0x2) 0x6010
PCI Express
[email protected]:rp_read_config (ioh3420, @0x60, len=0x2) 0x4005
Message Signaled Interrupts
[email protected]:rp_read_config (ioh3420, @0x40, len=0x2) 0xd
Bridge subsystem vendor/device ID
[email protected]:rp_read_config (ioh3420, @0x44, len=0x2) 0x8086
[email protected]:rp_read_config (ioh3420, @0x46, len=0x2) 0x0
[email protected]:rp_read_config (ioh3420, @0xe4, len=0x2) 0x8
[email protected]:rp_write_config (ioh3420, @0x10, 0x0, len=0x4)
BAR0
[email protected]:rp_write_config (ioh3420, @0x14, 0x0, len=0x4)
BAR1
[email protected]:rp_write_config (ioh3420, @0x1c, 0xff, len=0x2)
I/O Base
[email protected]:rp_write_config (ioh3420, @0x20, 0xfc10fc00,
len=0x4) Memory Base
[email protected]:rp_write_config (ioh3420, @0x24, 0xfeb0fea0,
len=0x4) Prefetchable Memory Base
[email protected]:rp_write_config (ioh3420, @0x28, 0x0, len=0x4)
Prefetchable Base Upper 32 Bits
[email protected]:rp_write_config (ioh3420, @0x2c, 0x0, len=0x4)
Prefetchable Limit Upper 32 Bits
[email protected]:rp_write_config (ioh3420, @0x30, 0x0, len=0x4)
I/O Base Upper 16 Bits
[email protected]:rp_write_config (ioh3420, @0x38, 0x0, len=0x4)
[email protected]:rp_write_config (ioh3420, @0x3c, 0x0, len=0x1)
Interrupt Line
[email protected]:rp_write_config (ioh3420, @0x3e, 0x2, len=0x2)
Bridge Control
[email protected]:rp_read_config (ioh3420, @0x3e, len=0x2) 0x2
Bridge Control
[email protected]:rp_write_config (ioh3420, @0x18, 0x0, len=0x1)
Primary Bus Number
[email protected]:rp_write_config (ioh3420, @0x19, 0x1, len=0x1)
Secondary Bus Number
[email protected]:rp_write_config (ioh3420, @0x1a, 0x1, len=0x1)
Subordiante Bus Number
[email protected]:rp_write_config (ioh3420, @0xc, 0x0, len=0x1)
Cacheline Size
[email protected]:rp_write_config (ioh3420, @0xd, 0x0, len=0x1)
Latency Timer
[email protected]:rp_write_config (ioh3420, @0x4, 0x500, len=0x2)
Command
[email protected]:rp_write_config (ioh3420, @0x98, 0x7, len=0x2)
PCIE: Device Control
[email protected]:rp_write_config (ioh3420, @0xb8, 0x0, len=0x2)
PCIE: Device Control 2
[email protected]:rp_read_config (ioh3420, @0x4, len=0x2) 0x500
Command
[email protected]:rp_write_config (ioh3420, @0x4, 0x506, len=0x2)
Command
[email protected]:rp_write_config (ioh3420, @0x6, 0xf900, len=0x2)
Status
[email protected]:rp_write_config (ioh3420, @0x1e, 0xf900, len=0x2)
Secondary Status
[email protected]:rp_read_config (ioh3420, @0x4, len=0x2) 0x506
Command
[email protected]:rp_write_config (ioh3420, @0x4, 0x506, len=0x2)
Command
[email protected]:rp_read_config (ioh3420, @0x62, len=0x2) 0x102
MSI: Message Control
[email protected]:rp_write_config (ioh3420, @0x62, 0x102, len=0x2)
MSI: Message Control
[email protected]:rp_read_config (ioh3420, @0x62, len=0x2) 0x102
MSI: Message Control
[email protected]:rp_write_config (ioh3420, @0x64, 0xfee0100c,
len=0x4) MSI: Message Address
[email protected]:rp_write_config (ioh3420, @0x68, 0x4950, len=0x2)
MSI: Message Upper Address
[email protected]:rp_write_config (ioh3420, @0x6c, 0xfffffffe,
len=0x4) MSI: Message Data
[email protected]:rp_write_config (ioh3420, @0x62, 0x103, len=0x2)
MSI: Message Control
[email protected]:rp_write_config (ioh3420, @0xac, 0x0, len=0x2)
PCIE: Root Control
[email protected]:rp_write_config (ioh3420, @0xa0, 0x0, len=0x2)
PCIE: Link Control
[email protected]:rp_read_config (ioh3420, @0x98, len=0x2) 0x7
PCIE: Device Control
[email protected]:rp_write_config (ioh3420, @0x98, 0x7, len=0x2)
PCIE: Device Control
[email protected]:rp_read_config (ioh3420, @0xb0, len=0x4) 0x0
PCIE: Root Status
[email protected]:rp_write_config (ioh3420, @0x12c, 0x7, len=0x4)
AER: Root Error Command
[email protected]:rp_write_config (ioh3420, @0xac, 0x8, len=0x2)
PCIE: Root Control
NET._PS0
[email protected]:rp_write_config (ioh3420, @0xb8, 0x0, len=0x2)
PCIE: Device Control 2
_______________________________________________
Nouveau mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/nouveau
