Hello!
I would like to pass through my VGA card to KVM and use to accelerate 3D,
however everytime Windows complains about the assigned device showing error
43 in device manager. No matter I try to assign it using q35 and VFIO or pc
and legacy pci-assign, the result is the same all the time.
The VGA adapter i try to assign is NVIDIA GeForce GTX 770M:
01:00.0 VGA compatible controller: NVIDIA Corporation GK106M [GeForce GTX
770M] (rev a1)
01:00.0 0300: 10de:11e0 (rev a1)
Used CPU (i7-4800mq) and mainboard support VT-d, what can be check bu
launching dmesg | grep -e DMAR -e IOMMU:
[ 0.000000] Warning: PCIe ACS overrides enabled; This may allow
non-IOMMU protected peer-to-peer DMA
[ 0.000000] ACPI: DMAR 00000000cb967040 0000B8 (v01 INTEL HSW
00000001 INTL 00000001)
[ 0.000000] Intel-IOMMU: enabled
[ 0.020208] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap
c0000020660462 ecap f0101a
[ 0.020213] dmar: IOMMU 1: reg_base_addr fed91000 ver 1:0 cap
d2008020660462 ecap f010da
[ 0.020279] IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.250772] DMAR: No ATSR found
[ 0.250795] IOMMU 0 0xfed90000: using Queued invalidation
[ 0.250796] IOMMU 1 0xfed91000: using Queued invalidation
[ 0.250797] IOMMU: Setting RMRR:
[ 0.250807] IOMMU: Setting identity map for device 0000:00:02.0
[0xcd000000 - 0xcf1fffff]
[ 0.251040] IOMMU: Setting identity map for device 0000:00:1d.0
[0xcbea6000 - 0xcbeb4fff]
[ 0.251065] IOMMU: Setting identity map for device 0000:00:1a.0
[0xcbea6000 - 0xcbeb4fff]
[ 0.251085] IOMMU: Setting identity map for device 0000:00:14.0
[0xcbea6000 - 0xcbeb4fff]
[ 0.251100] IOMMU: Prepare 0-16MiB unity mapping for LPC
[ 0.251107] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 -
0xffffff]
[ 0.290141] [drm] DMAR active, disabling use of stolen memory
What is more, I have not found any errors in qemu debug output and/or dmesg
(or I miss something). The only things showing, in dmesg, while starting up
VM are those:
[ 95.516726] pci-stub 0000:01:00.0: claimed by stub
[ 95.747926] pci-stub 0000:01:00.0: enabling device (0006 -> 0007)
[ 97.489931] pci-stub 0000:01:00.0: kvm assign device
[ 103.287508] kvm: zapping shadow pages for mmio generation wraparound
Similar output can be found there while using VFIO, while qemu outputs:
Start bios (version ?-20140426_124946-NoteBelliash)
No Xen hypervisor found.
Ram Size=0xc0000000 (0x0000000000000000 high)
Relocating init from 0x000e2241 to 0xbffe0900 (size 63032)
Found QEMU fw_cfg
CPU Mhz=2696
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 8 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: map device bdf=00:04.0 bar 5, addr 0000c000, size 00000080 [io]
PCI: map device bdf=00:03.0 bar 1, addr 0000c080, size 00000040 [io]
PCI: map device bdf=00:01.2 bar 4, addr 0000c0c0, size 00000020 [io]
PCI: map device bdf=00:01.1 bar 4, addr 0000c0e0, size 00000010 [io]
PCI: map device bdf=00:04.0 bar 0, addr fd000000, size 01000000 [mem]
PCI: map device bdf=00:03.0 bar 6, addr fe000000, size 00040000 [mem]
PCI: map device bdf=00:04.0 bar 6, addr fe040000, size 00040000 [mem]
PCI: map device bdf=00:03.0 bar 0, addr fe080000, size 00020000 [mem]
PCI: map device bdf=00:02.0 bar 6, addr fe0a0000, size 00010000 [mem]
PCI: map device bdf=00:02.0 bar 1, addr fe0b0000, size 00001000 [mem]
PCI: map device bdf=00:04.0 bar 1, addr e0000000, size 10000000 [prefmem]
PCI: map device bdf=00:02.0 bar 0, addr f0000000, size 02000000 [prefmem]
PCI: map device bdf=00:04.0 bar 3, addr f2000000, size 02000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PIIX3/PIIX4 init: elcr=00 0c
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.2
id=8086:7020
PCI: init bdf=00:01.3
id=8086:7113
Using pmtimer, ioport 0xb008, freq 3579
kHz
PCI: init bdf=00:02.0
id=1013:00b8
PCI: init bdf=00:03.0
id=8086:100e
PCI: init bdf=00:04.0
id=10de:11e0
PCI: Using 00:02.0 for primary VGA
Found 4 cpu(s) max supported 4 cpu(s)
MP table addr=0x000f1790 MPC table addr=0x000f17a0 size=224
SMBIOS ptr=0x000f1770 table=0x000f15b0 size=439
ACPI DSDT=0xbfffe360
ACPI tables: RSDP=0x000f1580 RSDT=0xbfffe320
Scan for VGA option rom
WARNING! Found unaligned PCI rom (vd=1013:00b8)
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version ?-20140426_124946-NoteBelliash)
UHCI init on dev 00:01.2 (io=c0c0)
Found 1 lpt ports
Found 1 serial ports
Searching bootorder for: /pci@i0cf8/isa@1/fdc@03f0/floppy@0
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
ata0-0: QEMU HARDDISK ATA-7 Hard-Disk (146 GiBytes)
Searching bootorder for: /pci@i0cf8/*@1,1/drive@0/disk@0
DVD/CD [ata1-0: QEMU DVD-ROM ATAPI-4 DVD/CD]
Searching bootorder for: /pci@i0cf8/*@1,1/drive@1/disk@0
PS2 keyboard initialized
All threads complete.
Scan for option roms
Running option rom at c900:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /pci@i0cf8/*@3
Searching bootorder for: /rom@genroms/kvmvapic.bin
Press F12 for boot menu.
Searching bootorder for: HALT
drive 0x000f1500: PCHS=16383/16/63 translation=lba LCHS=1024/255/63
s=307200000
Running option rom at ca00:0003
Space available for UMB: cc800-ee800, f0000-f14a0
Returned 57344 bytes of ZoneHigh
e820 map has 7 items:
0: 0000000000000000 - 000000000009fc00 = 1 RAM
1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
3: 0000000000100000 - 00000000bfffe000 = 1 RAM
4: 00000000bfffe000 - 00000000c0000000 = 2 RESERVED
5: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
6: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
NULL
Booting from Hard Disk...
Booting from 0000:7c00
I have already applied 2 patches on kernel 3.14.1 - vga arbiter and acs.
After that, nvidia gpu is the only device in group, so I'm assigning just
it. Also lspci does not show any NVIDIA HDMI Audio device. While trying to
solve this problem, I have came across patch prepared by Alex Williamson
that aims to fix error 43. I have applied it on top of qemu, but this have
not made any difference at all. However lspci -vvv shows NoSnoop- when
using VFIO.
In order to launch VM i use below script. Actually it contains both methods
(legacy pci-assign and VFIO:
DEVICES="01:00.0"
modprobe -r kvm_intel
modprobe -r kvm
for DEV in $DEVICES; do
echo $(lspci -n | grep ${DEV} | awk '{ split($3,a,":");
print(a[1],a[2]);}') > /sys/bus/pci/drivers/pci-stub/new_id || exit 1
if [ -e /sys/bus/pci/devices/0000\:${DEV}/driver ]; then
echo "0000:${DEV}" >
/sys/bus/pci/devices/0000\:${DEV}/driver/unbind || exit 1
fi
echo "0000:${DEV}" > /sys/bus/pci/drivers/pci-stub/bind
done
modprobe kvm ignore_msrs=1
modprobe kvm_intel emulate_invalid_guest_state=0
qemu-system-x86_64 \
-enable-kvm -M pc -m 3072 -cpu
host,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -smp
4,sockets=1,cores=2,threads=2 -bios /usr/share/seabios/bios.bin \
-chardev stdio,id=seabios -device
isa-debugcon,iobase=0x402,chardev=seabios \
-vga cirrus -device
pci-assign,host=01:00.0,romfile=/path/to/vga.rom,prefer_msi=off \
-hda /path/to/disk.img \
-boot menu=on -usb -usbdevice tablet -vnc 127.0.0.1:1
exit 0
for DEV in $DEVICES; do
if [ -e /sys/bus/pci/devices/0000\:${DEV}/driver ]; then
echo "0000:${DEV}" >
/sys/bus/pci/devices/0000\:${DEV}/driver/unbind || exit 1
echo "Device ${DEV} unbinded"
fi
echo $(lspci -n | grep ${DEV} | awk '{ split($3,a,":");
print(a[1],a[2]);}') > /sys/bus/pci/drivers/vfio-pci/new_id || exit 1
echo "Device ${DEV} binded to vfio-pci"
done
qemu-system-x86_64 \
-enable-kvm -M q35 -m 3072 -cpu
host,hv_relaxed,hv_vapic,hv_spinlocks=0x1000 -smp
4,sockets=1,cores=2,threads=2 -bios /usr/share/seabios/bios.bin \
-device
ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
-device vfio-pci,host=01:00.0,bus=root.1,multifunction=on,x-vga=on \
-device virtio-scsi-pci,id=scsi -vga none \
-device piix4-ide,bus=pcie.0,id=piix4-ide \
-drive file=/path/to/disk.img,id=disk,format=raw -device
ide-hd,bus=piix4-ide.0,drive=disk \
-drive file=/path/to/cd.iso,id=isocd -device
ide-cd,bus=piix4-ide.1,drive=isocd \
-boot menu=on -usb -usbdevice tablet -vnc 127.0.0.1:1
As you can see I have already tried many solutions, also tried to remove
root.1 and assign nvidia gpu directly to pcie.0. Unfortunately, w/o any
progress... All the time Windows displays exclamation mark and displays
error 43.
What else can I check/try and how can i solve this problem?
Is it possible for me, to use my 2nd GPU with KVM-based virtual machine?
I'm looking forward for your help and I would like to thank you for any
help and your time.
Best regards,
Rafal Kupiec
