On 24/02/16 23:18, Alex Williamson wrote:
> On Wed, Feb 24, 2016 at 2:45 PM, Daniel Pocock <[email protected]
> <mailto:[email protected]>> wrote:
>
>
>
> On 24/02/16 21:58, Alex Williamson wrote:
> >
> >
> > On Wed, Feb 24, 2016 at 1:26 PM, Bronek Kozicki <[email protected]
> <mailto:[email protected]>
> > <mailto:[email protected] <mailto:[email protected]>>> wrote:
> >
> > On 24/02/2016 15:41, Daniel Pocock wrote:
> >
> >
> >
> > Hi,
> >
> > I'm trying to use PCI passthrough to give an NVIDIA GPU to
> a VM with
> > qemu / KVM. I've summarized my environment below and the
> error
> > I get is
> > near the bottom. Any help would be appreciated.
> >
> > There are a few guides I've been referring to already:
> > https://wiki.debian.org/VGAPassthrough
> >
>
> https://www.pugetsystems.com/labs/articles/Multiheaded-NVIDIA-Gaming-using-Ubuntu-14-04-KVM-585/
> > https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF
> > https://bbs.archlinux.org/viewtopic.php?id=162768
> > http://www.linux-kvm.org/page/VGA_device_assignment
> >
> >
> >
> > Hi Daniel
> >
> >
> > I'm successfully passing through two Quadro M5000 (to two
> instances
> > of Windows 10), looking at
> >
>
> http://us.download.nvidia.com/Windows/Quadro_Certified/361.91/361.91-win10-quadro-grid-release-notes.pdf
> > (page 9) it is not obvious that this would work for either
> your K420
> > or mine M5000. One gotcha - when starting Windows I do not see
> boot
> > screen at all, only Windows logon scren after nVidia drivers had
> > loaded. Explanation is in Alex's email sent to this list on
> 6th Feb
> > 2016, subject "No boot screen on Quadro M5000?" - basically passed
> > through Quadro cards are meant to be secondary only (I use
> them as
> > only card, though). However this small quirk aside, this works for
> > me very well.
> >
> >
> >
> > Unfortunately K420 falls into the gap of unsupported and unknown
> between
> > GeForce, which are clearly unsupported, but we know how to make them
> > work, and Quardo K2000+, which are supported. Typically Quadro cards
> > don't work as primary like GeForce do, and the NVIDIA driver will
> pull a
> > Code 43 on them, so it probably requires some combination of secondary
> > plus hiding KVM & Hyper-v (or changing hyper-v vendor info) to make it
> > work. I'll note though that K420 is a GK107GL, as is the K2000, so
> > x-pci-device-id might be an interesting experiment. And of
> course, the
> > most important guide was missed from the list, the one at
> > vfio.blogspot.com <http://vfio.blogspot.com>
> <http://vfio.blogspot.com> ;) Thanks,
> >
>
>
> Thanks for all the replies about this
>
> What do you mean by primary vs secondary, is that documented somewhere?
> Do you mean I need some other card to be primary in the VM?
>
> I have two cards I can use for this VM, the K420 and a Quadro 2000.
>
> The Quadro 2000 is mentioned favorably in the Xen guide[1], am I likely
> to have more luck with that than the K420? The Quadro 2000 is a HP card
> and HP provides a BIOS download[2] too.
>
>
> In a secondary configuration, an emulated graphics is the primary
> graphics and is used for pre-boot, early boot, and install for the VM.
> Once accelerated drivers are installed and enabled, the primary graphics
> is generally disabled (automatically or manually depending on the guest
> OS). In a primary graphics configuration, the assigned GPU is the
> primary, typically exclusive, graphics for the VM. The 2000 is probably
> more likely to work because NVIDIA is potentially not explicitly
> preventing it from working like they do for the K420 and GeForce cards.
> It is a Fermi based card though, so it's not something we would
> actually claim as supported by NVIDIA. Add it to the VM just as you
> would any other PCI assigned device, keeping the emulated graphics
> configuration. Thanks,
>
OK, thanks for all the feedback, I've tried the following and the screen
is still blank (Code 12 in Device Manager, but at one stage it was Code
43 with the K420):
- updated from qemu in Debian jessie (2.1) to the version from
jessie-backports (2.5)
- using pci-stub on the host GPU (the K2200), X is not running at all on
the host
- adding the ignore_msrs=1 option to the kvm module, so it now uses:
options kvm allow_unsafe_assigned_interrupts=1 ignore_msrs=1
- adding rombar=0 to the -device config
- with the K420, I've tried adding
x-pci-vendor-id=0x10DE,x-pci-device-id=0x0FFE to the -device config and
Device Manager now thinks it is a K2000. It still says Code 12 though,
once it showed Code 43.
- I'm using emulated (qxl, also tried std) as primary display and
currently aiming to make either GPU (Quadro 2000 or K420) work as
secondary. I've also tried -nographic a few times with each.
- if I have the qemu monitor open and I try to supply the ROM to the
device, I see things like this at startup:
(qemu) dma: command 78 not supported
dma: command 78 not supported
dma: command 40 not supported
dma: command 40 not supported
- I can now get the OVMF shell on the emulated graphics (but not on
either GPU) and use the "pci" command to see the list of PCI devices,
the GPU is included in the list. OVMF won't boot the Windows 7 DVD or
the partition though, whenever I try to boot from either (using the boot
menu) it just fails and goes back to the OVMF shell, the error appears
too quickly for me to see what is happening.
Here are some relevant bits of the shell script I'm using to try
different permutations of qemu command lines:
#VGA="-nographic"
#VGA="-vga none"
#VGA="-vga std -vnc 0:15"
VGA="-vga qxl -vnc 0:15"
#ROMFILE="/root/NVIDIA-Quadro-2000-HP-70.06.3F.00.04_12320500.rom"
#ROMFILE="/root/nvidia-quadro-2000.rom"
#ROMFILE="/root/nvidia-k420.rom"
#ROMARG=",romfile=${ROMFILE}"
ROMARG=",rombar=0"
#ROMARG=",rombar=0,romfile=${ROMFILE}"
SIM_DEVICE=",x-pci-vendor-id=0x10DE,x-pci-device-id=0x0FFE"
# also consider x-pci-sub-vendor-id x-pci-sub-device-id?
#VGA_PASSTHROUGH="-device
ioh3420,bus=pcie.0,addr=1c.0,multifunction=on,port=1,chassis=1,id=root.1 \
#-device
vfio-pci,host=42:00.0,bus=root.1,addr=00.0,multifunction=on,x-vga=on${ROMARG}${SIM_DEVICE}
\
#-device vfio-pci,host=42:00.1,bus=root.1,addr=00.1"
VGA_PASSTHROUGH="-device
vfio-pci,host=42:00.0,bus=pcie.0,addr=09.0,multifunction=on,x-vga=on${ROMARG}${SIM_DEVICE}
\
-device vfio-pci,host=42:00.1,bus=pcie.0,addr=09.1"
Can anybody suggest anything else that I should try?
I've attached a list of cards listed by one of the application vendors,
are any of these easier to get going in a passthrough configuration? I
don't mind changing the card if an alternative will be easier to
configure or more reliable in the long term.
Regards,
Daniel
Card,Generation,Memory (MB),Type,Manufacturer
AMD FirePro V3900 (ATI FireGL),2011,1024,Workstation,AMD
AMD FirePro V4900 (FireGL V),2011,1024,Workstation,AMD
AMD FirePro V5900 (FireGL V),2011,2048,Workstation,AMD
AMD FirePro V7900 (FireGL V),2011,2048,Workstation,AMD
AMD FirePro W2100 (FireGL V), , ,Workstation,AMD
AMD FirePro W4100 (FireGL V), , ,Workstation,AMD
AMD FirePro W5000 (FireGL V), ,2048,Workstation,AMD
AMD FirePro W5100 (FireGL V), , ,Workstation,AMD
AMD FirePro W7000 (FireGL V), ,4096,Workstation,AMD
AMD FirePro W7100 (FireGL V), ,4096,Workstation,AMD
AMD FirePro W8000 (FireGL V), ,4096,Workstation,AMD
AMD FirePro W8100 (FireGL V), ,8192,Workstation,AMD
AMD FirePro W9000 (FireGL V), ,6144,Workstation,AMD
ATI FirePro V3800 (FireGL V),2010,512,Workstation,AMD
ATI FirePro V4800 (FireGL V),2010,1024,Workstation,AMD
ATI FirePro V5800 (FireGL V),2010,1024,Workstation,AMD
ATI FirePro V7800 (FireGL V),2010,2048,Workstation,AMD
ATI FirePro V8800 (FireGL V),2010,2048,Workstation,AMD
ATI FirePro V9800 (FireGL V),2010,4096,Workstation,AMD
Intel(R) HD Graphics P4600/P4700, , ,Workstation,INTEL
Intel(R) HD Graphics P530, , ,Workstation,INTEL
NVIDIA Quadro 1000M,Fermi,2048,Mobile,NVIDIA
NVIDIA Quadro 2000,Fermi,1024,Workstation,NVIDIA
NVIDIA Quadro 2000M,Fermi,2048,Mobile,NVIDIA
NVIDIA Quadro 3000M,Fermi,2048,Workstation,NVIDIA
NVIDIA Quadro 4000,Fermi,2048,Workstation,NVIDIA
NVIDIA Quadro 4000M,Fermi,2048,Mobile,NVIDIA
NVIDIA Quadro 410,Kepler,512,Workstation,NVIDIA
NVIDIA Quadro 5000,Fermi,2560,Workstation,NVIDIA
NVIDIA Quadro 5000M,Fermi,2048,Mobile,NVIDIA
NVIDIA Quadro 5010M,Fermi,4096,Workstation,NVIDIA
NVIDIA Quadro 600,Fermi,1024,Workstation,NVIDIA
NVIDIA Quadro 6000,Fermi,6144,Workstation,NVIDIA
NVIDIA Quadro K1000M,Kepler,2048,Mobile,NVIDIA
NVIDIA Quadro K1100M,Kepler,2048,Mobile,NVIDIA
NVIDIA Quadro K1200, , ,Workstation,NVIDIA
NVIDIA Quadro K2000,Kepler,2048,Workstation,NVIDIA
NVIDIA Quadro K2000M, , ,Mobile,NVIDIA
NVIDIA Quadro K2100M,Kepler,2048,Mobile,NVIDIA
NVIDIA Quadro K2200, ,4096,Workstation,NVIDIA
NVIDIA Quadro K2200M, ,2048,Mobile,NVIDIA
NVIDIA Quadro K3000M, , ,Mobile,NVIDIA
NVIDIA Quadro K3100M,Kepler,4096,Mobile,NVIDIA
NVIDIA Quadro K4000,Kepler,3072,Workstation,NVIDIA
NVIDIA Quadro K4000M, , ,Mobile,NVIDIA
NVIDIA Quadro K4100M,Kepler,4096,Mobile,NVIDIA
NVIDIA Quadro K420, ,1024,Workstation,NVIDIA
NVIDIA Quadro K4200, ,4096,Workstation,NVIDIA
NVIDIA Quadro K5000,Kepler,4096,Workstation,NVIDIA
NVIDIA Quadro K5000M,Kepler,4096,Mobile,NVIDIA
NVIDIA Quadro K5100M,Kepler,8192,Mobile,NVIDIA
NVIDIA Quadro K5200, ,8192,Workstation,NVIDIA
NVIDIA Quadro K600,Kepler,1024,Workstation,NVIDIA
NVIDIA Quadro K6000,Kepler,12288,Workstation,NVIDIA
NVIDIA Quadro K610M,Kepler,1024,Mobile,NVIDIA
NVIDIA Quadro K620, ,2048,Workstation,NVIDIA
NVIDIA Quadro M1000M,Maxwell,2048,Mobile,NVIDIA
NVIDIA Quadro M2000M,Maxwell,4096,Mobile,NVIDIA
NVIDIA Quadro M3000M,Maxwell,4096,Mobile,NVIDIA
NVIDIA Quadro M4000,Maxwell,8192,Workstation,NVIDIA
NVIDIA Quadro M4000M,Maxwell,4096,Mobile,NVIDIA
NVIDIA Quadro M5000,Maxwell,8192,Workstation,NVIDIA
NVIDIA Quadro M5000M,Maxwell,8192,Mobile,NVIDIA
NVIDIA Quadro M6000, ,12288,Workstation,NVIDIA
NVIDIA Quadro M600M,Maxwell,2048,Mobile,NVIDIA
_______________________________________________
vfio-users mailing list
[email protected]
https://www.redhat.com/mailman/listinfo/vfio-users
