Edit: the failure to find rom after restart issue actually occurs regardless of whether I start the nvidia driver or not. I've seen a lot of questions regarding that issue elsewhere so I'll start looking into it. On Feb 9, 2016 10:12 PM, "Ben J" <[email protected]> wrote:
> So I've made some progress. I pci stubbed all of my graphics devices and I > was able to succesfully bind my 580 and boot into Seabios. I attempted to > bind Nouveau to each of my other cards but it failed and crashes my system. > Clearly the drivers were fighting each other there. > > Oddly enough, the NVIDIA drivers actually worked fine with the VM. I'm > currently successfully running the VM on card 2 and Arch on the other 2 > cards. I did this by booting with all cards pci stubbed, using a standard > vfio bind script to bind card 2, starting the VM, and then binding card 1 > and 3 using an nvidia bind script (same script but replaces "vfio-pci" with > "nvidia"). Unfortunately this setup doesn't stand up to a restart of the > VM, it gives an error that it "Cannot read device rom" when I try to boot > the VM again. Clearly the nvidia driver is claiming some part of the guest > card and blocking access to its bios. > > As a next step I'm considering trying this arbiter patch: > https://lkml.org/lkml/2014/5/25/94 > > I noticed it's fairly old though, has that functionality already been > implemented into the kernel? Is there a way that I can check if it has on > my own? Are there any other nvidia specifc patches? I've seen ones for > significantly older drivers but I'm using the most up to date ones > currently. Thanks. > On Feb 8, 2016 11:50 PM, "Ben J" <[email protected]> wrote: > >> Switching over to Nouveau did the trick, at least for this issue. I was >> able to start the VM without hanging and had a different error which I >> fixed by passing it a ROM file for my GPU. I'm still not getting an output >> from the passed card though. Any ideas on how I would fix the arbiter issue >> then? Obviously I'd prefer to keep using the normal nvidia driver on linux >> if possible, although I suppose I could pass the other card to a Linux VM >> if all else fails. Thanks for the help so far, I'll continue working on >> this tomorrow. >> On Feb 8, 2016 11:19 PM, "Alex Williamson" <[email protected]> >> wrote: >> >>> On Mon, 8 Feb 2016 22:42:48 -0500 >>> Ben J <[email protected]> wrote: >>> >>> > Hi, I've been working to get VGA passthrough working for a Windows 7 >>> VM for >>> > the past few days. My setup is as follows: >>> > >>> > Processor: I7-3960X >>> > Graphics: 2 NVIDIA GTX 580's and one NVIDIA GT 420 >>> > Monitors: 1 Monitor hooked up to the 420, 1 monitor hooked up to both >>> 580's >>> > Motherboard: ASUS Rampage IV Extreme >>> > OS: Arch Linux >>> > Kernel: Standard, 4.4.1 >>> > >>> > My end goal is to bind one 580 to the VM and the other to Arch. The >>> 420 is >>> > just to allow X to start initially since I haven't found a way to >>> disable >>> > only one 580(They have the same vendor and model ID) at a time. I was >>> > initially following this guide >>> > < >>> https://www.reddit.com/r/pcmasterrace/comments/3lno0t/gpu_passthrough_revisited_an_updated_guide_on_how/ >>> >. >>> > My guest card was in its own IOMMU group and I'm not using integrated >>> > graphics so from what I understand I don't need the patched kernel. My >>> > cards bound to vfio-pci fine, but I got stuck when I realized that the >>> GTX >>> > 580 doesn't support UEFI booting. I changed over to booting with >>> Seabios >>> > but any time I've enabled "x-vga=on" QEMU just hangs. The QEMU monitor >>> > window freezes, there is no video output to my other monitor, and the >>> only >>> > way to get out of the window is to force close it in htop. It uses 0% >>> cpu >>> > during this. The only log put out into dmesg is "kvm: zapping shadow >>> pages >>> > for mmio generation wraparound" which looks fairly unrelated from my >>> > searches. >>> > >>> > I've tested the following things: >>> > Enabling vga cirrus and x-vga - Opens an empty, black graphical window, >>> > still hangs >>> > Enabling vga cirrus and no x-vga - Boots as expected, but obviously no >>> > passthrough >>> > Passing in a ROM for the graphics card - Hangs >>> > Enabling unsafe interrupts - Hangs >>> > Using linux-vfio kernel from AUR - Hangs >>> > Lowering RAM (based on this post >>> > <https://bugzilla.kernel.org/show_bug.cgi?id=107561>) - Hangs >>> > >>> > Here are my boot options, booting using rEFInd: >>> > "root=UUID=4accaa3c-6d16-40e9-bb95-e78d160962a5 ro intel_iommu=on >>> > vfio_iommu_type1.allow_unsafe_interrupts=1 >>> pci-stub.ids=10de:1080,10de:0e09" >>> > >>> > The two pci-stub ID's are for the GTX 580 and its audio card, it >>> disables >>> > both 580's. >>> > >>> > Here is my QEMU startup script: >>> > >>> > qemu-system-x86_64 -enable-kvm -m 1024 -cpu host,kvm=off \ >>> > -smp 4,sockets=1,cores=4,threads=1 \ >>> > -bios /usr/share/qemu/bios.bin -vga none \ >>> > -device vfio-pci,host=02:00.0,multifunction=on,x-vga=on \ >>> > -device vfio-pci,host=02:00.1 \ >>> > -drive >>> file=/mnt/LinuxStorage/VMs/VGAWin7.qcow2,id=disk,format=qcow2,if=virtio \ >>> > -boot menu=on \ >>> > >>> > At this point I don't know of any log files to look at so I'm >>> essentially >>> > taking shots in the dark. Are there any other logs that could be >>> useful? >>> > Has anyone else experienced issues like this? I've done a lot of >>> searching >>> > around and haven't come across anyone who has had this kind of hang >>> without >>> > an error message. Any ideas would be appreciated, thanks. >>> >>> Possibly a VGA arbiter deadlock with the host graphics not allowing the >>> the assigned graphics a chance to access VGA space. What happens if >>> you pci-stub all the NVIDIA devices (pci-stub.ids=10de:ffffffff)? X >>> obviously won't start on the host, but you should be able to login from >>> another system (VGA text mode on the 420 will get really dicey once the >>> VM starts, but the goal is just to test if the driver with X running on >>> the 420 is the problem). There used to be a problem with this using >>> the nvidia.ko driver and a workaround was to hack on the wrapper script >>> to prevent it from permanently grabbing the VGA arbiter resources. >>> That seems to be fixed for most folks, but perhaps it still exits with >>> the older driver you might be using with a 4-series card. Using >>> nouveau in the host may also avoid it. 5-series cards seem to cause >>> folks quite a bit of trouble when trying to assign them. Good luck, >>> >>> Alex >>> >>
_______________________________________________ vfio-users mailing list [email protected] https://www.redhat.com/mailman/listinfo/vfio-users
