Re: [vfio-users] NVidia code 43 with Winddows 10 guest - does NVidia have another trick up their sleeve?

2018-01-04 Thread Alex Williamson 
On Thu, 04 Jan 2018 17:29:28 +
Bronek Kozicki  wrote:

> Perhaps disabling ROM and adding spice as "boot" video card will help. That's 
> how I use my Quadro M5000 , apparently the only supported way of doing it. 
> Turns out spice is useful for other purposes, even if its Windows display 
> output is disabled.

No, Quadro and GeForce are different beasts, I don't know of any cases
of GeForce cards working as secondary to emulated graphics, while this
is really the only supported configuration for Quadro.  You'd need to
masquerade the GeForce as a Quadro to get that behavior.  Thanks,

Alex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] NVidia code 43 with Winddows 10 guest - does NVidia have another trick up their sleeve?

2018-01-04 Thread Bronek Kozicki 
Perhaps disabling ROM and adding spice as "boot" video card will help. That's 
how I use my Quadro M5000 , apparently the only supported way of doing it. 
Turns out spice is useful for other purposes, even if its Windows display 
output is disabled.


B.


Please excuse brevity, typing on a phone.

  Original Message  
From: alex.william...@redhat.com
Sent: 4 January 2018 4:32 pm
To: robert.j.dahlst...@gmail.com
Cc: vfio-users@redhat.com
Subject: Re: [vfio-users] NVidia code 43 with Winddows 10 guest - does NVidia 
have another trick up their sleeve?

On Wed, 3 Jan 2018 17:28:39 -0600
Robert Dahlstrom <robert.j.dahlst...@gmail.com> wrote:

> Hello, I have been trying to setup a Windows 10 guest on my Linux host,
> passing through an nvidia 1060. I've been having a very difficult time,
> getting the dreaded code 43. I haven't followed an individual guide, but
> have taken bits from https://wiki.archlinux.org/ind
> ex.php/PCI_passthrough_via_OVMF and Alex Williamson's when relevant among
> others.
> 
> I have tried just about every possible solution I've found, but have come
> up short.
> 
> One thing I'll point out right away: in my VM, I put two virtual disks. On
> one I installed windows, on the other I installed Debian 9. When booting
> into the linux guest OS, with the proprietary nvidia driver, **everything
> works**. When I boot into windows: code 43. Based on that, I think I have
> most of my stuff correctly setup, and I am just at the receiving end of
> nvidia treachery. I hope I'm wrong about that though.

Nobody else is reporting new treachery.

> Since both cards are nvidia cards, I use vfio.conf to prevent the nvidia
> driver from loading on the guest. The guest GPU is in an IOMMU group that
> consists solely of it, its HDMI audio device, and the Host Bridge/PCI
> Bridge devices. Both the video and audio devices are forwarded to the guest.
> 
> What I have tried:
> 
>  1) Seemingly every permutation of hidden kvm, vendor_id settings, hyperv
> settings, both i440fx and q35, BIOS and OVMF,

My suggestion for GeForce remains 440fx + OVMF.

>  2) My current situation has my guest GPU in the first PCIe slot, with my
> host GPU in the third. I think the first 2 slots share an IOMMU group,
> presumably for SLI or some such. This puts my guest GPU as the boot card,
> which is what the system BIOS and bootloader comes up on. I read that may
> cause trouble, and so I've tried swapping the cards.  This I don't believe
> works because there is a boatload of other stuff in the IOMMU group of the
> third slot (Group 0), such as my host SATA and USB controllers, that I
> can't forward.

Look for a BIOS update for your system and/or BIOS settings to enable
ACS, AMD added ACS support in one of the AGESA releases for Ryzen7
systems.  Using the host boot graphics device is very likely
contributing to your problems.  I would at a very minimum swap cards in
order to get a ROM dump from your GPU and use that rather than one from
techpowerup.

>  3) Passing disable_vga in vfio.conf, doesn't seem to do anything
> 
>  4) Setting MSI interrupt signaling in windows for both the video and audio
> devices.
> 
> What I have not tried:
> 
>  1) Switching which GPU I use for host and guest. If I can't use the 1060
> the whole thing is pointless. It may tell me if the boot GPU issue is the
> cause of my woe, however.

Yes, it very well could be.

>  2) I have read sometimes booting with the virtual graphics device (Spice +
> QXL) causes problems. Trying that, I have not been able to boot up at all
> without it. Presence of -nographics on qemu commandline seems to make no
> difference. When I blast the display from the config, the qemu process
> starts up and sits at 0% and never does anything, and I can't later remote
> desktop or ssh into the guest.

If you can't get a BIOS/UEFI splash screen out of the assigned
graphics, it's likely not going to work in the OS either.  You're
pretty much guaranteed a Code 43 so long as the emulated graphics are
present.

>  3) I have not yet successfully gone through the process of personally
> downloading the ROM from the card and using it. I have instead grabbed a
> ROM file from techpowerup.com, and verified that it has both BIOS and EFI
> portions. I can passthrough in linux regardless whether I use the ROM or
> not, and in Windows it makes no difference, code 43 for both. When I try to
> download the ROM myself I get the Invalid PCI ROM header signature error,
> and an input/output error.

That's because you're trying to dump the ROM from the boot graphics...

Thanks,
Alex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] NVidia code 43 with Winddows 10 guest - does NVidia have another trick up their sleeve?

2018-01-04 Thread Alex Williamson 
On Wed, 3 Jan 2018 17:28:39 -0600
Robert Dahlstrom  wrote:

> Hello, I have been trying to setup a Windows 10 guest on my Linux host,
> passing through an nvidia 1060. I've been having a very difficult time,
> getting the dreaded code 43. I haven't followed an individual guide, but
> have taken bits from https://wiki.archlinux.org/ind
> ex.php/PCI_passthrough_via_OVMF and Alex Williamson's when relevant among
> others.
> 
> I have tried just about every possible solution I've found, but have come
> up short.
> 
> One thing I'll point out right away: in my VM, I put two virtual disks. On
> one I installed windows, on the other I installed Debian 9. When booting
> into the linux guest OS, with the proprietary nvidia driver, **everything
> works**. When I boot into windows: code 43. Based on that, I think I have
> most of my stuff correctly setup, and I am just at the receiving end of
> nvidia treachery. I hope I'm wrong about that though.

Nobody else is reporting new treachery.
 
> Since both cards are nvidia cards, I use vfio.conf to prevent the nvidia
> driver from loading on the guest. The guest GPU is in an IOMMU group that
> consists solely of it, its HDMI audio device, and the Host Bridge/PCI
> Bridge devices. Both the video and audio devices are forwarded to the guest.
> 
> What I have tried:
> 
>  1) Seemingly every permutation of hidden kvm, vendor_id settings, hyperv
> settings, both i440fx and q35, BIOS and OVMF,

My suggestion for GeForce remains 440fx + OVMF.

>  2) My current situation has my guest GPU in the first PCIe slot, with my
> host GPU in the third. I think the first 2 slots share an IOMMU group,
> presumably for SLI or some such. This puts my guest GPU as the boot card,
> which is what the system BIOS and bootloader comes up on. I read that may
> cause trouble, and so I've tried swapping the cards.  This I don't believe
> works because there is a boatload of other stuff in the IOMMU group of the
> third slot (Group 0), such as my host SATA and USB controllers, that I
> can't forward.

Look for a BIOS update for your system and/or BIOS settings to enable
ACS, AMD added ACS support in one of the AGESA releases for Ryzen7
systems.  Using the host boot graphics device is very likely
contributing to your problems.  I would at a very minimum swap cards in
order to get a ROM dump from your GPU and use that rather than one from
techpowerup.
 
>  3) Passing disable_vga in vfio.conf, doesn't seem to do anything
> 
>  4) Setting MSI interrupt signaling in windows for both the video and audio
> devices.
> 
> What I have not tried:
> 
>  1) Switching which GPU I use for host and guest. If I can't use the 1060
> the whole thing is pointless. It may tell me if the boot GPU issue is the
> cause of my woe, however.

Yes, it very well could be.

>  2) I have read sometimes booting with the virtual graphics device (Spice +
> QXL) causes problems. Trying that, I have not been able to boot up at all
> without it. Presence of -nographics on qemu commandline seems to make no
> difference. When I blast the display from the config, the qemu process
> starts up and sits at 0% and never does anything, and I can't later remote
> desktop or ssh into the guest.

If you can't get a BIOS/UEFI splash screen out of the assigned
graphics, it's likely not going to work in the OS either.  You're
pretty much guaranteed a Code 43 so long as the emulated graphics are
present.

>  3) I have not yet successfully gone through the process of personally
> downloading the ROM from the card and using it. I have instead grabbed a
> ROM file from techpowerup.com, and verified that it has both BIOS and EFI
> portions. I can passthrough in linux regardless whether I use the ROM or
> not, and in Windows it makes no difference, code 43 for both. When I try to
> download the ROM myself I get the Invalid PCI ROM header signature error,
> and an input/output error.

That's because you're trying to dump the ROM from the boot graphics...

Thanks,
Alex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] NVidia code 43 with Winddows 10 guest - does NVidia have another trick up their sleeve?

2018-01-03 Thread Alex Kretzschmar 
I bet you've already tried this but looking through your xml file for the
guest I see a few inconsistencies when compared against the suggestions on
the Arch Wiki [here](https://wiki.archlinux.org/index.php/PCI_passthrough_
via_OVMF#.22Error_43:_Driver_failed_to_load.22_on_Nvidia_
GPUs_passed_to_Windows_VMs).

I found the important bits to the  tags. Hope this helps.

Alex

On 3 January 2018 at 23:28, Robert Dahlstrom 
wrote:

> Hello, I have been trying to setup a Windows 10 guest on my Linux host,
> passing through an nvidia 1060. I've been having a very difficult time,
> getting the dreaded code 43. I haven't followed an individual guide, but
> have taken bits from https://wiki.archlinux.org/ind
> ex.php/PCI_passthrough_via_OVMF and Alex Williamson's when relevant among
> others.
>
> I have tried just about every possible solution I've found, but have come
> up short.
>
> One thing I'll point out right away: in my VM, I put two virtual disks. On
> one I installed windows, on the other I installed Debian 9. When booting
> into the linux guest OS, with the proprietary nvidia driver, **everything
> works**. When I boot into windows: code 43. Based on that, I think I have
> most of my stuff correctly setup, and I am just at the receiving end of
> nvidia treachery. I hope I'm wrong about that though.
>
> Since both cards are nvidia cards, I use vfio.conf to prevent the nvidia
> driver from loading on the guest. The guest GPU is in an IOMMU group that
> consists solely of it, its HDMI audio device, and the Host Bridge/PCI
> Bridge devices. Both the video and audio devices are forwarded to the guest.
>
> What I have tried:
>
>  1) Seemingly every permutation of hidden kvm, vendor_id settings, hyperv
> settings, both i440fx and q35, BIOS and OVMF,
>
>  2) My current situation has my guest GPU in the first PCIe slot, with my
> host GPU in the third. I think the first 2 slots share an IOMMU group,
> presumably for SLI or some such. This puts my guest GPU as the boot card,
> which is what the system BIOS and bootloader comes up on. I read that may
> cause trouble, and so I've tried swapping the cards.  This I don't believe
> works because there is a boatload of other stuff in the IOMMU group of the
> third slot (Group 0), such as my host SATA and USB controllers, that I
> can't forward.
>
>  3) Passing disable_vga in vfio.conf, doesn't seem to do anything
>
>  4) Setting MSI interrupt signaling in windows for both the video and
> audio devices.
>
> What I have not tried:
>
>  1) Switching which GPU I use for host and guest. If I can't use the 1060
> the whole thing is pointless. It may tell me if the boot GPU issue is the
> cause of my woe, however.
>
>  2) I have read sometimes booting with the virtual graphics device (Spice
> + QXL) causes problems. Trying that, I have not been able to boot up at all
> without it. Presence of -nographics on qemu commandline seems to make no
> difference. When I blast the display from the config, the qemu process
> starts up and sits at 0% and never does anything, and I can't later remote
> desktop or ssh into the guest.
>
>  3) I have not yet successfully gone through the process of personally
> downloading the ROM from the card and using it. I have instead grabbed a
> ROM file from techpowerup.com, and verified that it has both BIOS and EFI
> portions. I can passthrough in linux regardless whether I use the ROM or
> not, and in Windows it makes no difference, code 43 for both. When I try to
> download the ROM myself I get the Invalid PCI ROM header signature error,
> and an input/output error.
>
>  4) Giving up and buying an AMD card.
>
> Any ideas what I'm missing? The oldest drivers from NVidia are from march
> 2017, most info I can find on this subject is from 2016 or earlier. Do the
> later drivers have additional checks that qemu is not able to fool at this
> time? Thanks for any help, and if you need any additional information I'll
> be happy to provide it.
>
> System information:
>
>  * Linux 4.9.0-3-amd64 (Debian 9)
>  * qemu-x86_64 version 2.8.1(Debian 1:2.8+dfsg-6+deb9u3)
>  * virsh -version: 3.0.0
>  * Processor: AMD Ryzen 7 1700X
>  * Host GPU: NVidia 740, evga
>  * Guest GPU: Nvidia 1060, Asus
>  * Guest NVidia Drivers: 378.78 - the oldest I can get off the website.
>
> /etc/modprobe.d/vfio.conf :
> options vfio-pci ids=10de:1c02,10de:10f1 disable_vga=1
>
> lspci -nnv: (This is after vfio is setup, vfio-pci drivers are in use as
> expected)
>
> 
> 21:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107
> [GeForce GT 740] [10de:0fc8] (rev a1) (prog-if 00 [VGA controller])
> Subsystem: eVga.com. Corp. GK107 [GeForce GT 740] [3842:3747]
> Flags: bus master, fast devsel, latency 0, IRQ 62
> Memory at f400 (32-bit, non-prefetchable) [size=16M]
> Memory at e000 (64-bit, prefetchable) [size=256M]
> Memory at f000 (64-bit, prefetchable) [size=32M]
> I/O ports at d000 [size=128]
>