Re: [vfio-users] "BUG: unable to handle kernel NULL pointer dereference at 00000000000000a8"

[Nick Sarnie]

On Fri, Feb 23, 2018 at 6:04 PM, Will Marler  wrote:
> I'm running Arch and updating recently has caused my VM to stop booting.
>
> Watching journalctl I see this when I try to start the VM:
>
> "Feb 23 15:04:07 haze kernel: BUG: unable to handle kernel NULL pointer
> dereference at 00a8
> Feb 23 15:04:07 haze kernel: IP: down_write+0x12/0x30
>
> and some more. Full pastebin is here: https://pastebin.com/8u70XLWJ
>
> This happens when I try to start my domain (called "Win10Full"). When it
> occurs, virsh locks up -- if I'm in virsh and do "list" it sits
> indefinitely. If I ctrl-C at this point, "End of file while reading data:
> Input/output error" gets written to the log, and my host cannot be rebooted
> without a physical power cycle.
>
> There is nothing that gets written to /var/log/libvirt/qemu/Win10Full.log.
>
> I'm not sure how to proceed; I figured this mailing list would be the first
> place to start. Any suggestions?
>
> Thanks in advance,
>
> Will
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>

Hi,

This is a bug in 4.15. You can either revert to 4.14, or revert this
patch and recompile the kernel:

https://patchwork.kernel.org/patch/10103043/raw/

Sarnex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] NPT Performance Bug Identified and Fixed

[Nick Sarnie]

Hi all,

I just wanted to send an update if you haven't heard. The cause of the
bad GPU performance with NPT enabled on AMD systems has been
identified and fixed.

The patch is located here: https://patchwork.kernel.org/patch/10027525/

I assume it will make it upstream soon. For now, you have to patch your kernel.

Thanks to Geoff and Paolo for their great work.

Sarnex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Minimal host with desktop guest and gaming guest. Is this possible?

[Nick Sarnie]

On Sat, Jul 15, 2017 at 8:59 PM, almer  wrote:
> Hi,
>
> my motivation: wattage concerns and not utilizing my hardware to the
> fullest. My setup is pretty much historically grown to 3 computers now. one
> for gaming (win10), one for storage/backup (debian stable) and one for
> desktop (latest xubuntu). And the desktop hardware is quite old and weak, so
> it bugs me that the good hardware is "wasted" on a windows system for games
> only.
> also with the next hardware purchase I would have to buy win10 because the
> free w10 upgrade from w7 will break. the short version: I won't buy w10 and
> have decided to switch to wine/steam-OS gaming. ditching windows completely,
> nearly all games that I play, run out of the box in wine/steam anyway.
>
> I've read that some setups utilize two graphics cards or IGP with a graphics
> card. Now I was wondering if both setups could be combined. A minimal stable
> distribution host for storage and providing what ever is needed for the
> guests on the IGP. first guest stable desktop for daily desktop needs and
> work with a passive graphics card for example a GT 710 passed through.
> second guest a more current distribution as wine/steam gaming guest with a
> more beefy graphics card passed through.
>
> Is this even possible? besides audio and mic config witch could get tricky I
> presume and keyboard+mouse for 3 systems.
>
> My hardware pick so far:
> - i5-7600k; mainly because of IGP, providing the 3. graphics card for the
> host.
> - ASUS Nvidia passive GT 710 and ASUS Nvidia GTX 1050 Ti STRIX for the two
> guests; Nvidia because I read that AMD has PCIE reset issues and I would
> like to stop and start guests with the PCIE cards more freely, without host
> reboots.
> - ASRock z270 Taichi; provides enough sata and usb for the storage/backup
> needs.
>
>
> So before I commit to this I would like to hear your thoughts on this.
>
> regards,
> Almer
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>

I believe only the AMD RX 200-300 have the reset issue, and only some
cards at that. My RX480 works fine. Also, if you plan to used the
closed source nvidia drivers, I heard you can't unbind cards or unload
the module, so you would have to reboot. Someone who has first hand
experience should confirm. AMDGPU unloads and unbinds cards fine. No
idea about nouveau.

Sarnex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] AMD Ryzen Nested Page Table Performance Oddities

[Nick Sarnie]

On Thu, Apr 20, 2017 at 12:57 PM, Graham Neville
 wrote:
> So turns out a Linux guest with Q35 and OVMF can work with npt on or off.
> However a Linux guest with i440fx only works with npt disabled.
>
> I'll follow the iommu mailing list for updates with regards to NPT affecting
> GPU FPS performance.
>
>
> On 18 Apr 2017 16:14, "Nick Sarnie"  wrote:
>>
>> On Tue, Apr 18, 2017 at 10:09 AM, Graham Neville
>>  wrote:
>> > Has there been any feedback on this at all?
>> >
>> > I'm still struggling to get a Windows and Linux VM working at the same
>> > time
>> > with npt=1. Annoying as I'm having to reboot in to a different kernel if
>> > I
>> > want to use one or the other.
>> >
>> > Are you able to share your Kernel command line and XML for the Linux
>> > host
>> > please?
>> >
>> >
>> > On 7 Apr 2017 22:54, "Nick Sarnie"  wrote:
>> >
>> > I can't reproduce that issue, I tested a Fedora VM with both npt 0 and
>> > 1, of course with the same GPU performance results as windows. I've
>> > mailed some AMD guys and KVM guys, and hopefully someone can at least
>> > figure out what is going on. Also, Alex has reproduced this himself on
>> > both Nvidia and AMD GPUs.
>> >
>> > On Fri, Apr 7, 2017 at 5:49 PM, Graham Neville 
>> > wrote:
>> >> Sarnex,
>> >>
>> >> I have similar oddities with npt with Ryzen. As you say with
>> >> kvm-amd.npt=0
>> >> the GPU performance is so much better (In my Witcher3 tests I'm going
>> >> from
>> >> 40fps to 75fps!). However with npt disabled the Windows10 VM slows down
>> >> an
>> >> awful lot in general tasks.
>> >>
>> >> I've also noticed that I can only run a Linux guest with kvm-amd.npt=0,
>> >> if
>> >> I
>> >> have it set to enabled then the Linux guest fails to start. I have the
>> >> same
>> >> issue even just trying to install Linux from an ISO, it will crash at
>> >> the
>> >> GRUB install menu.
>> >>
>> >> Hopefully someone knows a fix for this.
>> >>
>> >> ___
>> >> vfio-users mailing list
>> >> vfio-users@redhat.com
>> >> https://www.redhat.com/mailman/listinfo/vfio-users
>> >>
>> >
>> >
>>
>> Alex emailed Paolo, the KVM maintainer, some traces but he couldn't
>> find anything in them. Please follow the thread on the iommu mailing
>> list for more details. For now, I think we have to disable NPT.

What do you mean "work"? Do you not see the GPU performance
degradation with NPT enabled on Q35 and OVMF? All settings work for
me, it's just that enabling NPT nukes FPS.

Sarnex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] AMD Ryzen Nested Page Table Performance Oddities

[Nick Sarnie]

On Tue, Apr 18, 2017 at 10:09 AM, Graham Neville
 wrote:
> Has there been any feedback on this at all?
>
> I'm still struggling to get a Windows and Linux VM working at the same time
> with npt=1. Annoying as I'm having to reboot in to a different kernel if I
> want to use one or the other.
>
> Are you able to share your Kernel command line and XML for the Linux host
> please?
>
>
> On 7 Apr 2017 22:54, "Nick Sarnie"  wrote:
>
> I can't reproduce that issue, I tested a Fedora VM with both npt 0 and
> 1, of course with the same GPU performance results as windows. I've
> mailed some AMD guys and KVM guys, and hopefully someone can at least
> figure out what is going on. Also, Alex has reproduced this himself on
> both Nvidia and AMD GPUs.
>
> On Fri, Apr 7, 2017 at 5:49 PM, Graham Neville 
> wrote:
>> Sarnex,
>>
>> I have similar oddities with npt with Ryzen. As you say with kvm-amd.npt=0
>> the GPU performance is so much better (In my Witcher3 tests I'm going from
>> 40fps to 75fps!). However with npt disabled the Windows10 VM slows down an
>> awful lot in general tasks.
>>
>> I've also noticed that I can only run a Linux guest with kvm-amd.npt=0, if
>> I
>> have it set to enabled then the Linux guest fails to start. I have the
>> same
>> issue even just trying to install Linux from an ISO, it will crash at the
>> GRUB install menu.
>>
>> Hopefully someone knows a fix for this.
>>
>> ___
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>
>

Alex emailed Paolo, the KVM maintainer, some traces but he couldn't
find anything in them. Please follow the thread on the iommu mailing
list for more details. For now, I think we have to disable NPT.

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Frequent crashing with Overwatch

[Nick Sarnie]

Have you guys tried disabling MSRs?

On Fri, Apr 14, 2017 at 3:26 PM, Abdulla Bubshait  wrote:
> I would also like to chime in that this behaviour started popping up for me
> too. I am guessing it was introduced in a recent Overwatch update. I used to
> play the game fine on my setup until recently. Now, for some reason the game
> crashes after a while and I am unsure what could be causing it.
>
> On Tue, Apr 11, 2017 at 4:57 AM Jesse Kennedy  wrote:
>>
>> Hello,
>>
>> I've been experience very frequent crashing while playing Overwatch.
>> Usually, I can go for a while without crashing. But once it starts crashing,
>> it will continue to crash every few minutes until I restart the VM.
>> Sometimes I have to restart the host to fix it. The "crashes" are 70%
>> application crashing, 20% VM freezing, and 10% BSOD. The BSOD codes are
>> inconsistent, but one such code was MEMORY_MANAGEMENT. I play many other
>> games, but Overwatch is the only one that causes crashes.
>>
>> Nothing suspect in kernel logs or /var/log/libvirt/qemu/win10.log. Is
>> there anything I can do to get better visibility on what's happening so I
>> can troubleshoot? Could it be a hardware issue?
>>
>> qemu line from log:
>> LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
>> QEMU_AUDIO_DRV=none /usr/sbin/qemu-system-x86_64 -name
>> guest=win10,debug-threa
>> ds=on -S -object
>> secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-3-win10/master-key.aes
>> -machine pc-i440fx-2.6,accel=kvm,usb
>> =off,vmport=off,dump-guest-core=off -cpu
>> core2duo,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=0x1fff,hv_vendor_id=whatever,kvm=off
>> -drive file=/u
>> sr/share/ovmf/x64/ovmf_x64.bin,if=pflash,format=raw,unit=0,readonly=on
>> -drive file=/var/lib/libvirt/qemu/nvram/win10_VARS.fd,if=pflash,format=
>> raw,unit=1 -m 6000 -realtime mlock=off -smp 6,sockets=1,cores=3,threads=2
>> -uuid ca607e58-3a83-411c-b74c-65e4c2ca565d -display none -no-user-co
>> nfig -nodefaults -chardev
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-3-win10/monitor.sock,server,nowait
>> -mon chardev=charmonitor,
>> id=monitor,mode=control -rtc base=localtime,driftfix=slew -global
>> kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -global PIIX4_PM.disa
>> ble_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=off,strict=on -device
>> nec-usb-xhci,id=usb,bus=pci.0,addr=0x6 -device lsi,id=scsi0,bus=pci.0,
>> addr=0x9 -device ahci,id=sata0,bus=pci.0,addr=0xc -device
>> virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -device
>> usb-hub,id=hub0,bus=u
>> sb.0,port=1 -drive
>> file=/dev/nvme0n1,format=raw,if=none,id=drive-sata0-0-0,serial=windows,cache=none,aio=native
>> -device ide-hd,bus=sata0.0,dri
>> ve=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -drive
>> file=/dev/disk/by-id/ata-ST4000DX001-1CE168_Z3074HEX,format=raw,if=none,id=drive-virtio-dis
>> k0,cache=none,aio=native -device
>> virtio-blk-pci,scsi=off,bus=pci.0,addr=0xb,drive=drive-virtio-disk0,id=virtio-disk0
>> -netdev tap,fd=26,id=host
>> net0,vhost=on,vhostfd=29 -device
>> virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:5d:02:0a,bus=pci.0,addr=0x3
>> -chardev pty,id=charserial0 -
>> device isa-serial,chardev=charserial0,id=serial0 -chardev
>> spicevmc,id=charchannel0,name=vdagent -device
>> virtserialport,bus=virtio-serial0.0,nr
>> =2,chardev=charchannel0,id=channel0,name=com.redhat.spice.0 -chardev
>> socket,id=charchannel1,path=/var/lib/libvirt/qemu/win10.agent,server,nowa
>> it -device
>> virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0
>> -device usb-tablet,id=input2,
>> bus=usb.0,port=4 -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
>> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev spicevmc,id=ch
>> arredir0,name=usbredir -device
>> usb-redir,chardev=charredir0,id=redir0,bus=usb.0,port=1.1 -chardev
>> spicevmc,id=charredir1,name=usbredir -device
>> usb-redir,chardev=charredir1,id=redir1,bus=usb.0,port=1.2 -device
>> vfio-pci,host=01:00.0,id=hostdev0,bus=pci.0,addr=0xa -device vfio-pci,host=
>> 01:00.1,id=hostdev1,bus=pci.0,addr=0x2 -device
>> vfio-pci,host=00:14.0,id=hostdev2,bus=pci.0,addr=0xe -device
>> vfio-pci,host=00:1f.3,id=hostdev3,
>> bus=pci.0,addr=0xd -device
>> vfio-pci,host=00:1f.0,id=hostdev4,bus=pci.0,addr=0xf -device
>> vfio-pci,host=00:1f.2,id=hostdev5,bus=pci.0,addr=0x10
>> -device vfio-pci,host=00:1f.4,id=hostdev6,bus=pci.0,addr=0x11 -device
>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -object rng-random,id=
>> objrng0,filename=/dev/random -device
>> virtio-rng-pci,rng=objrng0,id=rng0,bus=pci.0,addr=0x8 -device
>> pvpanic,ioport=1285 -msg timestamp=on
>>
>> Specs:
>> Arch/Windows 10
>> 6700k, 3 cores/2 threads to guest, core2duo model
>> 32GB, 6GB to guest
>> 1080 GTX
>>
>> Kernel line:
>> rw quiet splash rd.modules-load=vfio-pci modules-load=vfio-pci
>> intel_iommu=on,igfx_off iommu=pt intremap=no_x2apic_optout
>> transparent_hugepage=never isolcpus=2-7
>>
>> Thanks
>> 

Re: [vfio-users] AMD Ryzen Nested Page Table Performance Oddities

[Nick Sarnie]

I can't reproduce that issue, I tested a Fedora VM with both npt 0 and
1, of course with the same GPU performance results as windows. I've
mailed some AMD guys and KVM guys, and hopefully someone can at least
figure out what is going on. Also, Alex has reproduced this himself on
both Nvidia and AMD GPUs.

On Fri, Apr 7, 2017 at 5:49 PM, Graham Neville  wrote:
> Sarnex,
>
> I have similar oddities with npt with Ryzen. As you say with kvm-amd.npt=0
> the GPU performance is so much better (In my Witcher3 tests I'm going from
> 40fps to 75fps!). However with npt disabled the Windows10 VM slows down an
> awful lot in general tasks.
>
> I've also noticed that I can only run a Linux guest with kvm-amd.npt=0, if I
> have it set to enabled then the Linux guest fails to start. I have the same
> issue even just trying to install Linux from an ISO, it will crash at the
> GRUB install menu.
>
> Hopefully someone knows a fix for this.
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] AMD Ryzen Nested Page Table Performance Oddities

[Nick Sarnie]

On Wed, Apr 5, 2017 at 11:32 PM, Nick Sarnie  wrote:
> Hi all,
>
> I'm seeing some strange issues regarding the Nested Page Table option
> of the kvm-amd module on my Ryzen system.
>
> First off, having the parameter enabled, as it is by default, clearly
> and consistently increases CPU performance inside the VM. I've tested
> this with a kernel compile, and it is around 5 times longer with npt
> disabled using tmpfs. Passing any PCI device through to the VM does
> not affect the CPU performance.
>
> Now, if I pass through my graphics card, we see some very strange
> issues. If npt is disabled, the FPS in 3D applications is between 2

Sorry, I mean 'if npt is enabled'.

> and 4 times lower than if npt is disabled. This is a consistent,
> cross-VM-platform result. In Windows 10, Dota 2 runs at around 35 FPS
> with npt enabled, and around 110 with npt disabled. In Fedora 25,
> Heaven benchmark results in an average of 32 FPS with npt enabled, and
> 48 with npt disabled. I haven't found a 3D application where there is
> not a significant GPU performance hit to enabling npt.
>
> I would like to try to understand this issue, as anyone wanting to use
> a GPU Passthrough config on Ryzen basically is forced to disable npt
> and suffer a CPU performance hit.
>
> I can provide any performance statistics, logs, or test any patches.
>
> Thank you,
> Sarnex

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] AMD Ryzen Nested Page Table Performance Oddities

[Nick Sarnie]

Hi all,

I'm seeing some strange issues regarding the Nested Page Table option of
the kvm-amd module on my Ryzen system.

First off, having the parameter enabled, as it is by default, clearly and
consistently increases CPU performance inside the VM. I've tested this with
a kernel compile, and it is around 5 times longer with npt disabled using
tmpfs. Passing any PCI device through to the VM does not affect the CPU
performance.

Now, if I pass through my graphics card, we see some very strange issues.
If npt is disabled, the FPS in 3D applications is between 2 and 4 times
lower than if npt is disabled. This is a consistent, cross-VM-platform
result. In Windows 10, Dota 2 runs at around 35 FPS with npt enabled, and
around 110 with npt disabled. In Fedora 25, Heaven benchmark results in an
average of 32 FPS with npt enabled, and 48 with npt disabled. I haven't
found a 3D application where there is not a significant GPU performance hit
to enabling npt.

I would like to try to understand this issue, as anyone wanting to use a
GPU Passthrough config on Ryzen basically is forced to disable npt and
suffer a CPU performance hit.

I can provide any performance statistics, logs, or test any patches.

Thank you,
Sarnex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups

[Nick Sarnie]

The ticket is still open, but it wouldn't fix a hang on startup. Do you see
any errors in dmesg?

On Mon, Mar 27, 2017 at 7:28 AM, Stano Lano  wrote:

> Hello,
>
> did you have any luck with the ASUS support.
> I should have the same ASUS Prime X370-Pro.
> But when I enable IOMMU I am not able to boot, when the IOMMU is disable I
> have no problem to boot.
>
> I am on BIOS 0511 and kernel 4.10.5
> Tried with 0502, 0504 & 0511 BIOSes but no luck with any of them.
> Also tried with Fedora 25, 25 beta & Ubuntu 17.04 beta. Same behavior.
>
> Thanks
> Stano
>
> --
>
>- *From*: Nick Sarnie 
>- *To*: Alex Williamson 
>- *Cc*: vfio-users 
>- *Subject*: Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups
>- *Date*: Fri, 10 Mar 2017 13:53:26 -0500
>
>
> Yeah, that is unfortunate.
>
> Thanks for helping with this issue. I've sent a ticket to Asus, but I'm
> not expecting much. Then again, I felt the same way and Gigabyte actually
> sent me a fixed bios, so who knows.
>
> I'll keep you updated.
>
> Thanks again,
> Sarnex
>
> On Fri, Mar 10, 2017 at 1:37 PM, Alex Williamson  com> wrote:
>
>> On Fri, Mar 10, 2017 at 11:05 AM, Nick Sarnie 
>> wrote:
>>
>>> Hi Alex,
>>>
>>> I don't see either of the options. I couldn't find Common Options at
>>> all, and here's a screenshot of the CBS settings:
>>>
>>> https://i.imgur.com/9hQUHX0.jpg
>>>
>>> Is this something I should ask Asus to add?
>>>
>>
>> I suppose it wouldn't hurt to try to start the discussion with Asus.  The
>> video I found was this one:
>>
>> https://youtu.be/pipR5xhrLo0?t=20
>>
>> At that start time you can see an NBIO Common Options menu on an ASRock
>> system, but I never saw him open it and I couldn't find any documentation
>> on what might be in there in an asrock mb manual (not an endorsement for
>> asrock, perhaps they just have a BIOS more similar to the AMD sample
>> implementation).  If AMD put it into a menu of debug options, it's really
>> no surprise that consumer firmware dropped it.  Too bad.  This feels like a
>> repeat of the difficulty we had trying to find motherboards that allowed
>> the IOMMU to be enabled when AMD-Vi came out.  I wonder if AVIC requires
>> yet another BIOS option that consumers will need to gamble with.
>>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Speed tips requested

[Nick Sarnie]

Sorry, is that a SCSI controller set to virtio type, or the virtio option
selected for the disk instead of SCSI? I've seen both recommended.

Thanks,
Sarnex

On Sun, Mar 26, 2017 at 12:58 PM, Zachary Boley  wrote:

> From what I've read Red Hat recommends virtio with raw on no cache with io
> thread due to the reasons listed above. Not sure about LVM but they did
> also say (or someone did) do not use BTRFS for keeping the image.
>
> The only optimization I would immediately recommended is to do
> host-passthrough as the CPU option in your guests xml. It's noticeably
> different assuming you haven't already done it
>
> On Mar 26, 2017 11:13 AM, "Bronek Kozicki"  wrote:
>
>> On 26/03/2017 15:31, Alex Williamson wrote:
>>
>>> On Sun, Mar 26, 2017 at 4:33 AM, Patrick O'Callaghan >> > wrote:
>>>
>>> On Sun, 2017-03-26 at 10:58 +0100, Bronek Kozicki wrote:
>>> > Assuming you use libvirt, make sure to use vCPU pinning. For disk
>>> access, try cache='writeback' io='threads'. If you switch to scsio-vfio,
>>> this will give you the ability to define queue length which might
>>> additionally improve IO. Also, try QCOW2 format for guest disk, it might
>>> enable some additional optimizations. However given you host seem to have
>>> little spare capacity, YMMV
>>>
>>> Thanks. I'm already using CPU pinning as I said. The disk options are
>>> both set to "hypervisor default" so I'll try changing them. I'd
>>> configured the guest disk as 'raw' assuming that would be faster than
>>> QCOW2 but I'll look into it.
>>>
>>>
>>>
>>> Generally the recommendation is to use raw (not sparse), LVM, or a block
>>> device for the best performance.  QCOW is never going to be as fast as
>>> these at writing unused blocks since it needs to go out and allocate new
>>> blocks from the underlying file system when this happens.
>>>
>>
>> I am not going to argue with your experience here, only wanted to note
>> that QCOW2 can be created with preallcation=falloc (or full, which is not
>> very useful) which means there will no extra allocations in the rutime.
>> Everything will be allocated at the moment of disk creation with qemu-img
>> create -f qcow2 -o preallcation=falloc 
>>
>>
>>
>> B.
>>
>> ___
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] kvm_get_msr_common: XXX callbacks suppressed

[Nick Sarnie]

No, it should be fine. It means the guest tried to access a CPU model
specific register, and KVM ignored it. This is usually a suggestion to fix
issues in the VM. I'm not sure how to actually disable the messages though.

On Wed, Mar 15, 2017 at 6:25 PM, sL1pKn07 SpinFlo 
wrote:

> Hi
>
> since, sincerely, idk, i get tons of this message in my dmesg
>
>
> [82704.254657] kvm_get_msr_common: 430 callbacks suppressed
> [82704.254660] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0xce
> [82704.254676] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x1fc
> [82704.254687] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x1a4
> [82704.254694] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x1a4
> [82704.254704] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x1ad
> [82704.254725] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x1a2
> [82704.254886] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x3f8
> [82704.254901] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x3f9
> [82704.254910] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x3fa
> [82704.254926] kvm [15180]: vcpu0, guest rIP: 0xf809eca717f3
> ignored rdmsr: 0x3fc
>
> Should i worry?
>
> └───╼  uname -a
> Linux sL1pKn07 4.9.11-1-ARCH #1 SMP PREEMPT Sun Feb 19 13:45:52 UTC
> 2017 x86_64 GNU/Linux
>
> └───╼  qemu-x86_64 -version
> qemu-x86_64 version 2.8.50 (v2.8.0-2135-gdd4d257821-dirty)
>
> └───╼  libvirtd --version
> libvirtd (libvirt) 3.2.0
> (from git)
>
> └───╼  cat /etc/modprobe.d/kvm.conf
> options kvm ignore_msrs=1
>
> └───╼  cat /etc/modprobe.d/vfio.conf
> options vfio_pci ids=10de:17c2,10de:0fb0
> options vfio_iommu_type1 allow_unsafe_interrupts=1
>
> 2x Xeon x5650
> EVGA SR-2
> host: Nvidia Titan Black
> Guest: Nvidia Titan X (maxwell)
> 48Gb ram
> SDD guest: Kingston SSDnow v300 240Gb (vfio driver)
> XML: https://sl1pkn07.wtf/paste/view/6b1ade1f
>
> gretings
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups

[Nick Sarnie]

Yeah, that is unfortunate.

Thanks for helping with this issue. I've sent a ticket to Asus, but I'm not
expecting much. Then again, I felt the same way and Gigabyte actually sent
me a fixed bios, so who knows.

I'll keep you updated.

Thanks again,
Sarnex

On Fri, Mar 10, 2017 at 1:37 PM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Fri, Mar 10, 2017 at 11:05 AM, Nick Sarnie 
> wrote:
>
>> Hi Alex,
>>
>> I don't see either of the options. I couldn't find Common Options at all,
>> and here's a screenshot of the CBS settings:
>>
>> https://i.imgur.com/9hQUHX0.jpg
>>
>> Is this something I should ask Asus to add?
>>
>
> I suppose it wouldn't hurt to try to start the discussion with Asus.  The
> video I found was this one:
>
> https://youtu.be/pipR5xhrLo0?t=20
>
> At that start time you can see an NBIO Common Options menu on an ASRock
> system, but I never saw him open it and I couldn't find any documentation
> on what might be in there in an asrock mb manual (not an endorsement for
> asrock, perhaps they just have a BIOS more similar to the AMD sample
> implementation).  If AMD put it into a menu of debug options, it's really
> no surprise that consumer firmware dropped it.  Too bad.  This feels like a
> repeat of the difficulty we had trying to find motherboards that allowed
> the IOMMU to be enabled when AMD-Vi came out.  I wonder if AVIC requires
> yet another BIOS option that consumers will need to gamble with.
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups

[Nick Sarnie]

Hi Alex,

I don't see either of the options. I couldn't find Common Options at all,
and here's a screenshot of the CBS settings:

https://i.imgur.com/9hQUHX0.jpg

Is this something I should ask Asus to add?

Thanks,
Sarnex

On Fri, Mar 10, 2017 at 8:50 AM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Thu, Mar 9, 2017 at 2:23 PM, Nick Sarnie 
> wrote:
>
>> Hi,
>>
>> You previously helped me with an issue on Z170 with the bios preventing
>> the quirk from kicking in on the root ports, and Gigabyte fixed the bios
>> for me. I must have thought that was more general.
>>
>> Anyway, here's the correct lspci.
>>
>> https://paste.pound-python.org/show/CgR8EioN3hZoVsp7KYrF/
>>
>> Let me know if you see anything,
>>
>
>
> Initial information from AMD suggests that ACS may in fact be enabled via
> a BIOS option.  I cannot confirm yet if this should be available on all
> versions of Ryzen.  Do you have an "NBIO Debug Options" selection available
> under AMD CBS?  The only video I've found shows various "Common Options"
> menus, including "NBIO Common Options", but the video didn't descend into
> that menu.  Maybe there's a global debug or advanced options flag that
> could make such a menu item appear.  Or perhaps it's not available in these
> initial firmware releases yet and vendors need pressure from AMD and/or
> users to make it available.
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups

[Nick Sarnie]

Hi,

You previously helped me with an issue on Z170 with the bios preventing the
quirk from kicking in on the root ports, and Gigabyte fixed the bios for
me. I must have thought that was more general.

Anyway, here's the correct lspci.

https://paste.pound-python.org/show/CgR8EioN3hZoVsp7KYrF/

Let me know if you see anything,

Thanks

On Thu, Mar 9, 2017 at 9:17 AM, Alex Williamson  wrote:

> On Thu, Mar 9, 2017 at 12:33 AM, Nick Sarnie 
> wrote:
>
>> I expect Alex to find it's a BIOS bug. Also, the ACS patch does allow the
>> groups to be made separating the GPUs(Wendell reported it didn't work), but
>> I haven't gotten around to testing actual passthrough to see if there's
>> some other issue(Maybe this is what he meant). I'll report if it works
>> tomorrow.
>>
>
> What evidence do you have to suggest a BIOS bug?  ACS is a hardware
> feature, so the only way I can imagine a BIOS bug being a factor is if the
> BIOS has some ability to disable it by dropping it from the capability
> chain.  If you add one more 'x' to make it lspci - then we can see the
> raw dump of the extended capability space, though ACS is a tiny capability,
> it might be impossible to positively identify it if the capability ID is
> masked.  My guess is that it's either an oversight or marketing distinction
> reserved for their server lines (we'll see once those ship), and that maybe
> we can work with them to create quirks to expose isolation if it exists.
> I've already started reaching out to AMD for this.  Thanks,
>
> Alex
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Ryzen/X370 Chipset and IOMMU Groups

[Nick Sarnie]

I expect Alex to find it's a BIOS bug. Also, the ACS patch does allow the
groups to be made separating the GPUs(Wendell reported it didn't work), but
I haven't gotten around to testing actual passthrough to see if there's
some other issue(Maybe this is what he meant). I'll report if it works
tomorrow.


On Thu, Mar 9, 2017 at 2:14 AM, taii...@gmx.com  wrote:

> On 03/09/2017 12:30 AM, Nick Sarnie wrote:
>
> Hi all,
> My Ryzen stuff finally arrived, so I can help get GPU Passthrough working.
> My CPU is the 1800X and my motherboard is the Asus Prime X370-Pro.
>
> I'm on kernel 4.10.1.
>
> The IOMMU groups are indeed 
> bad:https://paste.pound-python.org/show/LGsNqdfIO3xWNNv9Mslr/
>
> As you can see, the graphics cards are grouped together.
>
> Alex, I've linked lspci -xxx -vvv and dmesg below. Please let me know if
> you see anything, or need any more information.
>
> lspci: https://paste.pound-python.org/show/iOgBaLrDZEGefuJjZlgy/
>
> dmesg: https://paste.pound-python.org/show/3eevItiXBhgCjwDXVxQX/
>
>
> Thanks,
> Sarnex
>
>
>
>
> ___
> vfio-users mailing 
> listvfio-users@redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
>
> I hope this is just a bug and that AMD isn't going the intel route with
> making ACS a "server" feature.
>
> In comparison, my bulldozer rig with a SR5690 chipset where there is a
> turkey for every pot.
>
> /sys/kernel/iommu_groups/
> /sys/kernel/iommu_groups/7
> /sys/kernel/iommu_groups/7/devices
> /sys/kernel/iommu_groups/7/devices/:00:14.4 (7 is a pci bridge and
> onboard pci device)
> /sys/kernel/iommu_groups/7/devices/:08:01.0
> /sys/kernel/iommu_groups/5
> /sys/kernel/iommu_groups/5/devices
> /sys/kernel/iommu_groups/5/devices/:00:14.2
> /sys/kernel/iommu_groups/3
> /sys/kernel/iommu_groups/3/devices
> /sys/kernel/iommu_groups/3/devices/:00:14.0
> /sys/kernel/iommu_groups/11
> /sys/kernel/iommu_groups/11/devices
> /sys/kernel/iommu_groups/11/devices/:04:00.0
> /sys/kernel/iommu_groups/1
> /sys/kernel/iommu_groups/1/devices
> /sys/kernel/iommu_groups/1/devices/:00:12.2 (usb controller set)
> /sys/kernel/iommu_groups/1/devices/:00:12.0
> /sys/kernel/iommu_groups/1/devices/:00:12.1
> /sys/kernel/iommu_groups/8
> /sys/kernel/iommu_groups/8/devices
> /sys/kernel/iommu_groups/8/devices/:00:14.5
> /sys/kernel/iommu_groups/6
> /sys/kernel/iommu_groups/6/devices
> /sys/kernel/iommu_groups/6/devices/:00:14.3
> /sys/kernel/iommu_groups/4
> /sys/kernel/iommu_groups/4/devices
> /sys/kernel/iommu_groups/4/devices/:00:14.1
> /sys/kernel/iommu_groups/12
> /sys/kernel/iommu_groups/12/devices
> /sys/kernel/iommu_groups/12/devices/:05:00.0 (video card and audio
> device)
> /sys/kernel/iommu_groups/12/devices/:05:00.1
> /sys/kernel/iommu_groups/2
> /sys/kernel/iommu_groups/2/devices
> /sys/kernel/iommu_groups/2/devices/:00:13.1 (other usb controller set)
> /sys/kernel/iommu_groups/2/devices/:00:13.2
> /sys/kernel/iommu_groups/2/devices/:00:13.0
> /sys/kernel/iommu_groups/10
> /sys/kernel/iommu_groups/10/devices
> /sys/kernel/iommu_groups/10/devices/:03:00.0
> /sys/kernel/iommu_groups/0
> /sys/kernel/iommu_groups/0/devices
> /sys/kernel/iommu_groups/0/devices/:00:11.0
> /sys/kernel/iommu_groups/9
> /sys/kernel/iommu_groups/9/devices
> /sys/kernel/iommu_groups/9/devices/:01:00.0
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Ryzen/X370 Chipset and IOMMU Groups

[Nick Sarnie]

Hi all,
My Ryzen stuff finally arrived, so I can help get GPU Passthrough working.
My CPU is the 1800X and my motherboard is the Asus Prime X370-Pro.

I'm on kernel 4.10.1.

The IOMMU groups are indeed bad:
https://paste.pound-python.org/show/LGsNqdfIO3xWNNv9Mslr/

As you can see, the graphics cards are grouped together.

Alex, I've linked lspci -xxx -vvv and dmesg below. Please let me know if
you see anything, or need any more information.

lspci: https://paste.pound-python.org/show/iOgBaLrDZEGefuJjZlgy/

dmesg: https://paste.pound-python.org/show/3eevItiXBhgCjwDXVxQX/


Thanks,
Sarnex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] audio with IGD

[Nick Sarnie]

Can you send lspci?

On Wed, Feb 22, 2017 at 2:14 PM, José A. Gómez  wrote:

> Hello,
>
> How do you get audio when you are using the IGD? I have passed the audio
> controller (device 00:03.0) and in Windows 10 is detected as Intel Display
> Audio, but under Sound preferences, in playback section there is no audio
> devices installed.
>
> Regards
>
> Jose
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Freesync support

[Nick Sarnie]

I also need to hide the hypervisor for the "Display" tab to show up. I
assume it's a bug and not intentional. Hopefully they fix it.

On Fri, Feb 17, 2017 at 10:35 AM, Øyvind Aasen  wrote:

> It didn't seem like that to me. I had to do a reinstall off the VM a couple
> days ago. Because I had some issues with my computer, downloaded the newest
> relive driver 17.2.1 from the webpage and installed it. I could not find
> the display tab anywhere in the program, I did not try the 32-bit version
> of the app because I didn't find anything about the bug you mentioned.
>
> After setting the hidden state to on and reinstalling the driver from the
> same download did it show in the Settings app. My guess is that either the
> Relive installer or Crimson installer changed something in the display
> driver installation braking it if you run in a VM.
>
> Another issue I had was that my display showed up as Generic PnP monitor
> instead off the model name/number checking the device information in
> windows. Reinstalling the drivers fixed this issue.
>
> Øyvind
>
> On Fri, Feb 17, 2017 at 03:59:09PM +0100, Quentin Deldycke wrote:
> > I have freesync enabled on this page since a long time (r9 290).
> > I have kvm and hyperv features enabled. And the setting is still here.
> >
> > In the previous version of Relive drivers, there were a bug when the
> radeon app
> > in 64bit doesn't show the display tab.
> > Killing and lunching the 32b app corrects this.
> > Using the 17.2.1 drivers corrects this issue.
> >
> > --
> > Deldycke Quentin
> >
> >
> > On 17 February 2017 at 15:55, Øyvind Aasen  wrote:
> >
> > Hi,
> >
> > I verified it by opening up Radeon Settings, selecting the display
> tab,
> > and then checking that the freesync option is on.
> >
> > You will need to reinstall the drivers after hiding the hypervisor
> from the
> > VM if you can't see the display tab in Radeon Settings.
> >
> > I have not had any problems with unstable FPS but that might be
> related to
> > the games I play.
> >
> > Øyvind
> >
> > On Fri, Feb 17, 2017 at 12:44:44PM +0100, Quentin Deldycke wrote:
> > > Hi,
> > >
> > > I also have a freesync screen connected to my card.
> > >
> > > How you can verify that freesync is "really" activated?
> > >
> > > I found the latest amd driver to be not stable at all in term of
> FPS.
> > >
> > >
> > > --
> > > Deldycke Quentin
> > >
> > >
> > > On 17 February 2017 at 10:19, Øyvind Aasen 
> wrote:
> > >
> > > Hi,
> > >
> > > I have been fiddling around a bit to get AMD freesync to work
> on my
> > setup,
> > > and this is how I got it to work.
> > >
> > > The solution is the same workaround used by Nvidia users use
> to hide
> > that
> > > the machine is running inside a VM, the steps taken from Alex
> blog
> > post are
> > >
> > > "The GeForce card is nearly as easy, but we first need to work
> around
> > some
> > > of the roadblocks Nvidia has put in place to prevent you from
> using
> > the
> > > hardware you've purchased in the way that you desire (and by my
> > reading
> > > conforms to the EULA for their software, but IANAL).  For this
> step
> > we
> > > again need to run virsh edit on the VM.  Within the 
> > section,
> > > remove everything between the  tags, including the tags
> > > themselves.  In their place add the following tags:
> > >
> > > 
> > >   
> > > 
> > >
> > > Additionally, within the  tag, find the timer named
> > hypervclock,
> > > remove the line containing this tag completely.  Save and exit
> the
> > edit
> > > session."
> > >
> > > The issue seems to be that the crimsons installer does not
> install
> > the
> > > display driver part of the graphics card driver when it
> detects that
> > it is
> > > running inside a VM.
> > >
> > > Øyvind
> > >
> > > ___
> > > vfio-users mailing list
> > > vfio-users@redhat.com
> > > https://www.redhat.com/mailman/listinfo/vfio-users
> > >
> > >
> > >
> >
> > > ___
> > > vfio-users mailing list
> > > vfio-users@redhat.com
> > > https://www.redhat.com/mailman/listinfo/vfio-users
> >
> >
> >
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] AMD RX480 Not working with windows

[Nick Sarnie]

I couldn't get drivers installed on Q35 with my 480, and I think Alex
recommends i1440fx but he'd have to jump in to confirm.

On Sat, Feb 4, 2017 at 1:47 PM, Janusz  wrote:

> On 02/04/2017 12:17 PM, Javier Celaya wrote:
>
> This is my command line:
> qemu-system-x86_64
> -nodefaults -no-user-config
> -enable-kvm -m 8192
> -cpu Opteron_G1,+cx16,+lahf_lm,kvm=off
> -M q35
> -smp 4,sockets=1,cores=4,threads=1
> -realtime mlock=on -rtc base=localtime,driftfix=slew
> -drive file="OVMF_CODE-pure-efi.fd",i
> f=pflash,format=raw,unit=0,readonly=on
> -drive file="OVMF_VARS-pure-efi.fd",if=pflash,format=raw,unit=1
> -drive file="$HD",id=disk1,if=virtio,cache=none
> -netdev bridge,id=netuser,br=br0
> -device virtio-net-pci,netdev=netuser,
> id=net0,mac=52:54:00:72:75:9e
> -device ich9-intel-hda,id=sound0
> -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0
> -device ioh3420,bus=pcie.0,multifunction=on,port=1,chassis=1,id=
> root.0
> -usbdevice $JOYPAD
> -usbdevice $KEYBOARD
> -usbdevice $MOUSE
> -mem-path $MEM_PATH
>
> -nographic -vga none
> -device vfio-pci,host=01:00.0,id=hostd
> ev0,bus=root.0,multifunction=on
> -device vfio-pci,host=01:00.1,id=hostdev1,bus=pcie.0
>
> I see some differences with yours:
> - @Janusz: Q35 works fine for me, as does i440fx
>
> ok, I saw on #kvm and #vfio-users on freenode multiple reports that it
> doesn't work with q35 and when users switched to i440fx it started working.
>
>
>
>
> - Why are you using a ROM file? Your RX480 does not support UEFI?
> - You are not explicitly attaching the graphics card and the HDMI sound
> card to the PCI-E bus, not sure if that could be a problem.
> - "-cpu host" is known to have problems with Windows 10 guests. If it
> hangs, try using my config or "-cpu core2duo".
>
> Cheers
>
>
> Javi
>
> 2017-02-03 12:41 GMT+01:00 Janusz :
>
>> Q35 doesn't work with polaris gpus, try i440fx
>>
>> On 02/03/2017 08:29 AM, ryanly...@sigaint.org wrote:
>> > Hi
>> >
>> > I have a working configuration with ubuntu guest. I tried using 1G pages
>> > and found that I need at least 4G for the linux driver to load. I am
>> using
>> > the standard XFX roms that I used to flash the BIOS which are signed and
>> > not modded. I have tried taking the working native windows configuration
>> > and booting that up via qemu and after it installs the acpi drivers is
>> > fails with the same reason..
>> >
>> > This command line works for macOs sierra (unacellerated but it does the
>> > full screen 1920x2060 and 3840x2060) and works with glamour enabled
>> Ubuntu
>> > guest. I also found that you get a corrupt screen if you don't enable
>> > glamour under gentoo.
>> >
>> > img=win
>> > bios=/mnt/work/vm/etc
>> > mem="-m 4G -mem-path /mnt/hugepg1g"
>> > #cpus="-cpu host,nx=on,kvm=on,vendor=GenuineIntel -smp
>> > cpus=4,sockets=1,cores=2,threads=2"
>> > cpus="-cpu
>> > host,+kvmclock,+kvmclock-stable-bit,nx=on,kvm=on,vendor=GenuineIntel
>> -smp
>> > cpus=4,sockets=1,cores=2,threads=2"
>> > monitor="-monitor none"
>> > # rombar=1 = good
>> > vfio="-device
>> > vfio-pci,host=01:00.0,x-vga=off,multifunction=on,rombar=1,ro
>> mfile=/${bios}/xfx-rx480-stock.rom,addr=10.0
>> > -device vfio-pci,host=01:00.1,addr=10.1"
>> > machine="-machine q35,accel=kvm,usb=off,vmport=off,kernel_irqchip=on"
>> > net="-netdev user,id=vmnic,restrict=n,hostfwd=tcp::1022-:22 -device
>> > virtio-net-pci,romfile=,netdev=vmnic"
>> > display="-display none"
>> > graphic=""
>> > sound=""
>> > inputs=""
>> > image_file="windoze.img"
>> > serial="-serial stdio"
>> >
>> > if [ ! -z "${1}" ] ; then
>> >image_file="${1}"
>> > fi
>> >
>> > for dev in :01:00.0 :01:00.1 ; do
>> > vendor="$(cat /sys/bus/pci/devices/${dev}/vendor)"
>> > device="$(cat /sys/bus/pci/devices/${dev}/device)"
>> > if [ -e "/sys/bus/pci/devices/${dev}/driver" ]; then
>> > echo "${dev}" > "/sys/bus/pci/devices/${dev}/driver/unbind"
>> >  fi
>> >  echo "${vendor}" "${device}" > /sys/bus/pci/drivers/vfio-pci/
>> new_id
>> > done
>> >
>> > inputs="-usbdevice host::"
>> >
>> > if [ ! -f "${image_file}" ] ; then
>> > loop_dev="$(losetup --find)"
>> > qemu-img create -f raw "${image_file}" 12G
>> > sgdisk --zap-all "${image_file}"
>> > sgdisk --set-alignment=1 --new=1:34:131071 "${image_file}"
>> > sgdisk --typecode 1:ef00 "${image_file}"
>> > sgdisk --set-alignment=1 --new=2:131072: "${image_file}"
>> > sgdisk --typecode 2:8300 "${image_file}"
>> > sgdisk --partition-guid=1:R
>> > sgdisk --partition-guid=2:R
>> > gdisk -l  "${image_file}"
>> > tmpMount="$(mktemp -d)"
>> > losetup --partscan "${loop_dev}" "${image_file}"
>> > mkfs.vfat -f1 -F12 "${loop_dev}"p1
>> > mount -orw "${loop_dev}"p1 "${tmpMount}"
>> > mkdir -p "${tmpMount}"/EFI/BOOT
>> > c

Re: [vfio-users] AMD RX480 Not working with windows

[Nick Sarnie]

I only needed core2duo when installing the anniversary update. I heard it
tries to flash CPU microcode and QEMU blocks this so the update fails, but
it doesn't try to install it for core2duo.

On Sat, Feb 4, 2017 at 1:42 PM, Nick Sarnie  wrote:

> I only needed core2duo when installing the anniversary update. I heard it
> tries to flash CPU microcode and QEMU blocks this so the update fails, but
> it doesn't try to install it for core2duo.
>
> On Sat, Feb 4, 2017 at 6:17 AM, Javier Celaya  wrote:
>
>> This is my command line:
>> qemu-system-x86_64
>> -nodefaults -no-user-config
>> -enable-kvm -m 8192
>> -cpu Opteron_G1,+cx16,+lahf_lm,kvm=off
>> -M q35
>> -smp 4,sockets=1,cores=4,threads=1
>> -realtime mlock=on -rtc base=localtime,driftfix=slew
>> -drive file="OVMF_CODE-pure-efi.fd",i
>> f=pflash,format=raw,unit=0,readonly=on
>> -drive file="OVMF_VARS-pure-efi.fd",if=pflash,format=raw,unit=1
>> -drive file="$HD",id=disk1,if=virtio,cache=none
>> -netdev bridge,id=netuser,br=br0
>> -device virtio-net-pci,netdev=netuser,
>> id=net0,mac=52:54:00:72:75:9e
>> -device ich9-intel-hda,id=sound0
>> -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0
>> -device ioh3420,bus=pcie.0,multifuncti
>> on=on,port=1,chassis=1,id=root.0
>> -usbdevice $JOYPAD
>> -usbdevice $KEYBOARD
>> -usbdevice $MOUSE
>>
>> -mem-path $MEM_PATH
>>
>> -nographic -vga none
>> -device vfio-pci,host=01:00.0,id=hostd
>> ev0,bus=root.0,multifunction=on
>> -device vfio-pci,host=01:00.1,id=hostdev1,bus=pcie.0
>>
>> I see some differences with yours:
>> - @Janusz: Q35 works fine for me, as does i440fx
>> - Why are you using a ROM file? Your RX480 does not support UEFI?
>> - You are not explicitly attaching the graphics card and the HDMI sound
>> card to the PCI-E bus, not sure if that could be a problem.
>> - "-cpu host" is known to have problems with Windows 10 guests. If it
>> hangs, try using my config or "-cpu core2duo".
>>
>> Cheers
>>
>>
>> Javi
>>
>> 2017-02-03 12:41 GMT+01:00 Janusz :
>>
>>> Q35 doesn't work with polaris gpus, try i440fx
>>>
>>> On 02/03/2017 08:29 AM, ryanly...@sigaint.org wrote:
>>> > Hi
>>> >
>>> > I have a working configuration with ubuntu guest. I tried using 1G
>>> pages
>>> > and found that I need at least 4G for the linux driver to load. I am
>>> using
>>> > the standard XFX roms that I used to flash the BIOS which are signed
>>> and
>>> > not modded. I have tried taking the working native windows
>>> configuration
>>> > and booting that up via qemu and after it installs the acpi drivers is
>>> > fails with the same reason..
>>> >
>>> > This command line works for macOs sierra (unacellerated but it does the
>>> > full screen 1920x2060 and 3840x2060) and works with glamour enabled
>>> Ubuntu
>>> > guest. I also found that you get a corrupt screen if you don't enable
>>> > glamour under gentoo.
>>> >
>>> > img=win
>>> > bios=/mnt/work/vm/etc
>>> > mem="-m 4G -mem-path /mnt/hugepg1g"
>>> > #cpus="-cpu host,nx=on,kvm=on,vendor=GenuineIntel -smp
>>> > cpus=4,sockets=1,cores=2,threads=2"
>>> > cpus="-cpu
>>> > host,+kvmclock,+kvmclock-stable-bit,nx=on,kvm=on,vendor=GenuineIntel
>>> -smp
>>> > cpus=4,sockets=1,cores=2,threads=2"
>>> > monitor="-monitor none"
>>> > # rombar=1 = good
>>> > vfio="-device
>>> > vfio-pci,host=01:00.0,x-vga=off,multifunction=on,rombar=1,ro
>>> mfile=/${bios}/xfx-rx480-stock.rom,addr=10.0
>>> > -device vfio-pci,host=01:00.1,addr=10.1"
>>> > machine="-machine q35,accel=kvm,usb=off,vmport=off,kernel_irqchip=on"
>>> > net="-netdev user,id=vmnic,restrict=n,hostfwd=tcp::1022-:22 -device
>>> > virtio-net-pci,romfile=,netdev=vmnic"
>>> > display="-display none"
>>> > graphic=""
>>> > sound=""
>>> > inputs=""
>>> > image_file="windoze.img"
>>> > serial="-serial stdio"
>>> >
>>> > if [ ! -z "${1}" ] ; then
>>&

Re: [vfio-users] VGA Passthrough fails without NoSnoop patch

[Nick Sarnie]

Have you tried a BIOS install and setting x-vga=on? With my 480, there is
no UEFI image or it got corrupted somehow, so I need to pass a UEFI vbios I
downloaded to see the screen if I use UEFI. With BIOS, x-vga=on works.



On Sun, Jan 29, 2017 at 2:55 PM, Javier Celaya  wrote:

> Hello list
>
> I've been looking for it, but I cannot find a solution to this problem. I
> have a working Windows 10 VM with UEFI boot, which I pass an AMD Radeon RX
> 480. It also worked with a NVIDIA GTX 550Ti I had before. The thing is, it
> only works with a QEMU 2.2 patched with the NoSnoop fix. With any later
> release of QEMU, or without the patch, the VM:
> - Throws an Error 43 with the NVIDIA card.
> - Goes black on boot and reboots some seconds later with the AMD card
> The vfio FAQ says that patch is not needed with a later release of QEMU,
> so I'm wondering what I'm doing wrong...
>
> My hardware is an Intel i5 2500, ASUS motherboard with a Z68 chipset, 16GB
> of RAM. Both graphics cards appear in their own iommu group with the HDMI
> sound card.
>
> My QEMU command line is:
>
> src/qemu/build-2.2-NoSnoop/x86_64-softmmu/qemu-system-
> x86_64
>
> -nodefaults -no-user-config
>
>
> -enable-kvm -m 8192
>
>
> -cpu Opteron_G1,+cx16,+lahf_lm,kvm=off
>
>
> -M q35
>
>
> -smp 4,sockets=1,cores=4,threads=1
> -realtime mlock=on -rtc base=localtime,driftfix=slew
> -drive file="OVMF_CODE-pure-efi.fd",if=pflash,format=raw,unit=0,
> readonly=on
> -drive file="OVMF_VARS-pure-efi.fd",if=pflash,format=raw,unit=1
> -drive file="$HD",id=disk1,if=virtio,cache=none
> -netdev bridge,id=netuser,br=br0
> -device virtio-net-pci,netdev=netuser,
> id=net0,mac=52:54:00:72:75:9e
> -device ich9-intel-hda,id=sound0
> -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0
> -device ioh3420,bus=pcie.0,multifunction=on,port=1,
> chassis=1,id=root.0
> -usbdevice $JOYPAD
> -usbdevice $KEYBOARD
> -usbdevice $MOUSE
> -mem-path $MEM_PATH
>
> -nographic -vga none
> -device vfio-pci,host=01:00.0,id=hostdev0,bus=root.0,
> multifunction=on,x-vga=on
> -device vfio-pci,host=01:00.1,id=hostdev1,bus=pcie.0
>
> I have tried, with the same results:
> - Using the pc-i440fx machine
> - Connecting the HDMI sound card to another bus
> - Not connecting the HDMI sound card at all
> - Using -cpu host (Windows 10 hangs with this configuration)
> - Using -cpu core2duo
>
> Any other idea?
> Thank you
>
> Javi
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] No HDMI audio from AMD RX480 when passed through to Windows 8.1 VM.

[Nick Sarnie]

Did you set x-vga=on for the BIOS VM?

On Sat, Jan 21, 2017 at 6:37 PM, Luke Yelavich  wrote:

> On Wed, Jan 18, 2017 at 08:21:05PM AEDT, Luke Yelavich wrote:
> > Greetings.
> > I'm writing with a rather annoying problem. I am unable to get audio
> output
> > through HDMI on an AMD RX480 card that is being passed through to a
> Windows
> > 8.1 VM.
>
> So, I decided to create the VM again but using legacy BIOS instead of
> UEFI. Low and behold, the HDMI audio works fine, although I have to leave a
> spice/QXL video card attached to the VM otherwise it won't boot, unlike
> with UEFI where I need only have the AMD card passed through and the VM
> will boot.
>
> Is there any work-around for the above, or will I need to keep a spice
> video around as a secondary monitor, even though I don't use it?
>
> Thanks.
>
> Luke
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Wei,

I submitted a ticket here:
http://www.gigabyte.us/support-downloads/support-downloads.aspx

Thanks,
Nick

On Fri, Sep 23, 2016 at 12:06 AM, Brett Peckinpaugh 
wrote:

> Same here I have a ud5 that I would like a bios that does not need the ACS
> patch.
>
> On September 22, 2016 8:59:57 PM PDT, Wei Xu  wrote:
>
>>
>>
>> On 2016年09月23日 02:47, Nick Sarnie wrote:
>>
>>>  Hi again,
>>>
>>>  Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
>>>  new IOMMU groups (with ACS override patch kernel commandline removed for
>>>  this boot), as well as my lspci information are below. I see four
>>>  messages the following messages in dmesg now:
>>>
>>>  [0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>  [0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>  [0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>  [0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
>>>  enabled
>>>
>>>
>>>  IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
>>>
>>> lspci: http://pastebin.com/raw/1zAZuPBM
>>>
>>
>> That's cool, how did you report your issue to Gigabyte? I'd like to have
>> a try as well.
>>
>> Wei
>>
>>
>>>  Alex, please let me know if they missed anything else, so I can report
>>>  it to them.
>>>
>>>  Thanks,
>>>  Nick
>>>
>>>  On Sun, Sep 18, 2016 at 4:03 PM, Nick Sarnie >>  <mailto:commendsar...@gmail.com>> wrote:
>>>
>>>  Hi again,
>>>
>>>  Thanks a lot for investigating. I've reported the issue to the
>>>  manufacturer.
>>>
>>>
>>>  Thanks,
>>>  sarnex
>>>
>>>  On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson
>>>  mailto:alex.l.william...@gmail.com>>
>>>  wrote:
>>>
>>>  On Sat, Sep
>>> 17, 2016 at 12:29 PM, Nick Sarnie
>>>  mailto:commendsar...@gmail.com>> wrote:
>>>
>>>  Hi Alex,
>>>
>>>  The output is here: http://pastebin.com/raw/qjnpuaVr
>>>  <http://pastebin.com/raw/qjnpuaVr>
>>>
>>>
>>>  Ok, you need to go complain to your motherboard manufacturer,
>>>  they're the ones hiding the ACS capability.  PCIe capabilities
>>>  always start at 0x100, the dword there is:
>>>
>>>  100: 01 00 01 22 = 0x22010001
>>>
>>>  Breaking that down, the capability at 0x100 is ID 0x0001 (AER),
>>>  version 0x1, and the next capability is at 0x220.  So we do the
>>>  same there:
>>>
>>>  220: 19 00 01 00 = 0x00010019
>>>
>>>  Capability ID 0x0019 (Secondary PCIe), version 0x1, next
>>>
>>> capability 0x0, terminating the capability list.
>>>
>>>  Per Intel documentation for the chipset
>>>  
>>> (http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html
>>>  
>>> <http://www.intel.com/content/www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html>),
>>>  the ACS capability and control registers live at 0x144 and 0x148
>>>  respectively and we can see that you do have data here matching
>>>  the default value of the capability register:
>>>
>>>  140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
>>>
>>>  ie. default value of 0x144 is 0xf.  It appears that this BIOS
>>>  vendor didn't connect the capability into the chain or fill in
>>>  the
>>> capability header.  The registers to do this are RW/O, ie.
>>>  Read-Write-Once.  IOW, the registers can only be written once,
>>>  which is intended to be used by the BIOS.  The capability bits
>>>  themselves are RW/O, allowing vendors to expose different sets
>>>  of ACS capabilities.  Given that this vendor has not exposed the
>>>  capability, we have no basis to believe that the default value
>>>  of the register represents the real capabilities of the system
>>>  and therefore we cannot assume we're able to control ACS.  File
>>>  a bug with the 

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi again,

Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
new IOMMU groups (with ACS override patch kernel commandline removed for
this boot), as well as my lspci information are below. I see four messages
the following messages in dmesg now:

[0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
enabled
[0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
enabled
[0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
enabled
[0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
enabled


IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
lspci: http://pastebin.com/raw/1zAZuPBM

Alex, please let me know if they missed anything else, so I can report it
to them.

Thanks,
Nick

On Sun, Sep 18, 2016 at 4:03 PM, Nick Sarnie 
wrote:

> Hi again,
>
> Thanks a lot for investigating. I've reported the issue to the
> manufacturer.
>
>
> Thanks,
> sarnex
>
> On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson <
> alex.l.william...@gmail.com> wrote:
>
>> On Sat, Sep 17, 2016 at 12:29 PM, Nick Sarnie 
>> wrote:
>>
>>> Hi Alex,
>>>
>>> The output is here: http://pastebin.com/raw/qjnpuaVr
>>>
>>
>> Ok, you need to go complain to your motherboard manufacturer, they're the
>> ones hiding the ACS capability.  PCIe capabilities always start at 0x100,
>> the dword there is:
>>
>> 100: 01 00 01 22 = 0x22010001
>>
>> Breaking that down, the capability at 0x100 is ID 0x0001 (AER), version
>> 0x1, and the next capability is at 0x220.  So we do the same there:
>>
>> 220: 19 00 01 00 = 0x00010019
>>
>> Capability ID 0x0019 (Secondary PCIe), version 0x1, next capability 0x0,
>> terminating the capability list.
>>
>> Per Intel documentation for the chipset (http://www.intel.com/content/
>> www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html), the ACS
>> capability and control registers live at 0x144 and 0x148 respectively and
>> we can see that you do have data here matching the default value of the
>> capability register:
>>
>> 140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
>>
>> ie. default value of 0x144 is 0xf.  It appears that this BIOS vendor
>> didn't connect the capability into the chain or fill in the capability
>> header.  The registers to do this are RW/O, ie. Read-Write-Once.  IOW, the
>> registers can only be written once, which is intended to be used by the
>> BIOS.  The capability bits themselves are RW/O, allowing vendors to expose
>> different sets of ACS capabilities.  Given that this vendor has not exposed
>> the capability, we have no basis to believe that the default value of the
>> register represents the real capabilities of the system and therefore we
>> cannot assume we're able to control ACS.  File a bug with the vendor or
>> look for a BIOS update where they may have already fixed this.
>>
>>
>>> Also, is there any way we could move the USB controller into its own
>>> group, or remove the Ethernet and SATA controller into a seperate group?
>>> Ideally, I could pass the USB Controller in group 7 without the ACS patch.
>>>
>>
>> That's not how IOMMU groups works.  See  http://vfio.blogspot.com/2014
>> /08/iommu-groups-inside-and-out.html  We aren't creating these groups
>> arbitrarily, we base them on the information provided to use by the IOMMU
>> driver and PCI topology features, including ACS.  If we cannot determine
>> that there is isolation between components, we must assume that they are
>> not isolated.  Your choices are to run an unsupported (and unsupportable)
>> configuration using the ACS override patch, get your hardware vendor to fix
>> their platform, or upgrade to better hardware with better isolation
>> characteristics.
>>
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Lost link when pass through rtl8168 to guest

[Nick Sarnie]

Hi Wei,

My system is a desktop, so it must just be a general Gigabyte BIOS bug. I
submitted a help ticket about this issue and just gave a brief explanation
and then sent Alex's explanation. Hopefully it will be escalated correctly.

Thanks,
Nick

On Tue, Sep 20, 2016 at 1:08 PM, Wei Xu  wrote:

>
>
> On 2016年09月20日 22:20, Alex Williamson wrote:
>
>> On Tue, 20 Sep 2016 08:14:45 -0600
>> Alex Williamson  wrote:
>>
>> On Tue, 20 Sep 2016 21:56:33 +0800
>>> Wei Xu  wrote:
>>>
>>> On 2016年09月20日 09:59, Alex Williamson wrote:

> On Tue, 20 Sep 2016 09:28:57 +0800
> Wei Xu  wrote:
>
> Hi Guys,
>> I'm trying to pass through a rtl8168 nic to linux guest on my laptop
>> recently, the link is directly connected to my notebook with a cable.
>>
>> qemu: 2.7.0-rc4
>> host/guest kernel: 4.7.0-rc5
>> driver name: r8169
>>
>> After binding the driver to vfio-pci and launching the VM for a few
>> seconds, the connection led on the nic was turned off, and the guest
>> only see a link down nic with below messages.
>>
>> [6.173188] r8169 :00:04.0 ens4: rtl_phy_reset_cond == 1 (loop:
>> 100, delay: 1).
>> [6.177234] r8169 :00:04.0 ens4: link down
>> [6.177592] r8169 :00:04.0 ens4: link down
>> [6.177889] IPv6: ADDRCONF(NETDEV_UP): ens4: link is not ready
>>
>>
>> It's quite similar as below bug except it's for windows driver while
>> i'm testing linux.
>>
>> https://bugs.launchpad.net/qemu/+bug/1384892
>>
>>
>> More info:
>> My vm image is a pre-installed fedora 22 desktop, i also tried a fresh
>> fedora live iso, looks it can load the driver correctly.
>>
>> I tried to disable auto negotiation for the link but it didn't work
>> for me.
>>
>> I did the same test with my notebook with a Intel I218-LM ethernet
>> controller, it works pretty well every time.
>>
>> I googled around and looks it happened to bare metal too, so just
>> wonder
>> if this is a bug of network-manager, or is being caused by the nic
>> driver or an issue in qemu/kernel vfio, anybody can help?
>>
>
> Realtek nics don't work well with device assignment, they barely work
> well on bare metal.  Stick with the Intel nic or just use the rtl nic
> with virtio.  I've spent longer than I care to admit on the rtl quirks
> we have in QEMU and I expect they still only work with a few devices.
>

 OK, I'll ignore Realtek, so I added one Intel iwl6235 wireless nic to my
 laptop, the pci tree shows both the rtl and iwl are behind a separate
 pci bridge, after bind iwl to vfio-pci driver, i failed to pass through
 it again with error message from qemu.

 qemu-system-x86_64: -device vfio-pci,host=:02:00.0: vfio: error,
 group 5 is not viable, please ensure all devices within the iommu_group
 are bound to their vfio bus driver.
 qemu-system-x86_64: -device vfio-pci,host=:02:00.0: vfio: failed to
 get group 5
 qemu-system-x86_64: -device vfio-pci,host=:02:00.0: Device
 initialization failed

 Seems it's caused by the rtl nic is under the same iommu group with iwl
 as well, and when the kernel vfio driver checking the viablity, it'll
 make sure all the devices under the same group are viable, it works fine
 after i bound the rtl to vfio-pci too, i'm wonder if this a discipline?
 can't i just bind the iwl nic and pass through the the guest?

 pci tree:
 -[:00]-+-00.0 Intel Corporation Sky Lake Host Bridge/DRAM Registers
 +-1c.0-[01]00.0 Realtek Semiconductor Co., Ltd. RTL8111/8168/8411
 PCI Express Gigabit Ethernet Controller
 +-1c.7-[02]00.0 Intel Corporation Centrino Advanced-N 6235

>>>
>>> If your PCH root ports report an ACS capability then you can run kernel
>>> v4.7 kernel on the host to expose the isolation.  If the root ports
>>> (00:1c.*) do not expose an ACS capability, it's probably a BIOS bug
>>> similar to Nick's system in this thread
>>> https://www.redhat.com/archives/vfio-users/2016-September/msg00059.html
>>>
>>
>> And I see you're running a v4.7 kernel already (though I'm not sure why
>> you're running an rc release for kernel or QEMU since both of those
>> have been released).  So you need to check them with lspci to see if an
>> ACS capability is exposed on the root ports, check whether your root
>> ports are covered by the device IDs in this quirk:
>>
>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.
>> git/commit/?id=1bf2bf229b64540f91ac6fa3af37c81249037a0b
>>
>> If there's no ACS capability but the root ports fall within the quirk,
>> it's a BIOS bug on the system.  Sorry.
>>
>
> Unfortunately, the device id is within your list in the commit qurik
> but it failed still, my ACS dump of pci is as the same as Nick's, just
> wondering why the bios doesn't report it, looks it's a default optio

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi again,

Thanks a lot for investigating. I've reported the issue to the manufacturer.


Thanks,
sarnex

On Sat, Sep 17, 2016 at 5:35 PM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Sat, Sep 17, 2016 at 12:29 PM, Nick Sarnie 
> wrote:
>
>> Hi Alex,
>>
>> The output is here: http://pastebin.com/raw/qjnpuaVr
>>
>
> Ok, you need to go complain to your motherboard manufacturer, they're the
> ones hiding the ACS capability.  PCIe capabilities always start at 0x100,
> the dword there is:
>
> 100: 01 00 01 22 = 0x22010001
>
> Breaking that down, the capability at 0x100 is ID 0x0001 (AER), version
> 0x1, and the next capability is at 0x220.  So we do the same there:
>
> 220: 19 00 01 00 = 0x00010019
>
> Capability ID 0x0019 (Secondary PCIe), version 0x1, next capability 0x0,
> terminating the capability list.
>
> Per Intel documentation for the chipset (http://www.intel.com/content/
> www/us/en/chipsets/100-series-chipset-datasheet-vol-2.html), the ACS
> capability and control registers live at 0x144 and 0x148 respectively and
> we can see that you do have data here matching the default value of the
> capability register:
>
> 140: 00 00 00 00 0f 00 00 00 00 00 00 00 00 00 00 00
>
> ie. default value of 0x144 is 0xf.  It appears that this BIOS vendor
> didn't connect the capability into the chain or fill in the capability
> header.  The registers to do this are RW/O, ie. Read-Write-Once.  IOW, the
> registers can only be written once, which is intended to be used by the
> BIOS.  The capability bits themselves are RW/O, allowing vendors to expose
> different sets of ACS capabilities.  Given that this vendor has not exposed
> the capability, we have no basis to believe that the default value of the
> register represents the real capabilities of the system and therefore we
> cannot assume we're able to control ACS.  File a bug with the vendor or
> look for a BIOS update where they may have already fixed this.
>
>
>> Also, is there any way we could move the USB controller into its own
>> group, or remove the Ethernet and SATA controller into a seperate group?
>> Ideally, I could pass the USB Controller in group 7 without the ACS patch.
>>
>
> That's not how IOMMU groups works.  See  http://vfio.blogspot.com/
> 2014/08/iommu-groups-inside-and-out.html  We aren't creating these groups
> arbitrarily, we base them on the information provided to use by the IOMMU
> driver and PCI topology features, including ACS.  If we cannot determine
> that there is isolation between components, we must assume that they are
> not isolated.  Your choices are to run an unsupported (and unsupportable)
> configuration using the ACS override patch, get your hardware vendor to fix
> their platform, or upgrade to better hardware with better isolation
> characteristics.
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi Alex,

The output is here: http://pastebin.com/raw/qjnpuaVr
Also, is there any way we could move the USB controller into its own group,
or remove the Ethernet and SATA controller into a seperate group? Ideally,
I could pass the USB Controller in group 7 without the ACS patch.

Thanks,
sarnex

On Sat, Sep 17, 2016 at 2:28 PM, Nick Sarnie 
wrote:

> Hi Alex,
>
> The output is here: http://pastebin.com/raw/qjnpuaVr
> Also, is there any way we could move the USB controller into its own
> group, or remove the Ethernet and SATA controller into a seperate group?
> Ideally, I could pass the USB Controller in group 7 without the ACS patch.
>
> Thanks,
> sarnex
>
>
>
> On Sat, Sep 17, 2016 at 11:26 AM, Alex Williamson <
> alex.l.william...@gmail.com> wrote:
>
>> On Sat, Sep 17, 2016 at 9:12 AM, Alex Williamson <
>> alex.l.william...@gmail.com> wrote:
>>
>>> On Sat, Sep 17, 2016 at 9:00 AM, Nick Sarnie 
>>> wrote:
>>>
>>>> Hi Alex,
>>>>
>>>> I'm on 4.7.4 which includes this patch, and there are the IOMMU groups.
>>>> Is there some extra info I can provide?
>>>>
>>>
>>> Hmm, based on the info you sent me previously, your PCH root ports don't
>>> even attempt to include the broken ACS capability, therefore the quirk
>>> doesn't get enabled on your system.  Perhaps the Z170X is an especially
>>> broken version of Z170 :-\
>>>
>>
>> Could you pastebin a dump of PCI config space for the PCH root ports?
>>  ie. "sudo lspci -s 1c."
>>
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi Alex,

I'm on 4.7.4 which includes this patch, and there are the IOMMU groups. Is
there some extra info I can provide?

Thanks,
Sarnex

On Sat, Sep 17, 2016 at 4:40 AM, Philip Abernethy 
wrote:

> You definitely are. Running Skylake on Arch myself. Had the mainline
> package before, now using stock 4.7. I decided to buy a USB card to pass
> through.
>
> On Sat, 17 Sep 2016, 03:58 Brett Peckinpaugh,  wrote:
>
>> So I might be able to drop the ACS patch now that 4.7 is out on Arch.
>>
>> On September 16, 2016 5:39:43 PM PDT, Alex Williamson <
>> alex.l.william...@gmail.com> wrote:
>>>
>>>
>>> On Fri, Sep 16, 2016 at 6:30 PM, Nick Sarnie 
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I was wondering if we could split group 7 any more. CPU is the 6700k.
>>>> I'd like to be able to pass through the USB controller without the ACS
>>>> patch.
>>>>
>>>
>>> Run a newer kernel, Intel botched ACS in Skylake:
>>>
>>> http://git.kernel.org/cgit/linux/kernel/git/torvalds/
>>> linux.git/commit/drivers/pci/quirks.c?id=1bf2bf229b64540f91ac6fa3af37c8
>>> 1249037a0b
>>>
>>> PCH PCIe root ports have isolation starting in v4.7.
>>>
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>> ___
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Z170X IOMMU Groups

[Nick Sarnie]

Hi,

I was wondering if we could split group 7 any more. CPU is the 6700k. I'd
like to be able to pass through the USB controller without the ACS patch.

Thanks,
Sarnex

IOMMU group 0
00:00.0 Host bridge [0600]: Intel Corporation Skylake Host
Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU group 1
00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe
Controller (x16) [8086:1901] (rev 07)
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices,
Inc. [AMD/ATI] Ellesmere [Radeon RX 480] [1002:67df] (rev c7)
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI]
Device [1002:aaf0]
IOMMU group 2
00:02.0 Display controller [0380]: Intel Corporation HD Graphics
530 [8086:1912] (rev 06)
IOMMU group 3
00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H
USB 3.0 xHCI Controller [8086:a12f] (rev 31)
00:14.2 Signal processing controller [1180]: Intel Corporation
Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
IOMMU group 4
00:16.0 Communication controller [0780]: Intel Corporation Sunrise
Point-H CSME HECI #1 [8086:a13a] (rev 31)
IOMMU group 5
00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H
SATA controller [AHCI mode] [8086:a102] (rev 31)
IOMMU group 6
00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Root Port #17 [8086:a167] (rev f1)
IOMMU group 7
00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #1 [8086:a110] (rev f1)
00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #5 [8086:a114] (rev f1)
00:1c.5 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #6 [8086:a115] (rev f1)
03:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:00.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:01.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:02.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
04:04.0 PCI bridge [0604]: Intel Corporation DSL6540 Thunderbolt 3
Bridge [Alpine Ridge 4C 2015] [8086:1578]
07:00.0 USB controller [0c03]: Intel Corporation DSL6540 USB 3.1
Controller [Alpine Ridge] [8086:15b6]
09:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit
Network Connection [8086:1539] (rev 03)
0a:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062
Serial ATA Controller [1b21:0612] (rev 02)
IOMMU group 8
00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #9 [8086:a118] (rev f1)
00:1d.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI
Express Root Port #13 [8086:a11c] (rev f1)
0b:00.0 Network controller [0280]: Qualcomm Atheros AR93xx Wireless
Network Adapter [168c:0030] (rev 01)
IOMMU group 9
00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC
Controller [8086:a145] (rev 31)
00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H
PMC [8086:a121] (rev 31)
00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD
Audio [8086:a170] (rev 31)
00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus
[8086:a123] (rev 31)
IOMMU group 10
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet
Connection (2) I219-V [8086:15b8] (rev 31)
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Unbinding the RX 480 without blacklist

[Nick Sarnie]

Hi guys,

I recently purchased the RX 480, but I can't unbind it in the same way as I
could with radeon. If i boot with amdgpu not blacklisted nor the card bound
to pci-stub/vfio-pci automatically, and I try to unbind, I get a hard
kernel lockup. With radeon, I could just switch the default  BusID in
radeon.conf, stop X, unbind my original GPU and bind it to vfio-pci, and
restart X and it would work. With this card, I have to completely blacklist
amdgpu, restart the computer, and then bind it to vfio-pci.

Let me know if anyone has any ideas or needs more information.

Thanks,
sarnex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] gpu passthrough problems after upgrade qemu 2.5 -> 2.6

[Nick Sarnie]

Hi Jiri,

I just wanted to add that a similar issue is effecting me. With qemu 2.6, I
get a black screen after the seabios screen. Downgrading to 2.5.1 fixes it
for me. I tried 2.6 with the pc-i440fx-2.5 machine type, but that didn't
work either.

If anyone has any ideas, let us know.

Thanks

On Fri, Jun 3, 2016 at 3:26 AM, Jiri 'Ghormoon' Novak 
wrote:

> Hello,
>
> yesterday I've upgraded to qemu 2.6 (I've just rebuilt debian sid
> package for jessie, previously I did the same with 2.5 and just added
> input-linux patches).
>
> I'm using vga passthrough on HP Elitebook 8560p laptop with Radeon HD
> 6470M, passing it in with x-vga=on (no ovmf, it doesn't have ovmf firmware)
>
> After the upgrade, when booting linux, it shows bios screen, then
> alternates black and garbage screen for few minutes, then reboots (?),
> bios screen again, black screen, linux (only after driver kicks in, no
> grub/boot screen, previously it worked). Rebooting from guest makes it
> go through this all again including the few minutes of garbage sreens.
> There's nothing in guest logs for that time, likely it doesn't even pass
> grub.
>
> For windows it actually boots on first attempt, but also no output until
> driver kicks in (previosly it shown boot screen or worked without ati
> drivers installed)
>
> in the dmesg, there's quite a lot of rdmsr errors (on 2.5 there few a
> few too, but they appeared once, not in multiple batches like now).
>
> I assume some graphic modes are now handled differently. Is it only a
> change in defaults that I can override or is it a bug?
>
> dmesg:
> [  136.888758] e1000e :00:19.0: irq 46 for MSI/MSI-X
> [  136.992866] e1000e :00:19.0: irq 46 for MSI/MSI-X
> [  136.993181] IPv6: ADDRCONF(NETDEV_UP): eth0: link is not ready
> [  140.329803] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow
> Control: None
> [  140.329858] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> [  148.661906] VFIO - User Level meta-driver version: 0.3
> [  148.667113] ehci-pci :00:1a.0: remove, state 1
> [  148.667123] usb usb3: USB disconnect, device number 1
> [  148.667126] usb 3-1: USB disconnect, device number 2
> [  148.667128] usb 3-1.1: USB disconnect, device number 3
> [  148.667327] usb 3-1.2: USB disconnect, device number 4
> [  148.667931] cdc_ncm 3-1.2:1.6 usb0: unregister 'cdc_ncm'
> usb-:00:1a.0-1.2, CDC NCM
> [  148.701340] ehci-pci :00:1a.0: USB bus 3 deregistered
> [  148.876484] ehci-pci :00:1d.0: remove, state 1
> [  148.876505] usb usb4: USB disconnect, device number 1
> [  148.876512] usb 4-1: USB disconnect, device number 2
> [  148.876517] usb 4-1.1: USB disconnect, device number 3
> [  148.986636] usb 4-1.4: USB disconnect, device number 4
> [  149.045785] usb 4-1.6: USB disconnect, device number 5
> [  149.055202] ehci-pci :00:1d.0: USB bus 4 deregistered
> [  149.099334] xhci_hcd :05:00.0: remove, state 4
> [  149.099348] usb usb2: USB disconnect, device number 1
> [  149.116629] xhci_hcd :05:00.0: USB bus 2 deregistered
> [  149.116699] xhci_hcd :05:00.0: remove, state 4
> [  149.116710] usb usb1: USB disconnect, device number 1
> [  149.118085] xhci_hcd :05:00.0: USB bus 1 deregistered
> [  149.408028] IPv6: ADDRCONF(NETDEV_CHANGE): br0vmtap10: link becomes
> ready
> [  149.408081] br0: port 1(br0vmtap10) entered forwarding state
> [  149.408092] br0: port 1(br0vmtap10) entered forwarding state
> [  149.408305] IPv6: ADDRCONF(NETDEV_CHANGE): br1vmtap10: link becomes
> ready
> [  149.408333] br1: port 1(br1vmtap10) entered forwarding state
> [  149.408343] br1: port 1(br1vmtap10) entered forwarding state
> [  151.128482] vfio_cap_init: :00:1a.0 hiding cap 0xa
> [  151.232527] vfio_ecap_init: :00:1b.0 hiding ecap 0x5@0x130
> [  151.340394] vfio_cap_init: :00:1d.0 hiding cap 0xa
> [  156.620596] dmar: DRHD: handling fault status reg 3
> [  156.620662] dmar: DMAR:[DMA Read] Request device [00:1d.0] fault addr
> ef000
> DMAR:[fault reason 06] PTE Read access is not set
> [  157.585018] kvm: zapping shadow pages for mmio generation wraparound
> [  164.431957] br0: port 1(br0vmtap10) entered forwarding state
> [  164.431966] br1: port 1(br1vmtap10) entered forwarding state
> [  243.766285] kvm [4369]: vcpu0 unhandled rdmsr: 0x606
> [  243.766358] kvm [4369]: vcpu0 unhandled rdmsr: 0x34
> [  244.095770] vfio-pci :05:00.0: irq 47 for MSI/MSI-X
> [  244.149600] vfio-pci :05:00.0: irq 47 for MSI/MSI-X
> [  244.149606] vfio-pci :05:00.0: irq 48 for MSI/MSI-X
> [  244.149828] vfio-pci :05:00.0: irq 47 for MSI/MSI-X
> [  244.149832] vfio-pci :05:00.0: irq 48 for MSI/MSI-X
> [  244.149835] vfio-pci :05:00.0: irq 49 for MSI/MSI-X
> [  245.276448] kvm [4369]: vcpu0 unhandled rdmsr: 0x611
> [  245.276518] kvm [4369]: vcpu0 unhandled rdmsr: 0x639
> [  245.276582] kvm [4369]: vcpu0 unhandled rdmsr: 0x641
> [  245.276644] kvm [4369]: vcpu0 unhandled rdmsr: 0x619
> [  245.438562] vfio-pci :00:1b.0: irq 50 for MSI/MSI

Re: [vfio-users] Non-root Passthrough Audio with libvirt

[Nick Sarnie]

I managed to solve this. You run pulse as the user, with anonymous auth and
a unix socket at /tmp/pulse. Then, set PULSE_SERVER=unix:/tmp/pulse in the
envvars for the domain xml. Works from there.

On Sun, Apr 17, 2016 at 11:45 PM, Nick Sarnie 
wrote:

> Hi guys,
>
> I'm using libvirt and virt-manager for my GPU passthrough setup, with
> no-root. For the life of me, I can't figure out how to get pulseaudio
> working. Even though it is running as user, libvirt is trying to look in
> /root for pulse config files. The VM shows I have an audio device
> connected, but there is no sound passes through to me on the host. If I use
> a script with no libvirt, it works fine. If I use libvirt with no gpu
> passthrough, it works fine. I've pasted the log below. Please let me know
> if you have any ideas.
>
> Thanks,
> sarnex
>
>
> Home directory not accessible: Permission denied
> W: [pulseaudio] core-util.c: Failed to open configuration file
> '/root/.config/pulse//daemon.conf': Permission denied
> W: [pulseaudio] daemon-conf.c: Failed to open configuration file:
> Permission denied
> pulseaudio: pa_context_connect() failed
> pulseaudio: Reason: Connection refused
> pulseaudio: Failed to initialize PA contextaudio: Could not init `pa'
> audio driver
> Home directory not accessible: Permission denied
> W: [pulseaudio] core-util.c: Failed to open configuration file
> '/root/.config/pulse//daemon.conf': Permission denied
> W: [pulseaudio] daemon-conf.c: Failed to open configuration file:
> Permission denied
> audio: Failed to create voice `ac97.pi'
> Home directory not accessible: Permission denied
> W: [pulseaudio] core-util.c: Failed to open configuration file
> '/root/.config/pulse//daemon.conf': Permission denied
> W: [pulseaudio] daemon-conf.c: Failed to open configuration file:
> Permission denied
> ALSA lib
> /var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
> unable to open slave
> sdl: SDL_OpenAudio failed
> sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
> ALSA lib
> /var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
> unable to open slave
> sdl: SDL_OpenAudio failed
> sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
> audio: Failed to create voice `ac97.po'
> audio: Failed to create voice `ac97.mc'
> audio: Failed to create voice `ac97.pi'
> ALSA lib
> /var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
> unable to open slave
> sdl: SDL_OpenAudio failed
> sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
> ALSA lib
> /var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
> unable to open slave
> sdl: SDL_OpenAudio failed
> sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
> audio: Failed to create voice `ac97.po'
> audio: Failed to create voice `ac97.mc'
> 2016-04-18T02:27:04.033280Z qemu-system-x86_64: terminating on signal 15
> from pid 5395
> 2016-04-18 02:27:05.634+: shutting down
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Non-root Passthrough Audio with libvirt

[Nick Sarnie]

Hi guys,

I'm using libvirt and virt-manager for my GPU passthrough setup, with
no-root. For the life of me, I can't figure out how to get pulseaudio
working. Even though it is running as user, libvirt is trying to look in
/root for pulse config files. The VM shows I have an audio device
connected, but there is no sound passes through to me on the host. If I use
a script with no libvirt, it works fine. If I use libvirt with no gpu
passthrough, it works fine. I've pasted the log below. Please let me know
if you have any ideas.

Thanks,
sarnex


Home directory not accessible: Permission denied
W: [pulseaudio] core-util.c: Failed to open configuration file
'/root/.config/pulse//daemon.conf': Permission denied
W: [pulseaudio] daemon-conf.c: Failed to open configuration file:
Permission denied
pulseaudio: pa_context_connect() failed
pulseaudio: Reason: Connection refused
pulseaudio: Failed to initialize PA contextaudio: Could not init `pa' audio
driver
Home directory not accessible: Permission denied
W: [pulseaudio] core-util.c: Failed to open configuration file
'/root/.config/pulse//daemon.conf': Permission denied
W: [pulseaudio] daemon-conf.c: Failed to open configuration file:
Permission denied
audio: Failed to create voice `ac97.pi'
Home directory not accessible: Permission denied
W: [pulseaudio] core-util.c: Failed to open configuration file
'/root/.config/pulse//daemon.conf': Permission denied
W: [pulseaudio] daemon-conf.c: Failed to open configuration file:
Permission denied
ALSA lib
/var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
unable to open slave
sdl: SDL_OpenAudio failed
sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
ALSA lib
/var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
unable to open slave
sdl: SDL_OpenAudio failed
sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
audio: Failed to create voice `ac97.po'
audio: Failed to create voice `ac97.mc'
audio: Failed to create voice `ac97.pi'
ALSA lib
/var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
unable to open slave
sdl: SDL_OpenAudio failed
sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
ALSA lib
/var/tmp/portage/media-libs/alsa-lib-1.1.1/work/alsa-lib-1.1.1/src/pcm/pcm_dmix.c:1029:(snd_pcm_dmix_open)
unable to open slave
sdl: SDL_OpenAudio failed
sdl: Reason: ALSA: Couldn't open audio device: Device or resource busy
audio: Failed to create voice `ac97.po'
audio: Failed to create voice `ac97.mc'
2016-04-18T02:27:04.033280Z qemu-system-x86_64: terminating on signal 15
from pid 5395
2016-04-18 02:27:05.634+: shutting down
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Disk Performance

[Nick Sarnie]

Hi again,

Yep, I just copied and pasted it. Thanks for updating my script, I've added
the changes.

Thanks again,
sarnex

On Sun, Apr 3, 2016 at 5:33 AM, Bob Dawes  wrote:

> My apologies as you appear to have inherited all my unnecessary
> configuration options on the block device. As I'm messing around with qemu
> and I don't really understand that driver yet I tend to explicitly set
> every option so next time I use the VM with a new testing version of qemu
> my VM doesn't do unpredictable things because some feature I'm not messing
> with got switched on or off.
>
> You may prefer to always have the author's best settings which in the
> general case is usually advisable and is achieved by leaving parameters
> implicit. For your config this would be:
>
> -drive
> if=none,format=raw,cache=none,cache.direct=on,file=/media/500GB/win10.img,aio=native,id=ssd2
> \
> -object iothread,id=iothread2 \
> -device virtio-blk-pci,drive=ssd2,iothread=iothread2
>
> Naturally, you will find most authors tend to be cautious about switching
> on experimental or risky features, so to test those you usually have to
> explicitly set parameters. You can type 'info qtree' in the qemu console
> and check the configs parameters if you are interested in the fine details,
> but I wouldn't explicitly set the parameters unless you know you need
> something.
>
>
> On 03/04/16 03:26, Nick Sarnie wrote:
>
> Hi,
>
> Thanks for your detailed post. I believe I have already changed my script
> to make sense, thanks to the previous reply. I've attached it again below.
> For networking, I am using a wireless card where I cannot create a bridge,
> and my IP changes very often. Is there a simple way to manually create a
> tap with this? I wasn't able to get either of the 2 most popular tutorials
> working.
>
> Thanks,
> sarnex
>
> #!/bin/sh
> export QEMU_AUDIO_DRV=pa
> qemu-system-x86_64 -enable-kvm \
> -m 5120 \
> -cpu host \
> -smp 6,sockets=1,cores=6,threads=1 \
> -device vfio-pci,host=01:00.0,x-vga=on,multifunction=on \
> -device vfio-pci,host=01:00.1 \
> -vga none \
> -device vfio-pci,host=00:12.0 \
> -device vfio-pci,host=00:12.2 \
> -device vfio-pci,host=00:16.0 \
> -device vfio-pci,host=00:16.2 \
> -soundhw ac97 \
> -rtc base=localtime \
> -netdev user,id=net0 \
> -device virtio-net-pci,netdev=net0 \
> -drive
> if=none,format=raw,cache=none,cache.direct=on,file=/media/500GB/win10.img,aio=native,id=ssd2,discard=off,detect-zeroes=off
> \
> -object iothread,id=iothread2 \
> -device
> virtio-blk-pci,drive=ssd2,request-merging=on,iothread=iothread2,modern-pio-notify=on,config-wce=off
>
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Disk Performance

[Nick Sarnie]

Hi,

Thanks for your detailed post. I believe I have already changed my script
to make sense, thanks to the previous reply. I've attached it again below.
For networking, I am using a wireless card where I cannot create a bridge,
and my IP changes very often. Is there a simple way to manually create a
tap with this? I wasn't able to get either of the 2 most popular tutorials
working.

Thanks,
sarnex

#!/bin/sh
export QEMU_AUDIO_DRV=pa
qemu-system-x86_64 -enable-kvm \
-m 5120 \
-cpu host \
-smp 6,sockets=1,cores=6,threads=1 \
-device vfio-pci,host=01:00.0,x-vga=on,multifunction=on \
-device vfio-pci,host=01:00.1 \
-vga none \
-device vfio-pci,host=00:12.0 \
-device vfio-pci,host=00:12.2 \
-device vfio-pci,host=00:16.0 \
-device vfio-pci,host=00:16.2 \
-soundhw ac97 \
-rtc base=localtime \
-netdev user,id=net0 \
-device virtio-net-pci,netdev=net0 \
-drive
if=none,format=raw,cache=none,cache.direct=on,file=/media/500GB/win10.img,aio=native,id=ssd2,discard=off,detect-zeroes=off
\
-object iothread,id=iothread2 \
-device
virtio-blk-pci,drive=ssd2,request-merging=on,iothread=iothread2,modern-pio-notify=on,config-wce=off



On Sat, Apr 2, 2016 at 9:56 PM, Zir Blazer  wrote:

> I spend a lot of time experimenting what works and what doesn't, through I
> never benchmarked anything to see performance differences.
> The main issue is that QEMU has a load of old syntaxis and parameters
> mixed with new ones that you must use to get the latest features, and
> documentation is extremely poor, so as an end user, I have absolutely no
> way to know what extra parameters I can tweak beyond the common sense ones.
> It takes A LOT of googling to understand how things works, and even then,
> it just to get them working, not even performance optimization.
>
> The first thing you need to understand, is that just because QEMU seems to
> work and it starts your VM, does NOT MEANS that you get the things
> connected as you intend. For example, on your initial script, you are
> creating a VirtIO SCSI Controller, but you are NOT creating a scsi-hd to
> attach to it. Eventually, you will figure out that if you want to
> experiment, you need to use QEMU Monitor. Use -monitor stdio to have it at
> the Terminal you launch QEMU from. You can use info qtree to see a heavy
> Wall of Text that you will eventually figure out how to understand that
> tells you how things are connected, or basically, at what Bus or Controller
> each Device is attached.
>
>
> At the moment, the current Storage Controllers contenders are these:
>
> The PIIX3 IDE Controller provided by the i440FX platform, -device
> piix3-ide. Requires ide-hd/ide-cd
> The ICH9 SATA Controller provided by the Q35 platform, which always runs
> in AHCI Mode (No IDE emulation) and supports NCQ, -device ahci-ich9.
> Requires ide-hd/ide-cd
> The VirtIO Block Device, which is paravirtualized and the mainstream
> performance choice. -device virtio-blk-pci
> The VirtIO SCSI Controller, which is also paravirtualized. Performance was
> nearly that of the VirtIO Block Device, but you could attach several SCSI
> HDs to a single controller, instead of needing one controller per drive.
> -device virtio-scsi-pci. Requires scsi-hd/scsi-cd
>
> The Q35 ICH9 SATA Controller always runs in AHCI Mode, is not like the one
> you usually find in any Motherboard that you can set if you want IDE
> emulation. Basically, if you want to use Windows XP with it, you need to
> either slipstream the ICH9 AHCI Drivers with nLite, or pass to the VM a
> Floppy image with them to do the F6 installation procedure.
> The VirtIO Devices always need Drivers for Windows, easier way is to use
> the Fedora VirtIO Drivers ISO during Windows installation. VirtIO Block
> Device has Drivers for WXP, but VirtIO SCSI Controller does not, only
> Vista+.
>
> Specifying Drives with the latest QEMU syntax is a two or three step
> procedure, and must be made in the following order on the command line:
>
> 1) You need to specify the host-side place that will represent the
> contents of the virtual Hard Disk. You do that with -drive.
>
> -drive if=none,id=drive0,format=raw,file=/dev/vg0/w10x64
>
> file= could be an entire Hard Disk (/dev/sdb), a physical partition
> (/dev/sda3), a LVM volume (/dev/vg0/lv0), a file (filebackedvm.img), etc.
> It can also be empty, which you can use if you want to create a CD-ROM
> drive that has no ISO inserted (Since ide-cd and scsi-cd requires a valid
> drive id parameter, but -drive doesn't requires a file=).
>
> 2) You need to create a Controller that will provide the Bus for the HDs
> that will contain those host-side routes to attach to:
>
> -device virtio-scsi-pci,id=scsi0
>
> On i440FX and Q35 you already have an IDE Bus thanks to the PIIX3 and
> ICH9. But we want to use the latest and greatest, so here you have the
> VirtIO SCSI Controller.
>
> 3) You need to create a Device that represents the virtual IDE or SCSI
> HD/CD with the host route from -drive, and attach it to the correct
> controller (ID

Re: [vfio-users] Disk Performance

[Nick Sarnie]

Hi again,

Fluxion: Thanks, I've added that. Unfortunately, I'm stuck using net user
because I'm only on wireless so I can't bridge easily, and I haven't been
able to get any of the wireless bridging taps working. Also, I'm at
university so my IP changes a lot, which seems like I'd have to reconfigure
every boot.

Bob: Thanks! I've swapped in your disk configure. Let's hope it's faster.

Let me know if you have any ideas.

Thanks,
Sarnex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Disk Performance

[Nick Sarnie]

Hi guys,

It seems the biggest limiting factor of my GPU Passthrough setup is the
disk. Does anyone have any tips to optimize it? Also, which is faster,
having the disk with if=virtio, or having the disk with if=none and a
virtio SCSI controller. The raw image is 250GB on a 500GB ext4 hard drive,
which is encrypted with dm-crypt. Below is my current script.

Thanks,
Sarnex


#!/bin/sh
export QEMU_AUDIO_DRV=pa
qemu-system-x86_64 -enable-kvm \
-m 5120 \
-cpu host \
-smp 8,sockets=1,cores=8,threads=1 \
-device vfio-pci,host=01:00.0,x-vga=on,multifunction=on \
-device vfio-pci,host=01:00.1 \
-vga none \
-drive file=/media/500GB/win10.img,id=disk,if=virtio,cache=none,format=raw \
-device vfio-pci,host=00:12.0 \
-device vfio-pci,host=00:12.2 \
-device vfio-pci,host=00:16.0 \
-device vfio-pci,host=00:16.2 \
-soundhw ac97 \
-rtc base=localtime \
-netdev user,id=net0 \
-device virtio-net-pci,netdev=net0 \
-device virtio-scsi-pci,id=scsi
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Win10 guest stops responding. How to debug?

[Nick Sarnie]

Could it also be the host running out of RAM and thrashing? I've had that
issue produce these results.

On Fri, Feb 26, 2016 at 2:32 PM,  wrote:

> Hello,
>
> Here is my setup:
> CPU: i5-4460
> GPU: R9 290
> MB: Asrock H97M-ITX
> Host OS: Archlinux running linux-ck 4.3.6 (but same problem with linux 4.4)
> Guest OS: Windows10 Education
> Mouse and keyboard are shared with synergy.
> The script I use to launch the VM is attached.
>
> I can boot the VM fine most of the time. However, after a while, the VM
> stops responding. By this, I mean I can still move the mouse on the screen
> (so network and graphics seem to be working, but the programs inside the VM
> stop working and nothing happens when I type or click. My guess is either
> the disk (virtio scsi) or the cpu (-cpu
> host,hv_relaxed,hv_spinlocks=0x1fff,hv_vapic,hv_time)
> How can I troubleshoot this?
> Which steps could I try to solve the problem?
> Writing this, I realise I haven't setup hugepages, do hugepages have a
> chance of solving this?
>
> Cheers,
>
> Guillaume
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] when is ROM extraction needed and how to do it correctly?

[Nick Sarnie]

I believe it's only needed to check if your video card bios supports UEFI
with rom-parser. You can optionally pass it to qemu, but I haven't needed
to.

On Fri, Feb 26, 2016 at 3:54 AM, Daniel Pocock  wrote:

>
> In which cases are these ROMs actually needed? For example, is the ROM
> needed if the passthrough video device is secondary display? Is the ROM
> needed if using UEFI / OVMF instead of SeaBIOS?
>
> I've extracted ROMs from each GPU using the method:
>
>   find /sys/devices -name rom
>
> to discover the devices with a ROM.  For the device I want to use, I've
> used something like the following to extract the ROM (based on the paths
> found by the find command):
>
>
> ROM=/sys/devices/pci\:40/\:40\:03.0/\:42\:00.0/rom
>
> echo 1 | tee "${ROM}"
> cat "${ROM}" > my-vga-device.rom
>
> I've also read that some devices modify a copy of their ROM data during
> boot.  Does the method described above access the original ROM content
> or the modified content?  If it is accessing modified content, is there
> a way to get the original ROM content, either generically or in a method
> specific to each vendor?
>
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Not binding GPU to vfio-pci

[Nick Sarnie]

Awesome, thanks for the link.

On Sun, Feb 21, 2016 at 1:09 PM,  wrote:

> For your quick  question is there answer. I ask litle bit same in this
> thread.
> https://www.redhat.com/archives/vfio-users/2016-February/msg00033.html
>
>
> 2016-02-21 17:45 GMT+01:00 Nick Sarnie :
>
>> Hi Alex,
>>
>> Yeah it seems you're right, the qemu command has "-device
>> vfio-pci,host=01:00.0,id=hostdev2,bus=pci.0,addr=0x2", so it seems it is
>> using vfio-pci.Thanks for the information.
>>
>> One more quick question: I'm using SeaBIOS for my VM. My GPU doesn't
>> support UEFI Secure/Fast booting. Can I use a GPU like this with OVMF and
>> more importantly, is there any performance difference using OVMF over
>> SeaBios? Thanks alot,
>> sarnex
>>
>> On Sun, Feb 21, 2016 at 11:36 AM, Alex Williamson <
>> alex.william...@redhat.com> wrote:
>>
>>> On Sun, Feb 21, 2016 at 9:06 AM, Nick Sarnie 
>>> wrote:
>>>
>>>> Sorry, I'm not sure if I'm using legacy device assignment. My script is
>>>> stop xdm, echo ":01:00.0" > /sys/bus/pci/drivers/radeon/unbind echo
>>>> ":01:00.1" > /sys/bus/pci/drivers/radeon/unbind and then start xdm. I
>>>> use virt-manager with libvirt, and I added the gpu to the VM with PCI Host
>>>> Device under Add Hardware. Is this what you're talking about?
>>>>
>>>
>>> Somehow I doubt that both function 0 and 1 are bound to the radeon
>>> driver, but if you're not crafting your own QEMU command line or specifying
>>> a driver for the hostdev entries in your XML, then you're probably using
>>> vfio.  You can verify by looking at the libvirt log for the VM
>>> (/var/log/libvirt/qemu/$domain.log) for either vfio-pci or pci-assign
>>> devices.
>>>
>>
>>
>> ___
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Not binding GPU to vfio-pci

[Nick Sarnie]

Hi Alex,

Yeah it seems you're right, the qemu command has "-device
vfio-pci,host=01:00.0,id=hostdev2,bus=pci.0,addr=0x2", so it seems it is
using vfio-pci.Thanks for the information.

One more quick question: I'm using SeaBIOS for my VM. My GPU doesn't
support UEFI Secure/Fast booting. Can I use a GPU like this with OVMF and
more importantly, is there any performance difference using OVMF over
SeaBios? Thanks alot,
sarnex

On Sun, Feb 21, 2016 at 11:36 AM, Alex Williamson <
alex.william...@redhat.com> wrote:

> On Sun, Feb 21, 2016 at 9:06 AM, Nick Sarnie 
> wrote:
>
>> Sorry, I'm not sure if I'm using legacy device assignment. My script is
>> stop xdm, echo ":01:00.0" > /sys/bus/pci/drivers/radeon/unbind echo
>> ":01:00.1" > /sys/bus/pci/drivers/radeon/unbind and then start xdm. I
>> use virt-manager with libvirt, and I added the gpu to the VM with PCI Host
>> Device under Add Hardware. Is this what you're talking about?
>>
>
> Somehow I doubt that both function 0 and 1 are bound to the radeon driver,
> but if you're not crafting your own QEMU command line or specifying a
> driver for the hostdev entries in your XML, then you're probably using
> vfio.  You can verify by looking at the libvirt log for the VM
> (/var/log/libvirt/qemu/$domain.log) for either vfio-pci or pci-assign
> devices.
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Not binding GPU to vfio-pci

[Nick Sarnie]

Sorry, I'm not sure if I'm using legacy device assignment. My script is
stop xdm, echo ":01:00.0" > /sys/bus/pci/drivers/radeon/unbind echo
":01:00.1" > /sys/bus/pci/drivers/radeon/unbind and then start xdm. I
use virt-manager with libvirt, and I added the gpu to the VM with PCI Host
Device under Add Hardware. Is this what you're talking about?


Thanks,
sarnex

On Sat, Feb 20, 2016 at 7:24 PM, Alex Williamson  wrote:

> On Sat, 20 Feb 2016 18:47:29 -0500
> Nick Sarnie  wrote:
>
> > Hi guys,
> > When i want to do gpu-passthrough, i just unbind my GPU from radeon, and
> > then pass it to qemu. I don't bind it to vfio-pci or any driver and it
> > works fine. Is there any downside or performance loss to doing it this
> way?
>
> Does this mean you're using the pci-assign driver with qemu?  If so, be
> advised that legacy KVM device assignment is deprecated upstream, so it
> will likely go away at some point and it mostly unsupported already.
> Generally it's advised to avoid host drivers because they don't like
> releasing devices and things like Xorg get unhappy as well.  If you
> have a usage model that allows you to release the device, go for it,
> but you probably want to migrate away from legacy KVM device
> assignment.
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Not binding GPU to vfio-pci

[Nick Sarnie]

Hi guys,
When i want to do gpu-passthrough, i just unbind my GPU from radeon, and
then pass it to qemu. I don't bind it to vfio-pci or any driver and it
works fine. Is there any downside or performance loss to doing it this way?

Thanks,
sarnex
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users