Re: [vfio-users] Stability Testing/Benchmarking software for Windows VMs

[Rich Mingin (vfio-users)]

It's been a long time since I did it, but I remember 3Dmark (commercial and
free versions) having a looping mode, and I know Furmark was a real GPU
killer last time I looked. If you have an eVGA graphics card, their
Precision software includes a multiple-test-type version of Furmark built
in.

On Mon, Jul 11, 2016 at 9:19 PM, Brian Yglesias <
br...@atlanticdigitalsolutions.com> wrote:

> I'm not much of a gamer, so I'm looking for software to use for
> troubleshooting stability issues.
>
> E.g. the overclockers all use prime95, or at least they used to.  Is there
> some analog de facto standard for GPU testing?
>
> The software I've found so far doesn't have a continuous testing mode.
> Anything with diagnostic info would be a plus.
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Boot using second GPU?

[Rich Mingin (vfio-users)]

I'm using an Asus Z97 board and a Gigabyte Z97 board currently, and both
allow for switching the boot GPU between PCIE1, PCIE[2-5] or the iGPU. All
slots are one ACS group, though.

On Sun, Aug 7, 2016 at 10:14 AM, Rokas Kupstys  wrote:

> Thanks for reply. Since i am making a new build i am looking for proper
> motherboard. One i sided with is from asus, but it seems asus
> motherboards do not support switching primary GPU. I might go with
> gigabyte if there is no way to solve this. I am not sure yet though.
> Switching primary gpu in bios is ultimate solution. Directions i
> provided in earlier mail are workaround for people who do not have such
> capability. It is a tricky choice.. Go with beloved asus and suffer
> minor inconvenience or go with gigabyte.. Did you have any problems with
> gigabyte motherboard(s)?
>
>
> Rokas Kupstys
>
> On 2016.08.05 14:44, Hristo Iliev wrote:
> > Am 05.08.2016 10:22, schrieb Rokas Kupstys:
> >
> >> Okay this is unexpected luck. After more tinkering i got it to work!
> >> Here is my setup:
> >>
> >> * AMD FX-8350 CPU + Sabertooth 990FX R2 motherboard
> >> * :01:00.0 - gpu in first slot
> >> * :06:00.0 - gpu in third slot
> >> * UEFI on host and guest.
> >> * Archlinux
> >>
> >> In order to make host use non-boot GPU:
> >>
> >> 1. Add Kernel boot parameter "video=efifb:off". This makes kernel not
> >> use first gpu and boot messages appear on second gpu.
> >>
> >> 2. Bind first gpu (:01:00.0) to vfio-pci driver. I did this by
> >> adding line
> >>
> >>> options vfio-pci ids=1002:677B,1002:AA98
> >> to /etc/modprobe.d/kvm.conf. They are obtained from "lspci -n" which
> >> in my case show:
> >>
> >>> 01:00.0 0300: 1002:677B
> >>> 01:00.1 0403: 1002:AA98
> >> 3. Configure xorg to use second gpu (:06:00.0). I added file
> >> /etc/X11/xorg.conf.d/secondary-gpu.conf with contents:
> >>
> >>> Section "Device"
> >>> Identifier "Device0"
> >>> Driver "radeon"
> >>> VendorName "AMD Corporation"
> >>> BoardName  "AMD Secondary"
> >>> BusID  "PCI:6:0:0"
> >>> EndSection
> >> And thats it! Now when machine boots it shows POST messages and
> >> bootloader on first gpu, but as soon as boot option is selected
> >> display goes blank and kernel boot messages show on second gpu. After
> >> boot you can assign first gpu to VM as usual and it works.
> >> HELP REQUEST: could someone with intel hardware (ideally x99 chipset)
> >> test this method? I am planning a build and if this works i could
> >> settle with 28 lane cpu and save couple hundred dollars. Intel's 40
> >> lane cpus are way overpriced.. And with 28 lane cpus only first slot
> >> can run at x16 speed while other slots downgrade to x8 or less.
> >> Anyhow i would love to hear if this works on intel hardware.
> >>
> >
> > Hi,
> >
> > I have a Gigabyte GA-X99-UD4 motherboard and i7-5820K. There are two GPUs
> > in it - a GTX 970 for pass-through on 03:00.0 and a GT 730 as primary GPU
> > on 06:00.0. The PCIE slot of the GT is selected as primary video output
> > in the UEFI settings. All text and graphics output goes to it - the
> > output
> > of the GTX card remains off the entire time until the VM is booted. The X
> > server does see both cards but since the nvidia module is prevented from
> > binding to the GTX, X cannot use it and starts on the GT. No fiddling
> > with
> > the console driver parameters necessary.
> >
> > Distribution:
> >Arch Linux, 4.6.4-1-ARCH
> >
> > Kernel parameters:
> >... pci-stub.ids=10de:13c2,10de:0fbb,8086:8d20 nvidia-drm.modeset=1
> > ...
> >
> > /etc/modprobe.d/vfio.conf:
> >options vfio-pci ids=10de:13c2,10de:0fbb,8086:8d20
> >
> > /etc/mkinitcpio.conf:
> >...
> >MODULES="vfio vfio_iommu_type1 vfio_pci vfio_virqfd vfat aes_x86_64
> > crc32c_intel nvidia nvidia_modeset nvidia_uvm nvidia_drm"
> >...
> >
> > /etc/X11/xorg.conf.d/20-nvidia.conf:
> >Section "Device"
> > Identifier"Device0"
> > Driver"nvidia"
> > VendorName"NVIDIA Corporation"
> > Option "ConnectToAcpid"   "0"
> >EndSection
> >
> > The only problem with my setup is that the GTX is in PCIE_2, which works
> > as x8 with i7-5820K installed. I cannot fit the card in PCIE_1 because of
> > the oversized CPU cooler. This doesn't actually bug me at all as multiple
> > tests (for example, [1]) have shown negligible difference in gaming FPS
> > between PCI-e 3.0 x8 and x16. The GT card is in PCIE_4.
> >
> > Kind regards,
> > Hristo
> >
> > [1]
> > http://www.gamersnexus.net/guides/2488-pci-e-3-x8-vs-x16-
> performance-impact-on-gpus
> >
> >> Rokas Kupstys
> >>
> >> On 2016.08.05 10:34, Rokas Kupstys wrote:
> >>
> >> I think i got half-way there.. My primary gpu is at :01:00.0 and
> >> secondary on :06:00.0. I used following xorg config:
> >>
> >> Section "Device"
> >> Identifier "Device0"
> >> Driver "radeon"
> >> VendorName "AMD Corporation"
> >> 

Re: [vfio-users] Z170X IOMMU Groups

[Rich Mingin (vfio-users)]

Well, I'd been considering getting a Z170 setup, if I do, I now know what
vendor. Kudos indeed.

On Thu, Sep 22, 2016 at 2:55 PM, Alex Williamson <
alex.l.william...@gmail.com> wrote:

> On Thu, Sep 22, 2016 at 12:47 PM, Nick Sarnie 
> wrote:
>
>> Hi again,
>>
>> Very much to my surprise, Gigabyte replied and sent me a fixed BIOS. The
>> new IOMMU groups (with ACS override patch kernel commandline removed for
>> this boot), as well as my lspci information are below. I see four messages
>> the following messages in dmesg now:
>>
>> [0.523892] pci :00:1c.0: Intel SPT PCH root port ACS workaround
>> enabled
>> [0.524031] pci :00:1c.4: Intel SPT PCH root port ACS workaround
>> enabled
>> [0.524159] pci :00:1c.5: Intel SPT PCH root port ACS workaround
>> enabled
>> [0.524292] pci :00:1d.0: Intel SPT PCH root port ACS workaround
>> enabled
>>
>>
>> IOMMU Groups: http://pastebin.com/raw/0dcHk8Xk
>> lspci: http://pastebin.com/raw/1zAZuPBM
>>
>> Alex, please let me know if they missed anything else, so I can report it
>> to them.
>>
>
> :-O
>
> /me picks jaw off the ground
>
> Kudos to Gigabyte!
>
> So we now have:
>
>  100: 01 00 01 14
>
> 0x14010001 = ID 0x0001 (AER), version 0x1, next 0x140
>
> 140: 0d 00 01 22
>
> 0x2201000d = ID 0x000d (ACS), version 0x1, next 0x220
>
> 220: 19 00 01 00
>
> 0x00010019 = ID 0x0019 (Secondary PCIe), version 0x1, next 0x0
>
> So they've just added ACS into the chain, perfect.  Thanks,
>
> Alex
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] USB >>port<< passthrough

[Rich Mingin (vfio-users)]

Just to chime in, yes and no. You can't pass a port, you can pass a single
device, but it's best to pass a whole USB controller. I also picked up a
USB PCIe card for vfio use, it has two USB 3.0 ports on the PCI bracket and
a header for two front mounted ports, and I have labels from my Brother
labeller to mark those as being the VM ports. I considered using an A/B
switch like Jayme described, but ended up using a piece of software called
Synergy (popular and mentioned elsewhere on the list before) to switch
targets. I turn off scroll lock, I can mouse from host to guest and back
(and even copy/paste), and when I hit scroll lock again, the mouse and
keyboard are locked to the instance where they are currently. It works very
nicely once you're used to it, it cuts down on cabling, and you can plug in
a spare keyboard/mouse to connect directly to the guest if needed.

On Tue, Aug 23, 2016 at 2:09 PM, Jayme Howard  wrote:

> You can't pass the port itself.  You can pass individual devices attached
> to a port, but the latency can get weird sometimes.  I had pretty
> significant problems when I did it that way.  I opted to use Francisco's
> solution above, with a slight adjustment.  I have a dedicated card that
> gets allocated to the VM.  Connected to that, and to one of the host's USB
> ports, I have a USB A/B switch.  Connected to the switch is a 4 port hub.
> Connected to the hub is my keyboard and mouse.  All I have to do to switch
> the inputs from the host to the guest and back is to choose A or B on the
> switch, and a few seconds later all input is directed to the correct
> location.
>
> On Tue, Aug 23, 2016 at 5:28 AM, Quentin Deldycke <
> quentindeldy...@gmail.com> wrote:
>
>> Yes, it is best way.
>>
>> My motherboard have 2 controllers. One is all time on Linux, the other
>> one move between both.
>>
>> Keyboard mouse on the moving one, printer and other device all time on
>> Linux.
>>
>> On 23 Aug 2016 12:16 p.m., "Francisco Menendez" 
>> wrote:
>>
>>> If you don't mind spending a few dollars, what I did was basically buy
>>> a PCIe USB port card, then give it exclusive control of the device to
>>> the guest. I can't use it in the host, but I have other USB ports for
>>> the host, so no biggie.
>>>
>>> On Tue, Aug 23, 2016 at 4:07 PM, Rokas Kupstys  wrote:
>>> > Is it possible to pass-through USB port? I know we can pass-through
>>> > specific usb devices or entire usb controllers however it is not ideal
>>> > in all cases. For instance in my case single pci device drives all usb
>>> > ports on the back panel. Naturally i cant pass-through that pci device
>>> > because it would leave me without access to the host. Passing through
>>> > USB devices is of little use if one wishes to use kvm switch -
>>> > mouse/keyboard switched to another port would still have same
>>> > vendor/device ids. Is there any solution to this?
>>> >
>>> > --
>>> > Rokas Kupstys
>>> >
>>> >
>>> > ___
>>> > vfio-users mailing list
>>> > vfio-users@redhat.com
>>> > https://www.redhat.com/mailman/listinfo/vfio-users
>>>
>>> ___
>>> vfio-users mailing list
>>> vfio-users@redhat.com
>>> https://www.redhat.com/mailman/listinfo/vfio-users
>>>
>>
>> ___
>> vfio-users mailing list
>> vfio-users@redhat.com
>> https://www.redhat.com/mailman/listinfo/vfio-users
>>
>>
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] threadripper slowness

[Rich Mingin (vfio-users)]

You need to tweak NUMA because your previous CPU was not NUMA, and your new
one is. Having the VM running on some CPU cores on one physical package,
but accessing ram that's attached to other CPU cores on a different package
can cause a lot of overhead. You want to avoid that.

On Wed, Aug 14, 2019 at 7:31 PM Roger Lawhorn  wrote:

> Just a note:
> So far to resolve this I have had to use the kernel boot line options
> "rcu_nocbs=0-23 processor.max_cstate=5".
> This fixes a known hardware bug in ryzen cpus that leads to constant
> crashing in linux.
> I also compiled qemu 4.1.0-rc4.
> Though it seems better I still suffer severe lag during heavy loading of
> textures and 3d objects.
> The test games are fallout 4 and fallout 76.
> The virtual machine is stored on a 1tb intel ssd.
> On a 4 core i7 4940mx it was perfect.
> On my new 12 core threadripper it lags out.
> As long as the game isnt loading anything new it runs fine.
> I never tweaked 'numa' before and I don't see why i need to now.
>
> Any ideas?
>
>
> I am on this till it works or I am out $3000.
>
>
> On 8/12/19 6:56 AM, Roger Lawhorn wrote:
>
>
>
> On 8/11/19 9:59 PM, gamana...@gmail.com wrote:
>
> You could probably get rid of all the lag you experience. Possibly qemu 4.0
> provides better topology to the guest. Generally Threadripper CPUs have 2
> NUMA nodes, and you would want to pin the vcpus of the guest to one NUMA
> node, and also allocate RAM from the same NUMA node. You would also have to
> make sure the passthrough devices sit on corresponding PCIe lanes which that
> NUMA node controls. There is lots of good info in reddit.com/r/vfio. Please
> also provide your xml file (or script), and what you use as a virtualized
> hard drive. Is it a passthrough NVMe, or virtio-scsi? In the latter case you
> should also pin iothreads on the same NUMA node as everything else.
> You can see the topology and NUMA nodes of your cpu by running lstopo.
>
> I am eager to see a configuration that uses both NUMA nodes of a
> Threadripper CPU in one guest with minimal latency.
>
>
> -Original Message-
> From: vfio-users-boun...@redhat.com  
> 
> On Behalf Of Roger Lawhorn
> Sent: Sunday, August 11, 2019 9:31 PM
> To: vfio-users@redhat.com
> Subject: Re: [vfio-users] threadripper slowness
>
> Hello,
>
> Just wanted to mention that I got rid of a lot of the lag by compiling
>
> qemu 4.0.
>
> I dod not know if its a threadripper issue or not.
> The lag directly coincides with hard drive access.
> No hard drive access, no game lag.
> I have redhat virtio drivers installed for the hard drive and the nic.
> Are these drivers intel only?
> They are left only from my i7-4940mx processor install.
>
>
> On 8/9/19 11:51 AM, Roger Lawhorn wrote:
>
> Hello.
> I am new to the list.
> I have been doing gpu passthrough for almost 5 years now with a i7
> 4940mx cpu on a msi gt70 laptop.
>
> My new PC build is:
> MSI x399 carbon pro motherboard : bios xx.1c0 : svm=enabled:
> iommu=enabled
> Threadripper 2920 12 core
> Radeon pro duo R9 Fury X
> Nvidia 980 TI OC
>
> I was unable to passthrough the nvidia card due to it not getting past
> the "Running option rom..." message when debugging.
>
> The Radeon passed through with flying colors.
>
> The issue:
> I used the radeon to run triple A games in linux using steam for linux
> and proton. No problems.
>
> In Windows 7 in qemu I have passed all 24 cpus to it.
> They max out to 80% while playing the same triple A games and the
> games lag.
> I have never had this issue before.
>
> Admittedly I am using the same script I used on the i7 to get going.
> I had to remove threads from the -smp switch and use cores only.
>
> I also removed the kvm=off which is used for nvidia cards.
>
> I am looking for links to articles or direct info on what to do to
> optimize the cpus.
> When not lagging the radeon performs extremely well.
>
> I am concerned that perhaps kvm is not being used.
>
>
> Thanks
>
> ___
> vfio-users mailing 
> listvfio-users@redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
> ___
> vfio-users mailing 
> listvfio-users@redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
>
>
> ___
> vfio-users mailing 
> listvfio-users@redhat.comhttps://www.redhat.com/mailman/listinfo/vfio-users
>
>
> ___
> vfio-users mailing list
> vfio-users@redhat.com
> https://www.redhat.com/mailman/listinfo/vfio-users
>
___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users