Re: [vfio-users] optical eject issue

[Samuel Holland]

On 5/23/20 5:43 PM, Roger Lawhorn wrote:
> 
> Well,
> 
> Got an error:
> qemu4.0-system-x86_64: -device scsi-generic,drive=cd0: SG_GET_SCSI_ID ioctl 
> failed

I haven't seen that error before.

Make sure you're specifying the /dev/sgX node that matches your optical drive.
Try `dmesg | grep sr.*sg` to find it.

> The drive itself is sata so I don't think it will accept scsi commands.

PATA/SATA optical drives are actually all SCSI on the inside. ATAPI is SCSI
tunneled over ATA. So I think it should work :)

Cheers,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] optical eject issue

[Samuel Holland]

Hello,

On 5/23/20 12:12 AM, Roger Lawhorn wrote:
> ok, sorry, did a reply instead of a reply to all
> 
> I have used the -cdrom /dev/sr0 option.
> When you say 'virtual' optical drive, are you talking about an iso or a 
> physical
> optical drive?
> I am adding a physical optical drive.

To pass through a physical optical drive, you need to use qemu's SCSI generic
device, not a SCSI CD device. In this case, qemu will forward SCSI commands
as-is instead of adding its own emulation layer on top. This means that an
"eject" command will be handled by your physical drive, and not by qemu.

For example, from one of my own qemu scripts:

  -device virtio-scsi-pci,bus=pcie.0 \
-device scsi-hd,drive=hd0 \
  -drive file=/dev/sdc,format=raw,id=hd0,if=none,unit=0 \
-device scsi-generic,drive=cd0 \
  -drive file=/dev/sg2,format=raw,id=cd0,if=none,unit=1 \

I've successfully used this arrangement to rip dozens of CD-ROMs using CUETools.
Attempting to eject sometimes takes 3 or 4 tries, especially if the device is
still in use, but it does eventually work. Another thing you can do is run
`eject /dev/sr0` on the host, even while the VM is running, which always ejected
immediately.

Cheers,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Ongoing mouse issues

[Samuel Holland]

On 11/19/17 18:08, Patrick O'Callaghan wrote:



On 19 November 2017 at 23:13, Scott C > wrote:


You can pass your existing USB devices back and forth between host 
and guest with some commands. It’s been discussed on this mailing 
list before.


It’s scriptable


I have a vague memory of that but can't seem to find it.


https://www.redhat.com/archives/vfio-users/2016-February/msg00110.html
http://www.sholland.org/thoughts.html#2016-02-29

Regards,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Trying to pass through USB controller.

[Samuel Holland]

Hello,

On 08/31/17 22:45, Andrew A. wrote:

I tried the following to no avail:
setpci -s 00:14.0 0xd4.L=0x
setpci -s 00:14.0 0xdc.L=0x

This seems not to have taken hold upon readback:
setpci -s 00:14.0 0xd4.L
3fff
setpci -s 00:14.0 0xdc.L
003f


This is also what I get on my machine.


(And indeed, the same problem persists -- qemu changes the port
mappings as soon as I boot a guest, locking me out of my host machine
until I ssh in and run 'setpci .. d0' again)

I took a look at what I believe is the R/WL property for these
settings, which is bit 31 of D20:F0:40h according to the datasheet,
so:
setpci -s 00:14.0 0x40.L
803609fd
sudo setpci -s 00:14.0 0x40.L=0x003609fd


Note that the ACCTRL bit is not documented in that register, further
suggesting that it is a BIOS-only control bit. Likely it is a write-once
bit, so once it has been set, it cannot be unset.


But this again didn't seem to have an effect:
setpci -s 00:14.0 0x40.L
803609fd


Note that other bits in the register can be changed (e.g. you can change
the 6 to a 0 to turn off power management, and change it back to a 6).
Just the top bit is write-protected.

Another potential problem is that your BIOS on the host may 
write-protect the mask register. In that case, see the datasheet 
for how to unlock it, if possible.


Right, and despite trying to remove the write protect I'm inclined
to think my unlocking commands are not working for some reason. Do
you see anything wrong with what I've tried here?


No, other than that it doesn't work :) I guess we're just out of luck
then, without doing something like patching your BIOS (so the bit never
gets set).


Thanks.


Regards,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

[vfio-users] Bisected an issue with OVMF

[Samuel Holland]

Hello,

I've been using vfio to pass through a GPU[1] to a QEMU virtual machine
for several years. I've been using OVMF since some time in 2015.
Initially, I updated my OVMF binary from Gerd's repo on a regular basis,
but some time around February 2016, the OVMF binaries stopped booting. I
could pass through other devices (EHCI controllers, audio) fine, but if
I attached the GPU, I would have one core stuck at 100% usage
indefinitely, and a blank screen and no serial console output from OVMF.

The OVMF hang would happen regardless of the number of cores, or the
amount of RAM given to the VM, or whether or not the GPU was attached to
a root port.

By the time I got around to investigating the issue, I'd forgotten the
date of the last good revision (I only had the fd images on hand, not
the RPM), and nobody else had heard of my issue. This spring, I tried
compiling various revisions of the edk2 source repo, but none of them
booted.

Thanks to the recent thread where a couple of old OVMF builds were
posted, I was finally able to bisect the issue! The build[2] from 2015
booted! The one from mid-2016 did not, as expected. Anyway, I tracked
the breakage down to this commit:

commit 7daf2401d420573f50e3d00ae3a89e54914ef056
Author: Laszlo Ersek 
Date:   Fri Mar 4 01:49:54 2016 +0100

OvmfPkg: PciHostBridgeLib: permit access to the full extended 
config space


By now OVMF makes MdeModulePkg/Bus/Pci/PciHostBridgeDxe go through
MMCONFIG (when running on Q35). Enable the driver to address each 
B/D/F's

config space up to and including offset 0xFFF.

With that commit reverted, current edk2 master (2c2c68b9d3e8) also boots
successfully. I don't know if there's a bug somewhere in the software
stack, or if my graphics card has a bug and needs a quirk, or what the
issue is. But I'd love to know why this doesn't work, and get whatever's
broken fixed.

For what it's worth, back when I first started using this GPU and
motherboard[3] combination, I had to manually apply this patch[4] to
QEMU for assignment to work, until it was included in a released version
of QEMU.

Thanks,
Samuel

[1]: https://www.gigabyte.com/Graphics-Card/GV-N660OC-2GD
[2]: 
https://github.com/jkoelndorfer/local-tools/blob/master/workstation/vfio/edk2.git-ovmf-x64-0-20150804.b1143.g8ca1489.noarch.rpm

[3]: http://www.asrock.com/mb/Intel/Z97E-ITXac/
[4]: 
https://github.com/qemu/qemu/commit/f5793fd9e1fd89808f4adbfe690235b094176a37


___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Newbie steps

[Samuel Holland]

Hello,

On 03/11/17 07:24, Patrick O'Callaghan wrote:

GRUB_CMDLINE_LINUX="rd.blacklist=nvidia rd.blacklist=nouveau
intel_iommu=on iommu=pt rd.driver.pre=vfio-pci
vconsole.font=latarcyrheb-sun16 $([ -x /usr/sbin/rhcrashkernel-param
] && /usr/sbin/rhcrashkernel-param || :) rhgb quiet resume=U
UID=1431e6d2-531e-46cd-8633-1cf878c6b2a1" audit=0

(Note that I have the Nvidia proprietary driver so need to mask that
as well as Nouveau).


rd.* options only matter inside the initramfs. Once your root filesystem
is mounted and udev is restarted, it will pick up the nvidia drivers
at that point. You need to mask nvidia and nouveau both in the initramfs
and once the system is booted.

Change rd.blacklist to modprobe.blacklist, and regenerate your GRUB config.

On 03/11/17 15:54, Patrick O'Callaghan wrote:

On 11 March 2017 at 21:26, wiwitop wiwitop wrote:

Hi. You should try with the real ids of your gpu. The trick didn't
work with my AMD GPU, perhaps it's the same for you.

I'm not sure what you mean by "real ids". How is this different from
what I'm doing?


Specify the PCI IDs (vendor and product) as they appear in lspci -nn,
not with the really long string (replacing with the correct product
numbers for your card, of course):

options vfio-pci ids=10de:abcd,10de:1234

Then be sure to regenerate your initramfs.

Regards,
Samuel Holland

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Trying to pass through USB controller.

[Samuel Holland]

Sorry for the delay getting back to you,

On 03/01/17 14:32, Andrew A. wrote:

Does anyone know of a sensible way to more permanently set the port
routing mask so it survives these pci resets?


Unfortunately, the xHCI controller fully controls which ports go where.
That means if you give it to the guest, since the guest driver controls
the PCI device, it controls the ports too. It's not necessarily the PCI
reset causing the switch--see for example the Linux driver:

http://lxr.free-electrons.com/source/drivers/usb/host/pci-quirks.c#L926

It moves all available ports to xHCI on powerup (including waking up
from runtime power-management). What you can try is setting the "xHC USB
2.0 Port Routing Mask Register" (section 16.1.37 in the 9 series PCH
datasheet). If you look at line 930 of the source linked above, you see
that the Linux driver only tries to switch the ports marked as
switchable. This should also work on Windows guests, as the datasheet
specifies "When set to 0, The OS shall not modify the corresponding
USB2HCSEL bit". Of course, since it's closed-source, there's no way to
know if Windows behaves properly without trying.

Masking off those ports so the OS doesn't switch them should not prevent
you from switching them manually. This also means that even if you get
the guest drivers to cooperate, a malicious application running in the
guest could take over your keyboard.

Another potential problem is that your BIOS on the host may
write-protect the mask register. In that case, see the datasheet for how
to unlock it, if possible.


Thanks for writing this up last year,


I'm glad it has been of use!


Andrew


Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] remote 3D acceleration with gpu passthrough

[Samuel Holland]

Hello,

On 12/21/16 20:01, Chen Fan wrote:

I assigned a GPU to a virtual machine, how can I get a good
performance with remote connection?


If you are on a low-latency, connection, I recommend Steam streaming.
You can use it with any application/game by adding it as a non-Steam
game to your library.

For across the internet, RDP is generally much better than VNC.


Thanks,
Chen


Regards,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Linux 4.8.x

[Samuel Holland]

Hello,

On 10/22/2016 12:54 PM, Philip Abernethy wrote:

as far as I'm aware the ACS patch is part of the mainline kernel
since 4.7. I'm currently running a VM that requires ACS in its
current configuration and I'm just using the linux package from the
Arch repos.


The ACS quirks for Skylake are in 4.7, reducing the need for the ACS
override patch. The ACS override patch will never be in the mainline kernel.


Philip


Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] USB >>port<< passthrough

[Samuel Holland]

Hello,

On 08/23/2016 02:07 AM, Rokas Kupstys wrote:

Is it possible to pass-through USB port? I know we can pass-through
specific usb devices or entire usb controllers however it is not
ideal in all cases.


Sorry for all of the not-quite-accurate answers you've gotten. You
_can_, in fact, pass through a USB port by its location:

$ qemu-system-x86_64 -device usb-host,? 2>&1 | sort
usb-host.bootindex=int32
usb-host.full-path=bool (on/off)
usb-host.hostaddr=uint32
usb-host.hostbus=uint32
usb-host.hostport=str
usb-host.isobsize=uint32
usb-host.isobufs=uint32
usb-host.loglevel=uint32
usb-host.msos-desc=bool (on/off)
usb-host.pipeline=bool (on/off)
usb-host.port=str
usb-host.productid=uint32
usb-host.serial=str
usb-host.vendorid=uint32

See the hostbus and hostport options. I don't know if this is exposed by
libvirt, since I don't use that, but at least qemu has support for it.


Is there any solution to this?


Passing through a port using `-device usb-host,hostbus=foo,hostport=bar`
is one solution. Another solution can be to use your several integrated
USB controllers (if you have an Intel chipset). Even though all of your
devices may show up on one controller, you can change which controller
the ports are attached to in the PCI configuration space. See [1] and 
the thread at [2].


Hope that helps,
Samuel

[1] http://sholland.org/thoughts.html#2016-02-29
[2] https://www.redhat.com/archives/vfio-users/2016-February/msg00110.html

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] UEFI not respecting video output selection

[Samuel Holland]

Hello,

On 07/28/2016 10:39 AM, Ryan A Young wrote:

Forgot to add my motherboard. ASRock H97M.


This seems to be a problem with all ASRock boards, at least 9-series. Go
into the firmware setup. First, set fast boot to disabled. then set CSM
to enabled. Then go into the CSM configuration. You can set boot mode to
UEFI only, since I assume you're using UEFI boot on the host already.
The important setting is to change the video option ROM to Legacy. The
other option ROMs can be set to UEFI or disabled (depending on if you
use PXE or chipset RAID).

Regards,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] NVIDIA GPU Passthrough to Win10 - Driver Disabled (Code 43)

[Samuel Holland]

On 07/25/2016 04:29 PM, Steven Bell wrote:

On Mon, Jul 25, 2016 at 5:23 PM, Philip Abernethy wrote:

Did you make sure that the card actually supports UEFI mode?
Because my GTX 660 didn't. But the OS (Arch in my case) would
happily boot with the UEFI set to UEFI-only mode and the card would
silently run in legacy mode. I had to update the card's ROM. I used
nvflash to check the card's UEFI capability and update the ROM.


Interesting. I'll give the nvflash a try and post back my results.


Note that you do not need to actually flash a new ROM to the card to use
it with qemu/KVM. You can just specify a path to the ROM file in your
XML. This is much safer than flashing your card, especially if there is
no UEFI-capable firmware available for your specific video card model.

Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] vm lags/pauses when IO overloaded on host

[Samuel Holland]

On 07/20/2016 02:41 AM, Marcin Falkiewicz wrote:

ZFS's zvol is really bad for virtual disk backend when you have only
few HDDs (or even SSDs) - I had a lot problems with latency and
throughput on my 2-disk mirrored testbench. Hard to tell if it's ZFS
itself, ZoL or just not enough disks and slow controller.


I haven't had any problems with zvols on a mirror of 7200RPM drives.
It's not as snappy as my SSD, for sure, but it's reasonably fast, and I
haven't noticed it bogging down the host.

Some things to consider: ARC size and logbias. I have 16GiB of RAM, and
8GiB goes to the VM, so I limited my ARC to 4 GiB to avoid memory
pressure issues on the host. Normally it would use half of RAM, but that
would make host applications have to constantly reclaim memory from the
cache.

Second, I have logbias=throughput on the whole pool. I do not have a
ZIL. There's a few pathological cases (on the host side) where I can get
my desktop to lock up for a minute or two, such as rsyncing several
hundred GiB of files from the SSD, but generally the performance is
better than logbias=latency (which is the default). I find there's
actually less stuttering. Plus it decreases fragmentation and makes my
drives quieter (less seeking).


On the other hand, LVM runs great on single SSD (directsync+native)
or even RAID1 HDDs (mdadm; none+native), achieving close to bare
metal performance with virtio-scsi.


Out of curiosity, are you using scsi-hd, scsi-block, or scsi-generic?

--
Regards,
Samuel

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Preventing host from going into sleep

[Samuel Holland]

On 06/22/2016 12:01 AM, Nicolas Roy-Renaud wrote:

By the looks of it, I can't write a wrapper script for qemu-system
either since libvirt doesn't launch it with the same capabilities as
a regular user, and the dbus calls coming from sysd-inhibit fail.


You should be able to change the permissions to the dbus interface so
that you can call it from a wrapper.

--
Regards,
Samuel Holland <sam...@sholland.org>

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Distinguishing Logical and HT Cores, Cpu-latencies Script Results

[Samuel Holland]

On 03/04/2016 05:37 PM, Jeff wrote:

The output of lstopo shows that 0-6, 1-7, etc... are paired, which
is what I originally assumed (as a 4790k is 0-4, 1-5, with similar
architecture so I'd assume similar), but different from the virsh
capabilities output (which did not list siblings correctly for me)


If you want to get the information directly from the kernel, try:

grep . /sys/bus/cpu/devices/*/topology/core_id
grep . /sys/bus/cpu/devices/*/topology/thread_siblings_list


Pic: https://www.dropbox.com/s/m79l36e32f08b8d/out.png?dl=0 XML:
https://www.dropbox.com/s/wr93edyszlpcosm/summary.xml?dl=0 So with
all of that said, is it safe to say the pairings are 0-6, 1-7, 2-8,
3-9, 4-10, 5-11 for a 5930k?


That seems to be generally how it works. My Xeon X5675 also has:

/sys/bus/cpu/devices/cpu0/topology/thread_siblings_list:0,6
/sys/bus/cpu/devices/cpu1/topology/thread_siblings_list:1,7
/sys/bus/cpu/devices/cpu2/topology/thread_siblings_list:2,8
/sys/bus/cpu/devices/cpu3/topology/thread_siblings_list:3,9
/sys/bus/cpu/devices/cpu4/topology/thread_siblings_list:4,10
/sys/bus/cpu/devices/cpu5/topology/thread_siblings_list:5,11
/sys/bus/cpu/devices/cpu6/topology/thread_siblings_list:0,6
/sys/bus/cpu/devices/cpu7/topology/thread_siblings_list:1,7
/sys/bus/cpu/devices/cpu8/topology/thread_siblings_list:2,8
/sys/bus/cpu/devices/cpu9/topology/thread_siblings_list:3,9
/sys/bus/cpu/devices/cpu10/topology/thread_siblings_list:4,10
/sys/bus/cpu/devices/cpu11/topology/thread_siblings_list:5,11

I know that in the past, there could be some variation between boots,
but that seems to no longer be the case.


Just want to make sure.

> Thanks!

--
Regards,
Samuel Holland <sam...@sholland.org>

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users

Re: [vfio-users] Trying to pass through USB controller.

[Samuel Holland]

Hello,

On 02/23/2016 02:06 PM, Jonathan Scruggs wrote:

Okay, I disabled xHCI in the bios and mapped all the ports. I want
all 6 on Host Controller #2 to be passed through to the quest. It's
00:1a.0. I select that in Libvirt, so how do I make the mask which I
want permanently passed through.


You want a mask of 00   
   \_/ \___/
   EHCI #2  EHCI #1

In hexadecimal, that's 0x00ff. If you want it to be permanent, then you
can just stick the setpci command in /etc/rc.local or somewhere, and not
bother with any of the toggling setup.


There are 6 ports on Controller #2. For Controller #1, there are 10
ports. However, 4 are under a hub that is on port 5. I could not
find a port 6 for controller #1.


There should be exactly 8 ports on controller #1, ignoring any hubs
connected downstream.


BTW, if you have one guest, you may want to check out the patches
for qemu that use your host keyboard and mouse and pass them through
as a PS2/USB device and you use both ctrl keys to switch back and
forth.


That sounds interesting, but my main reason for doing this is to avoid
USB passthrough. I originally set this up to use the onboard Bluetooth
module in Windows, with bluetooth headphones, so I could avoid audio
emulation (since my monitor doesn't support audio over DVI). When I used
USB passthrough, playback would stutter every minute or so; this way
works flawlessly. Keyboard and mouse was an afterthought.

--
Regards,
Samuel Holland <sam...@sholland.org>

___
vfio-users mailing list
vfio-users@redhat.com
https://www.redhat.com/mailman/listinfo/vfio-users