Re: [Xen-devel] PCI passthrough of USB controllers on Xen 4.8.1, Linux 4.9.29, stubdomain
On Tue, Jun 06, 2017 at 04:55:20AM -0600, Jan Beulich wrote:
> >>> On 06.06.17 at 12:41, wrote:
> > [root@dom0 ~]# lspci -s 00:14.0 -vvv
> > 00:14.0 USB controller: Intel Corporation Wildcat Point-LP USB xHCI
> > Controller (rev 03) (prog-if 30 [XHCI])
> > Subsystem: Intel Corporation Device 7270
> > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr-
> > Stepping- SERR- FastB2B- DisINTx+
> > Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> > SERR- > Latency: 0
> > Interrupt: pin A routed to IRQ 170
> > Region 0: Memory at b220 (64-bit, non-prefetchable) [size=64K]
> > Capabilities: [70] Power Management version 2
> > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA
> > PME(D0-,D1-,D2-,D3hot+,D3cold+)
> > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
> > Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+
> > Address: fee00338 Data:
>
> So as I did expect the field accessed is the MSI capability structure.
> Such accesses shouldn't reach pciback, but instead be taken care
> of by qemu. I can't really comment on the qemu side though, but
> this at least makes me think the underlying cause of the problems
> you see with the two controllers is the same.
>
> Is this a regression of some sort, i.e. did the same setup work in
> earlier Xen versions?
This was working with pure PV, but this is probably irrelevant here.
I can't test if switching only Xen version changes anything (that would
require a lot of recompiling), but very similar setup with mini-os based
stubdom on Xen 4.6.5 and Xen 4.8.1 also doesn't work. Even with a qemu
patch disabling MSI reporting (attached). In that case, I don't get any
message from qemu, but very similar messages from xhci driver.
BTW Other patches:
https://github.com/QubesOS/qubes-vmm-xen/tree/xen-4.8/patches.misc
I've also checked Xen 4.8.1 with qemu-upstream in dom0 (instead of
Linux-based stubdom) and it also doesn't work, but with slightly
different messages:
qemu:
[00:05.0] Write-back to unknown field 0xd8 (partially) inhibited
(0x)
[00:05.0] If the device doesn't work, try enabling permissive mode
[00:05.0] (unsafe) and if it helps report the problem to xen-devel
[00:06.0] Write-back to unknown field 0x6c (partially) inhibited
(0x)
[00:06.0] If the device doesn't work, try enabling permissive mode
[00:06.0] (unsafe) and if it helps report the problem to xen-devel
Linux in domU:
[ 51.679188] xhci_hcd :00:05.0: Error while assigning device slot ID
[ 51.679264] xhci_hcd :00:05.0: Max number of devices this xHCI host
supports is 32.
[ 51.679359] usb usb1-port2: couldn't allocate usb_device
(and nothing about ehci, no devices connected to it is visible)
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
MiniOS + QEMU do not appear to work with either MSI or MSI-X. Some guests
or device drivers do not gracefully handle being told a PCI device has
MSI/MSI-X and then they don't actually get it. Disable the MSI and MSI-X
capability reporting in PCI config space, making only INTX legacy interrupts
available.
Signed-off-by: Eric Shelton
--- a/tools/qemu-xen-traditional/hw/pass-through.c 2016-10-31
12:48:42.924026468 -0400
+++ b/tools/qemu-xen-traditional/hw/pass-through.c 2016-10-31
12:50:52.953026468 -0400
@@ -874,6 +874,10 @@
.grp_size = 0x04,
.size_init = pt_reg_grp_size_init,
},
+#ifndef CONFIG_STUBDOM
+/* At present stubdom doesn't support MSI for passthrough, so let's not
+ * expose MSI capability to stubdom HVM guest for now.
+ */
#ifndef __ia64__
/* At present IA64 Xen doesn't support MSI for passthrough, so let's not
* expose MSI capability to IA64 HVM guest for now.
@@ -886,7 +890,8 @@
.size_init = pt_msi_size_init,
.emu_reg_tbl= pt_emu_reg_msi_tbl,
},
-#endif
+#endif /* __ia64__ */
+#endif /* !CONFIG_STUBDOM */
/* PCI-X Capabilities List Item reg group */
{
.grp_id = PCI_CAP_ID_PCIX,
@@ -931,6 +936,10 @@
.size_init = pt_pcie_size_init,
.emu_reg_tbl= pt_emu_reg_pcie_tbl,
},
+#ifndef CONFIG_STUBDOM
+/* At present stubdom doesn't support MSI for passthrough, so let's not
+ * expose MSI-X capability to stubdom HVM guest for now.
+ */
#ifndef __ia64__
/* At present IA64 Xen doesn't support MSI for passthrough, so let's not
* expose MSI-X capability to IA64 HVM guest for now.
@@ -943,7 +952,8 @@
.size_init = pt_msix_size_init,
.emu_reg_tbl= pt_emu_reg_msix_tbl,
},
-#endif
+#endif /* __ia64__ */
+#endif /* !CONFIG_STUBDOM */
{
.grp_size = 0,
},
signature.asc
Description: PGP signature
___
Xen-devel
Re: [Xen-devel] PCI passthrough of USB controllers on Xen 4.8.1, Linux 4.9.29, stubdomain
>>> On 06.06.17 at 12:41, wrote: > [root@dom0 ~]# lspci -s 00:14.0 -vvv > 00:14.0 USB controller: Intel Corporation Wildcat Point-LP USB xHCI > Controller (rev 03) (prog-if 30 [XHCI]) > Subsystem: Intel Corporation Device 7270 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- > SERR-Latency: 0 > Interrupt: pin A routed to IRQ 170 > Region 0: Memory at b220 (64-bit, non-prefetchable) [size=64K] > Capabilities: [70] Power Management version 2 > Flags: PMEClk- DSI- D1- D2- AuxCurrent=375mA > PME(D0-,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- > Capabilities: [80] MSI: Enable+ Count=1/8 Maskable- 64bit+ > Address: fee00338 Data: So as I did expect the field accessed is the MSI capability structure. Such accesses shouldn't reach pciback, but instead be taken care of by qemu. I can't really comment on the qemu side though, but this at least makes me think the underlying cause of the problems you see with the two controllers is the same. Is this a regression of some sort, i.e. did the same setup work in earlier Xen versions? Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] PCI passthrough of USB controllers on Xen 4.8.1, Linux 4.9.29, stubdomain
On Tue, Jun 06, 2017 at 02:37:01AM -0600, Jan Beulich wrote: > >>> On 02.06.17 at 12:57, wrote: > > And in this case, dom0 also prints: > > > > [ 49.155606] pciback :00:14.0: Driver tried to write to a > > read-only configuration space field at offset 0x82, size 2. This may be > > harmless, but if you have problems with your device: > >1) see permissive attribute in sysfs > >2) report problems to the xen-devel mailing list along > > with details of your device obtained from lspci. > > [ 66.247644] pciback :00:14.0: cache line size of 64 is not > > supported > > [ 66.247646] xen_pciback: :00:14.0: cannot enable > > memory-write-invalidate (-22) > > > > Enabling permissive mode doesn't change anything. > > I doubt this - the first of the messages won't be logged in permissive > mode. Yes, of course. But the other lines and overall effect is the same. > We'll also need to know what register there is at address 0x82 > (possibly visible from a sufficiently verbose lspci in Dom0). > > As to the latter two - lspci output may also help understand > what the issue with cache line size here is. A second source of > information may be lspci output for the device with its normal > driver loaded and attached in Dom0. Below is lspci of those two devices, in dom0, with normal driver attached. Would lspci from domU be useful too? [root@dom0 ~]# lspci -s 00:1d.0 -vvv 00:1d.0 USB controller: Intel Corporation Wildcat Point-LP USB EHCI Controller (rev 03) (prog-if 20 [EHCI]) Subsystem: Intel Corporation Device 7270 Control: I/O- Mem- BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- SERR- TAbort- SERR- signature.asc Description: PGP signature ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] PCI passthrough of USB controllers on Xen 4.8.1, Linux 4.9.29, stubdomain
>>> On 02.06.17 at 12:57, wrote: > And in this case, dom0 also prints: > > [ 49.155606] pciback :00:14.0: Driver tried to write to a > read-only configuration space field at offset 0x82, size 2. This may be > harmless, but if you have problems with your device: >1) see permissive attribute in sysfs >2) report problems to the xen-devel mailing list along > with details of your device obtained from lspci. > [ 66.247644] pciback :00:14.0: cache line size of 64 is not > supported > [ 66.247646] xen_pciback: :00:14.0: cannot enable > memory-write-invalidate (-22) > > Enabling permissive mode doesn't change anything. I doubt this - the first of the messages won't be logged in permissive mode. We'll also need to know what register there is at address 0x82 (possibly visible from a sufficiently verbose lspci in Dom0). As to the latter two - lspci output may also help understand what the issue with cache line size here is. A second source of information may be lspci output for the device with its normal driver loaded and attached in Dom0. Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] PCI passthrough of USB controllers on Xen 4.8.1, Linux 4.9.29, stubdomain
Hi, I'm trying to passthrough USB controllers (details below) to a HVM domain with qemu running in stubdomain. There are two of them (EHCI and xHCI), neither of them works. The stubdomain is non-standard Linux based with Qemu 2.8.0. Both dom0 and guest are running Linux 4.9.29 very close to vanilla. I've started debugging with EHCI. On modprobe ehci-pci I get: [52453.633768] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [52453.634496] ehci-pci: EHCI PCI platform driver [52453.640088] ehci-pci :00:06.0: EHCI Host Controller [52453.640466] ehci-pci :00:06.0: new USB bus registered, assigned bus number 5 [52462.453251] ehci-pci :00:06.0: can't setup: -110 [52462.453315] ehci-pci :00:06.0: USB bus 5 deregistered [52462.458375] ehci-pci :00:06.0: init :00:06.0 fail, -110 [52462.458421] ehci-pci: probe of :00:06.0 failed with error -110 And at this time, stubdomain print this: [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 [00:06.0] xen_pt_bar_read: Error: Should not read BAR through QEMU. @0x0004 repeated a lot more times, sometimes with slightly different addresses and/or for write. Just to be sure, I've checked with mini-os based stubdomain, but there I don't even see the EHCI one (only xHCI, which doesn't work the same way as on updated stubdomain). Regarding xHCI, the domU messages are (this one is from older kernel, but the same happens on 4.9 too): [ 347.822393] xhci_hcd :00:05.0: xHCI Host Controller [ 347.822764] xhci_hcd :00:05.0: new USB bus registered, assigned bus number 2 [ 347.825391] xhci_hcd :00:05.0: hcc params 0x200077c1 hci version 0x100 quirks 0x0004b810 [ 347.832346] usb usb2: New USB device found, idVendor=1d6b, idProduct=0002 [ 347.832482] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 347.832643] usb usb2: Product: xHCI Host Controller [ 347.832779] usb usb2: Manufacturer: Linux 4.8.13-100.fc23.x86_64 xhci-hcd [ 347.832970] usb usb2: SerialNumber: :00:05.0 [ 347.833867] hub 2-0:1.0: USB hub found [ 347.834089] hub 2-0:1.0: 11 ports detected [ 347.835127] xhci_hcd :00:05.0: xHCI Host Controller [ 347.836927] xhci_hcd :00:05.0: new USB bus registered, assigned bus number 3 [ 347.837382] usb usb3: New USB device found, idVendor=1d6b, idProduct=0003 [ 347.837582] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 347.837784] usb usb3: Product: xHCI Host Controller [ 347.837921] usb usb3: Manufacturer: Linux 4.8.13-100.fc23.x86_64 xhci-hcd [ 347.838159] usb usb3: SerialNumber: :00:05.0 [ 347.838942] hub 3-0:1.0: USB hub found [ 347.839158] hub 3-0:1.0: 4 ports detected [ 363.487226] xhci_hcd :00:05.0: Error while assigning device slot ID [ 363.487368] xhci_hcd :00:05.0: Max number of devices this xHCI host supports is 32. [ 363.487508] usb usb2-port1: couldn't allocate usb_device [ 383.967268] xhci_hcd :00:05.0: Error while assigning device slot ID [ 383.967495] xhci_hcd :00:05.0: Max number of devices this xHCI host supports is 32. [ 383.967715] usb usb2-port2: couldn't allocate usb_device And in this case, dom0 also prints: [ 49.155606] pciback :00:14.0: Driver tried to write to a read-only configuration space field at offset 0x82, size 2. This may be harmless, but if you have problems with your device: 1) see permissive attribute in sysfs 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci. [ 66.247644] pciback :00:14.0: cache line size of 64 is not supported [ 66.247646] xen_pciback: :00:14.0: cannot enable memory-write-invalidate (-22) Enabling permissive mode doesn't change anything. Any idea what's wrong? -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature ___ Xen-dev
