> Date: Fri, 17 Jun 2022 17:22:58 +1000
> From: Jonathan Gray <[email protected]>
>
> On Fri, Jun 17, 2022 at 12:33:47AM +0200, Mark Kettenis wrote:
> > On the Ampere Altra machines, some PCIe devices show up 32 times; once
> > for each possible PCI device number. This is a hardware bug, since a
> > downstream switch port (or root port) is supposed to terminate
> > configuration request targeted at device numbers 1-31. But it is a
> > somewhat common bug since I have seen this before on other hardware.
> > Linux and FreeBSD both have code that only scans device 0 on
> > downstream switch ports. So let's do that in OpenBSD as well.
> >
> > ok?
> >
>
> ok jsg@
>
> Linux and NetBSD also handle additional types as downstream.
> Referred to in the pcie base spec as:
>
> 0100b Root Port of PCI Express Root Complex*
> 1000b PCI/PCI-X to PCI Express Bridge*
>
> *This value is only valid for Functions that implement a Type
> 01h PCI Configuration Space header.
>
> FreeBSD also checks for Root Port but not the bridge.
Ah, I missed that Linux also checks for the root port. That's
probably a good idea. The PCI-PCIe bridge is probably rare enough not
to matter, but lets align with Linux and NetBSD here.
How about this?
Index: dev/pci/pci.c
===================================================================
RCS file: /cvs/src/sys/dev/pci/pci.c,v
retrieving revision 1.124
diff -u -p -r1.124 pci.c
--- dev/pci/pci.c 11 Mar 2022 18:00:51 -0000 1.124
+++ dev/pci/pci.c 17 Jun 2022 08:34:30 -0000
@@ -807,11 +807,31 @@ pci_enumerate_bus(struct pci_softc *sc,
{
pci_chipset_tag_t pc = sc->sc_pc;
int device, function, nfunctions, ret;
+ int maxndevs = sc->sc_maxndevs;
const struct pci_quirkdata *qd;
- pcireg_t id, bhlcr;
+ pcireg_t id, bhlcr, cap;
pcitag_t tag;
- for (device = 0; device < sc->sc_maxndevs; device++) {
+ /*
+ * PCIe downstream ports and root ports should only forward
+ * configuration requests for device number 0. However, not
+ * all hardware implements this correctly, and some devices
+ * will respond to other device numbers making the device show
+ * up 32 times. Prevent this by only scanning a single
+ * device.
+ */
+ if (sc->sc_bridgetag && pci_get_capability(pc, *sc->sc_bridgetag,
+ PCI_CAP_PCIEXPRESS, NULL, &cap)) {
+ switch (PCI_PCIE_XCAP_TYPE(cap)) {
+ case PCI_PCIE_XCAP_TYPE_RP:
+ case PCI_PCIE_XCAP_TYPE_DOWN:
+ case PCI_PCIE_XCAP_TYPE_PCI2PCIE:
+ maxndevs = 1;
+ break;
+ }
+ }
+
+ for (device = 0; device < maxndevs; device++) {
tag = pci_make_tag(pc, sc->sc_bus, device, 0);
bhlcr = pci_conf_read(pc, tag, PCI_BHLC_REG);
Index: dev/pci/pcireg.h
===================================================================
RCS file: /cvs/src/sys/dev/pci/pcireg.h,v
retrieving revision 1.60
diff -u -p -r1.60 pcireg.h
--- dev/pci/pcireg.h 31 Dec 2021 11:24:24 -0000 1.60
+++ dev/pci/pcireg.h 17 Jun 2022 08:34:30 -0000
@@ -563,6 +563,10 @@ typedef u_int8_t pci_revision_t;
#define PCI_PCIE_XCAP 0x00
#define PCI_PCIE_XCAP_SI 0x01000000
#define PCI_PCIE_XCAP_VER(x) (((x) >> 16) & 0x0f)
+#define PCI_PCIE_XCAP_TYPE(x) (((x) >> 20) & 0x0f)
+#define PCI_PCIE_XCAP_TYPE_RP 0x4
+#define PCI_PCIE_XCAP_TYPE_DOWN 0x6
+#define PCI_PCIE_XCAP_TYPE_PCI2PCIE 0x8
#define PCI_PCIE_DCAP 0x04
#define PCI_PCIE_DCSR 0x08
#define PCI_PCIE_DCSR_ERO 0x00000010
