> Date: Tue, 25 Aug 2020 18:00:11 +0000 > From: Mikolaj Kucharski <[email protected]> > Reply-To: [email protected], [email protected] > Content-Type: text/plain; charset=us-ascii > Content-Disposition: inline > > On Sun, Aug 16, 2020 at 01:23:54PM +0000, Mikolaj Kucharski wrote: > > On Mon, Aug 10, 2020 at 08:34:36PM +0000, Mikolaj Kucharski wrote: > > > >Synopsis: LTE mini-PCIe modem not showing up after reboot > > > >Category: kernel > > > >Environment: > > > System : OpenBSD 6.7 > > > Details : OpenBSD 6.7-current (GENERIC.MP) #15: Sun Aug 9 17:48:40 > > > MDT 2020 > > > > > > [email protected]:/usr/src/sys/arch/amd64/compile/GENERIC.MP > > > > > > Architecture: OpenBSD.amd64 > > > Machine : amd64 > > > >Description: > > > I have PC Engines APU2 board with Sierra Wireless MC7455 LTE modem on > > > mini-PCIe. > > > When I cold boot the system, device is properly detected: > > > > > > uhub2 at uhub1 port 1 configuration 1 interface 0 "Advanced Micro Devices > > > Hub" rev 2.00/0.18 addr 2 > > > umsm0 at uhub2 port 4 configuration 1 interface 0 "Sierra Wireless, > > > Incorporated Sierra Wireless MC7455 Qualcomm\M-. Snapdragon? X7 LTE-A" > > > rev 2.10/0.06 addr 3 > > > ucom0 at umsm0 > > > umsm1 at uhub2 port 4 configuration 1 interface 2 "Sierra Wireless, > > > Incorporated Sierra Wireless MC7455 Qualcomm\M-. Snapdragon? X7 LTE-A" > > > rev 2.10/0.06 addr 3 > > > ucom1 at umsm1 > > > umsm2 at uhub2 port 4 configuration 1 interface 3 "Sierra Wireless, > > > Incorporated Sierra Wireless MC7455 Qualcomm\M-. Snapdragon? X7 LTE-A" > > > rev 2.10/0.06 addr 3 > > > ucom2 at umsm2 > > > umsm3 at uhub2 port 4 configuration 1 interface 8 "Sierra Wireless, > > > Incorporated Sierra Wireless MC7455 Qualcomm\M-. Snapdragon? X7 LTE-A" > > > rev 2.10/0.06 addr 3 > > > ucom3 at umsm3 > > > umsm4 at uhub2 port 4 configuration 1 interface 10 "Sierra Wireless, > > > Incorporated Sierra Wireless MC7455 Qualcomm\M-. Snapdragon? X7 LTE-A" > > > rev 2.10/0.06 addr 3 > > > ucom4 at umsm4 > > > > > > However when I reboot OpenBSD Sierra Wireless card is no longer detected > > > by the OS and > > > is not visible any more from the system: > > > > > > uhub2 at uhub1 port 1 configuration 1 interface 0 "Advanced Micro Devices > > > Hub" rev 2.00/0.18 addr 2 > > > > > > >How-To-Repeat: > > > Cold boot APU2 with 1199:9071 Sierra Wireless card, then reboot the OS > > > and card is missing. > > > > > > >Fix: > > > Unknown. I've looked around and found following resources: > > > > > > - OpenBSD misc, broken EHCI USB on AMD chipset? > > > https://marc.info/?t=151192838800001&r=1&w=2 > > > > > > - PCI: Workaround AMD EHCI controller PME bug > > > https://patchwork.kernel.org/patch/9797105/ > > > > > > - usb: host: ehci: workaround PME bug on AMD EHCI controller > > > https://patchwork.kernel.org/patch/9783041/ > > > > > > - Linux kernel, pci_fixup_amd_ehci_pme() function > > > https://github.com/torvalds/linux/blob/fc80c51fd4b23ec007e88d4c688f2cac1b8648e7/arch/x86/pci/fixup.c#L580-L593 > > > > > > - SB700 Family Product Errata > > > https://www.amd.com/system/files/TechDocs/46837.pdf > > > > > > - AMD SB700/710/750 Register Programming Requirements > > > http://ftp.loongnix.org/doc/02data%20sheet/loongson3a/SB/42413_sb7xx_rpr_pub_1.00.pdf > > > > > > Unfortunately, I don't know how to apply solution from Linux code to > > > OpenBSD :/ > > > > This is what I came up for a moment, but it doesn't work. If someone > > sees why, I would appreciated a hint why this approach is wrong. Here is > > my debugging patch. After the patch more comments. > > > > > > Index: dev/pci/ehci_pci.c > > =================================================================== > > RCS file: /cvs/src/sys/dev/pci/ehci_pci.c,v > > retrieving revision 1.31 > > diff -u -p -u -r1.31 ehci_pci.c > > --- dev/pci/ehci_pci.c 2 May 2019 20:28:46 -0000 1.31 > > +++ dev/pci/ehci_pci.c 16 Aug 2020 13:16:17 -0000 > > @@ -146,6 +146,51 @@ ehci_pci_attach(struct device *parent, s > > } > > break; > > > > + case PCI_VENDOR_AMD: > > + /* AMD Hudson-2 USB2 [1022:7808] */ > > + if (PCI_PRODUCT(pa->pa_id) == 0x7808) { > > + printf(" XXX AMD Hudson-2 USB2 XXX"); > > + > > + int offset; > > + if (pci_get_capability(sc->sc_pc, sc->sc_tag, > > PCI_CAP_PWRMGMT, &offset, 0)) { > > + pcireg_t reg = pci_conf_read(sc->sc_pc, > > sc->sc_tag, offset); > > + printf(" reg=0x%08x", reg); > > + > > + printf(" pme_d0=%d", (reg & (PCI_PM_CAP_PME_D0 > > << 16)) ? 1 : 0); > > + printf(" pme_d1=%d", (reg & (PCI_PM_CAP_PME_D1 > > << 16)) ? 1 : 0); > > + printf(" pme_d2=%d", (reg & (PCI_PM_CAP_PME_D2 > > << 16)) ? 1 : 0); > > + printf(" pme_d3hot=%d", (reg & > > (PCI_PM_CAP_PME_D3_HOT << 16)) ? 1 : 0); > > + printf(" pme_d3cold=%d", (reg & > > (PCI_PM_CAP_PME_D3_COLD << 16)) ? 1 : 0); > > + > > + printf( " *"); > > + > > + reg &= ~((PCI_PM_CAP_PME_D3_HOT | > > PCI_PM_CAP_PME_D3_COLD) << 16); > > + printf(" val=0x%08x", reg); > > + > > + printf(" pme_d0=%d", (reg & (PCI_PM_CAP_PME_D0 > > << 16)) ? 1 : 0); > > + printf(" pme_d1=%d", (reg & (PCI_PM_CAP_PME_D1 > > << 16)) ? 1 : 0); > > + printf(" pme_d2=%d", (reg & (PCI_PM_CAP_PME_D2 > > << 16)) ? 1 : 0); > > + printf(" pme_d3hot=%d", (reg & > > (PCI_PM_CAP_PME_D3_HOT << 16)) ? 1 : 0); > > + printf(" pme_d3cold=%d", (reg & > > (PCI_PM_CAP_PME_D3_COLD << 16)) ? 1 : 0); > > + > > + printf( " !"); > > + pci_conf_write(sc->sc_pc, sc->sc_tag, offset, > > reg); > > + delay(10 * 1000); > > + printf( "!"); > > + > > + reg = pci_conf_read(sc->sc_pc, sc->sc_tag, > > offset); > > + printf(" reg=0x%08x", reg); > > + > > + delay(10 * 1000); > > + > > + reg = pci_conf_read(sc->sc_pc, sc->sc_tag, > > offset); > > + printf(" reg=0x%08x", reg); > > + } else { > > + printf(" pci_get_capability PCI_CAP_PWRMGMT > > err"); > > + } > > + } > > + break; > > + > > case PCI_VENDOR_VIATECH: > > if (PCI_PRODUCT(pa->pa_id) == PCI_PRODUCT_VIATECH_VT6202 && > > (PCI_REVISION(pa->pa_class) & 0xf0) == 0x60) { > > Index: dev/pci/pcireg.h > > =================================================================== > > RCS file: /cvs/src/sys/dev/pci/pcireg.h,v > > retrieving revision 1.59 > > diff -u -p -u -r1.59 pcireg.h > > --- dev/pci/pcireg.h 2 Nov 2019 10:14:57 -0000 1.59 > > +++ dev/pci/pcireg.h 16 Aug 2020 13:16:17 -0000 > > @@ -533,6 +533,14 @@ typedef u_int8_t pci_revision_t; > > #define PCI_PMCSR_STATE_D1 0x0001 > > #define PCI_PMCSR_STATE_D2 0x0002 > > #define PCI_PMCSR_STATE_D3 0x0003 > > + > > +/* XXX AMD Hudson-2 USB2 XXX */ > > +#define PCI_PM_CAP_PME_D0 0x0800 /* PME can be asserted from D0 */ > > +#define PCI_PM_CAP_PME_D1 0x1000 /* PME can be asserted from D1 */ > > +#define PCI_PM_CAP_PME_D2 0x2000 /* PME can be asserted from D2 */ > > +#define PCI_PM_CAP_PME_D3_HOT 0x4000 /* PME can be asserted from > > D3hot */ > > +#define PCI_PM_CAP_PME_D3_COLD 0x8000 /* PME can be asserted from > > D3cold */ > > + > > #define PCI_PMCSR_PME_STATUS 0x8000 > > #define PCI_PMCSR_PME_EN 0x0100 > > > > > > Dmesg line with above patch looks as follow: > > > > # dmesg | grep -w XXX > > ehci0 at pci0 dev 19 function 0 "AMD Hudson-2 USB2" rev 0x39 XXX AMD > > Hudson-2 USB2 XXX reg=0xfe02e401 pme_d0=1 pme_d1=1 pme_d2=1 pme_d3hot=1 > > pme_d3cold=1 * val=0x3e02e401 pme_d0=1 pme_d1=1 pme_d2=1 pme_d3hot=0 > > pme_d3cold=0 !! reg=0xfe02e401 reg=0xfe02e401: apic 4 int 18 > > > > so as you can see reg=0xfe02e401 before pci_conf_write() and after the > > function call is also 0xfe02e401 where I wanted it to be 0x3e02e401. > > > > Executing pcidump and lspci also shows unwanted value 0xfe02e401: > > > > # pcidump -vvv -xx 0:19:0 > > 0:19:0: AMD Hudson-2 USB2 > > 0x0000: Vendor ID: 1022, Product ID: 7808 > > 0x0004: Command: 0006, Status: 02b0 > > 0x0008: Class: 0c Serial Bus, Subclass: 03 USB, > > Interface: 20, Revision: 39 > > 0x000c: BIST: 00, Header Type: 00, Latency Timer: 40, > > Cache Line Size: 10 > > 0x0010: BAR mem 32bit addr: 0xf7f26000/0x00000100 > > 0x0014: BAR empty (00000000) > > 0x0018: BAR empty (00000000) > > 0x001c: BAR empty (00000000) > > 0x0020: BAR empty (00000000) > > 0x0024: BAR empty (00000000) > > 0x0028: Cardbus CIS: 00000000 > > 0x002c: Subsystem Vendor ID: 1022 Product ID: 7808 > > 0x0030: Expansion ROM Base Address: 00000000 > > 0x0038: 00000000 > > 0x003c: Interrupt Pin: 01 Line: 05 Min Gnt: 00 Max Lat: 00 > > 0x00c0: Capability 0x01: Power Management > > State: D0 > > 0x00e4: Capability 0x0a: Debug Port > > 0x0000: 78081022 02b00006 0c032039 00004010 > > 0x0010: f7f26000 00000000 00000000 00000000 > > 0x0020: 00000000 00000000 00000000 78081022 > > 0x0030: 00000000 000000c0 00000000 00000105 > > 0x0040: 00000000 00000000 00000000 00000000 > > 0x0050: a42a1141 fb7fda7b 00000000 00000000 > > 0x0060: 00002020 00000000 00000000 00000000 > > 0x0070: 000fffff c000002c 800000ff 00000000 > > 0x0080: 79000438 fcfe0018 00200081 64327ec7 > > 0x0090: 10011d4c 00000000 00000000 000f0f00 > > 0x00a0: 00000001 c0080000 00000000 00000000 > > 0x00b0: 00000000 00000000 00000000 00000000 > > 0x00c0: fe02e401 00400000 00000000 00000000 > > 0x00d0: 00000000 00000000 00000000 00000000 > > 0x00e0: 00000000 20e0000a 00000000 01000020 > > 0x00f0: 00000000 00000000 00000000 00000000 > > > > # lspci -vvv -xxxx -s 00:13.0 > > 00:13.0 USB controller: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI > > Controller (rev 39) (prog-if 20 [EHCI]) > > Subsystem: Advanced Micro Devices, Inc. [AMD] FCH USB EHCI Controller > > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > > Stepping- SERR- FastB2B- DisINTx- > > Status: Cap+ 66MHz+ UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort- > > <TAbort- <MAbort- >SERR- <PERR- INTx- > > Latency: 64, Cache Line Size: 64 bytes > > Interrupt: pin A routed to IRQ 5 > > Region 0: Memory at f7f26000 (32-bit, non-prefetchable) > > Capabilities: [c0] Power Management version 2 > > Flags: PMEClk- DSI- D1+ D2+ AuxCurrent=0mA > > PME(D0+,D1+,D2+,D3hot+,D3cold+) > > Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- > > Bridge: PM- B3+ > > Capabilities: [e4] Debug port: BAR=1 offset=00e0 > > 00: 22 10 08 78 06 00 b0 02 39 20 03 0c 10 40 00 00 > > 10: 00 60 f2 f7 00 00 00 00 00 00 00 00 00 00 00 00 > > 20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 08 78 > > 30: 00 00 00 00 c0 00 00 00 00 00 00 00 05 01 00 00 > > 40: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 50: 41 11 2a a4 7b da 7f fb 00 00 00 00 00 00 00 00 > > 60: 20 20 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > 70: ff ff 0f 00 2c 00 00 c0 ff 00 00 80 00 00 00 00 > > 80: 38 04 00 79 18 00 fe fc 81 00 20 00 c7 7e 32 64 > > 90: 4c 1d 01 10 00 00 00 00 00 00 00 00 00 0f 0f 00 > > a0: 01 00 00 00 00 00 08 c0 00 00 00 00 00 00 00 00 > > b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > c0: 01 e4 02 fe 00 00 40 00 00 00 00 00 00 00 00 00 > > d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > e0: 00 00 00 00 0a 00 e0 20 00 00 00 00 20 00 00 01 > > f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 > > > > What I am doing wrong here? > > > > Any kind soul is willing to help me and push me in the correct > direction? As you can see I don't have experience in kernel space > development and I imagine one would need to have plenty of patience with > me, but maybe there is someone who is willing to try?
I think you're looking at the wrong thing. PME is about the ability to wake up the machine if the device (in this case the USB controller) has been put into a certain power save state. But that's not the case here. In the end this is almost certainly a BIOS bug. What could help is doing a more thorough reset of the USB controller when OpenBSD prepares the system for a reboot. But we already do a full software reset of the controller. So I have no idea what to do.
