Re: armv7/imx: imxpcie(4) driver & device tree problem

[Mark Kettenis]

> Date: Thu, 14 Jul 2016 11:56:47 +0200 (CEST)
> From: Mark Kettenis 
>
> > > Time to investigate the
> > > 
> > >   sdmmc0: can't enable card
> > > 
> > > message I see on my CuBox-i4Pro once more.
> > 
> > I'd look at the no-1-8v property to start with (ommmc should be looking
> > at ti,dual-volt before using 3.0v as well).
> > 
> > Documented as:
> > 
> > - no-1-8-v: when present, denotes that 1.8v card voltage is not supported on
> >   this system, even if the controller claims it is.
> 
> I'll look at it, but I think the 1.8v thing doesn't apply to SDIO and
> was only handled to work around an issue in the Linux sdhc code.

And indeed, after doing all the mmc-pwrseq and vmmc-supply magic, I
now get a response from the SDIO card:

(manufacturer 0x2d0, product 0x4330)at sdmmc0 function 1 not configured
(manufacturer 0x2d0, product 0x4330)at sdmmc0 function 2 not configured

which indicates that we're dealing with a BCM4330 (or at least one of
its family members) here.

Re: armv7/imx: imxpcie(4) driver & device tree problem

[Patrick Wildt]

On Thu, Jul 14, 2016 at 11:34:05AM +0200, Mark Kettenis wrote:
> > Date: Thu, 14 Jul 2016 17:53:22 +1000
> > From: Jonathan Gray 
> > 
> > On Thu, Jul 14, 2016 at 03:20:34AM -0400, Ian Sutton wrote:
> > > I am working on a MD PCI-E driver for the armv7/imx platform. It
> > > attaches via FDT/simplebus, ascertaining its physical memory addresses
> > > from the fdt structure originating from the dtb files present on the
> > > msdos bootloader partition. The dtb files we're using now have the
> > > cubox's 'pcie' nodes disabled via a "status=disabled" property. The
> > > following patches fix this in the decompiled dtb outfiles:
> > 
> > I thought the wlan was attached to pcie but it seems it is
> > actually an sdio device.
> > 
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/imx6qdl-microsom.dtsi
> > 
> > /* USDHC1 - Connected to optional BRCM Wifi/BT/FM */
> >  {
> > pinctrl-names = "default";
> > pinctrl-0 = <_microsom_brcm_wifi _microsom_usdhc1>;
> > bus-width = <4>;
> > mmc-pwrseq = <_pwrseq>;
> > keep-power-in-suspend;
> > no-1-8-v;
> > non-removable;
> > vmmc-supply = <_brcm>;
> > status = "okay";
> > };
> 
> WiFi over SDIO is quite common for these SoC-type systems.  The Bay
> Trail-based ASUS EeeBook I have has a similar setup.  I suppose PCIe
> sucks quite a bit of power even if you do proper link management.  I
> actually believe that Broadcom's PCIe solutions are essentially SDIO
> with a PCIe to SDIO bridge on front of them.
> 
> Time to investigate the
> 
>   sdmmc0: can't enable card
> 
> message I see on my CuBox-i4Pro once more.
> 

I stumbled over quite a lot SDIO-connected Broadcom FullMACs lately.
The raspberry Pi 3 contains a BCM43438, some Intel HW I have has
some BCM4334.  I have a USB dongle with a similar chip and now the
CuBox-i also contains one...  I'm sure I forgot at least one other
device that features that chip.

I had started with some simple device attachment and firmware
uploading for SDIO and USB, using the dual-licensed brcmfmac driver
as reference:

https://github.com/Bluerise/openbsd/compare/aecf6dba489bdcb0237695815df519e17b88a850...bcm

Patrick

Re: armv7/imx: imxpcie(4) driver & device tree problem

[Mark Kettenis]

> Date: Thu, 14 Jul 2016 19:45:32 +1000
> From: Jonathan Gray 
> 
> On Thu, Jul 14, 2016 at 11:34:05AM +0200, Mark Kettenis wrote:
> > > Date: Thu, 14 Jul 2016 17:53:22 +1000
> > > From: Jonathan Gray 
> > > 
> > > On Thu, Jul 14, 2016 at 03:20:34AM -0400, Ian Sutton wrote:
> > > > I am working on a MD PCI-E driver for the armv7/imx platform. It
> > > > attaches via FDT/simplebus, ascertaining its physical memory addresses
> > > > from the fdt structure originating from the dtb files present on the
> > > > msdos bootloader partition. The dtb files we're using now have the
> > > > cubox's 'pcie' nodes disabled via a "status=disabled" property. The
> > > > following patches fix this in the decompiled dtb outfiles:
> > > 
> > > I thought the wlan was attached to pcie but it seems it is
> > > actually an sdio device.
> > > 
> > > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/imx6qdl-microsom.dtsi
> > > 
> > > /* USDHC1 - Connected to optional BRCM Wifi/BT/FM */
> > >  {
> > > pinctrl-names = "default";
> > > pinctrl-0 = <_microsom_brcm_wifi 
> > > _microsom_usdhc1>;
> > > bus-width = <4>;
> > > mmc-pwrseq = <_pwrseq>;
> > > keep-power-in-suspend;
> > > no-1-8-v;
> > > non-removable;
> > > vmmc-supply = <_brcm>;
> > > status = "okay";
> > > };
> > 
> > WiFi over SDIO is quite common for these SoC-type systems.  The Bay
> > Trail-based ASUS EeeBook I have has a similar setup.  I suppose PCIe
> > sucks quite a bit of power even if you do proper link management.  I
> > actually believe that Broadcom's PCIe solutions are essentially SDIO
> > with a PCIe to SDIO bridge on front of them.
> > 
> > Time to investigate the
> > 
> >   sdmmc0: can't enable card
> > 
> > message I see on my CuBox-i4Pro once more.
> 
> I'd look at the no-1-8v property to start with (ommmc should be looking
> at ti,dual-volt before using 3.0v as well).
> 
> Documented as:
> 
> - no-1-8-v: when present, denotes that 1.8v card voltage is not supported on
>   this system, even if the controller claims it is.

I'll look at it, but I think the 1.8v thing doesn't apply to SDIO and
was only handled to work around an issue in the Linux sdhc code.

Re: armv7/imx: imxpcie(4) driver & device tree problem

[Jonathan Gray]

On Thu, Jul 14, 2016 at 11:34:05AM +0200, Mark Kettenis wrote:
> > Date: Thu, 14 Jul 2016 17:53:22 +1000
> > From: Jonathan Gray 
> > 
> > On Thu, Jul 14, 2016 at 03:20:34AM -0400, Ian Sutton wrote:
> > > I am working on a MD PCI-E driver for the armv7/imx platform. It
> > > attaches via FDT/simplebus, ascertaining its physical memory addresses
> > > from the fdt structure originating from the dtb files present on the
> > > msdos bootloader partition. The dtb files we're using now have the
> > > cubox's 'pcie' nodes disabled via a "status=disabled" property. The
> > > following patches fix this in the decompiled dtb outfiles:
> > 
> > I thought the wlan was attached to pcie but it seems it is
> > actually an sdio device.
> > 
> > https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/imx6qdl-microsom.dtsi
> > 
> > /* USDHC1 - Connected to optional BRCM Wifi/BT/FM */
> >  {
> > pinctrl-names = "default";
> > pinctrl-0 = <_microsom_brcm_wifi _microsom_usdhc1>;
> > bus-width = <4>;
> > mmc-pwrseq = <_pwrseq>;
> > keep-power-in-suspend;
> > no-1-8-v;
> > non-removable;
> > vmmc-supply = <_brcm>;
> > status = "okay";
> > };
> 
> WiFi over SDIO is quite common for these SoC-type systems.  The Bay
> Trail-based ASUS EeeBook I have has a similar setup.  I suppose PCIe
> sucks quite a bit of power even if you do proper link management.  I
> actually believe that Broadcom's PCIe solutions are essentially SDIO
> with a PCIe to SDIO bridge on front of them.
> 
> Time to investigate the
> 
>   sdmmc0: can't enable card
> 
> message I see on my CuBox-i4Pro once more.

I'd look at the no-1-8v property to start with (ommmc should be looking
at ti,dual-volt before using 3.0v as well).

Documented as:

- no-1-8-v: when present, denotes that 1.8v card voltage is not supported on
  this system, even if the controller claims it is.

Re: armv7/imx: imxpcie(4) driver & device tree problem

[Mark Kettenis]

> Date: Thu, 14 Jul 2016 17:53:22 +1000
> From: Jonathan Gray 
> 
> On Thu, Jul 14, 2016 at 03:20:34AM -0400, Ian Sutton wrote:
> > I am working on a MD PCI-E driver for the armv7/imx platform. It
> > attaches via FDT/simplebus, ascertaining its physical memory addresses
> > from the fdt structure originating from the dtb files present on the
> > msdos bootloader partition. The dtb files we're using now have the
> > cubox's 'pcie' nodes disabled via a "status=disabled" property. The
> > following patches fix this in the decompiled dtb outfiles:
> 
> I thought the wlan was attached to pcie but it seems it is
> actually an sdio device.
> 
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/imx6qdl-microsom.dtsi
> 
> /* USDHC1 - Connected to optional BRCM Wifi/BT/FM */
>  {
> pinctrl-names = "default";
> pinctrl-0 = <_microsom_brcm_wifi _microsom_usdhc1>;
> bus-width = <4>;
> mmc-pwrseq = <_pwrseq>;
> keep-power-in-suspend;
> no-1-8-v;
> non-removable;
> vmmc-supply = <_brcm>;
> status = "okay";
> };

WiFi over SDIO is quite common for these SoC-type systems.  The Bay
Trail-based ASUS EeeBook I have has a similar setup.  I suppose PCIe
sucks quite a bit of power even if you do proper link management.  I
actually believe that Broadcom's PCIe solutions are essentially SDIO
with a PCIe to SDIO bridge on front of them.

Time to investigate the

  sdmmc0: can't enable card

message I see on my CuBox-i4Pro once more.

Re: armv7/imx: imxpcie(4) driver & device tree problem

[Mark Kettenis]

> From: Ian Sutton 
> Date: Thu, 14 Jul 2016 03:20:34 -0400 (EDT)
> 
> I am working on a MD PCI-E driver for the armv7/imx platform.

Before you continue you should talk to patrick@.  I believe he had
some working code already.

Re: armv7/imx: imxpcie(4) driver & device tree problem

[Jonathan Gray]

On Thu, Jul 14, 2016 at 03:20:34AM -0400, Ian Sutton wrote:
> I am working on a MD PCI-E driver for the armv7/imx platform. It
> attaches via FDT/simplebus, ascertaining its physical memory addresses
> from the fdt structure originating from the dtb files present on the
> msdos bootloader partition. The dtb files we're using now have the
> cubox's 'pcie' nodes disabled via a "status=disabled" property. The
> following patches fix this in the decompiled dtb outfiles:

I thought the wlan was attached to pcie but it seems it is
actually an sdio device.

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm/boot/dts/imx6qdl-microsom.dtsi

/* USDHC1 - Connected to optional BRCM Wifi/BT/FM */
 {
pinctrl-names = "default";
pinctrl-0 = <_microsom_brcm_wifi _microsom_usdhc1>;
bus-width = <4>;
mmc-pwrseq = <_pwrseq>;
keep-power-in-suspend;
no-1-8-v;
non-removable;
vmmc-supply = <_brcm>;
status = "okay";
};

http://wiki.solid-run.com/doku.php?id=products:imx6:microsom:quad
Wifi: 802.1 b/g/n - 2.4 Ghz / Bluetooth: V 4.0 (BCM4330)

So it looks like there isn't anything attached to pcie.

armv7/imx: imxpcie(4) driver & device tree problem

[Ian Sutton]

I am working on a MD PCI-E driver for the armv7/imx platform. It
attaches via FDT/simplebus, ascertaining its physical memory addresses
from the fdt structure originating from the dtb files present on the
msdos bootloader partition. The dtb files we're using now have the
cubox's 'pcie' nodes disabled via a "status=disabled" property. The
following patches fix this in the decompiled dtb outfiles:

--- imx6q-cubox-i.dts   Thu Jul 14 01:26:57 2016
+++ imx6q-cubox-i.post.dts  Thu Jul 14 01:27:35 2016
@@ -212,7 +212,7 @@
phandle = <0x46>;
};
 
-   pcie@0x0100 {
+   pcie@0100 {
compatible = "fsl,imx6q-pcie", "snps,dw-pcie";
reg = <0x1ffc000 0x4000 0x1f0 0x8>;
reg-names = "dbi", "config";
@@ -228,7 +228,6 @@
interrupt-map = <0x0 0x0 0x0 0x1 0x1 0x0 0x7b 0x4 0x0 
0x0 0x0 0x2 0x1 0x0 0x7a 0x4 0x0 0x0 0x0 0x3 0x1 0x0 0x79 0x4 0x0 0x0 0x0 0x4 
0x1 0x0 0x78 0x4>;
clocks = <0x2 0x90 0x2 0xce 0x2 0xbd>;
clock-names = "pcie", "pcie_bus", "pcie_phy";
-   status = "disabled";
};
 
pmu {

--- imx6dl-cubox-i.dts  Thu Jul 14 02:04:27 2016
+++ imx6dl-cubox-i.post.dts Thu Jul 14 02:05:04 2016
@@ -191,7 +191,7 @@
phandle = <0x36>;
};
 
-   pcie@0x0100 {
+   pcie@0100 {
compatible = "fsl,imx6q-pcie", "snps,dw-pcie";
reg = <0x1ffc000 0x4000 0x1f0 0x8>;
reg-names = "dbi", "config";
@@ -207,7 +207,6 @@
interrupt-map = <0x0 0x0 0x0 0x1 0x1 0x0 0x7b 0x4 0x0 
0x0 0x0 0x2 0x1 0x0 0x7a 0x4 0x0 0x0 0x0 0x3 0x1 0x0 0x79 0x4 0x0 0x0 0x0 0x4 
0x1 0x0 0x78 0x4>;
clocks = <0x2 0x90 0x2 0xce 0x2 0xbd>;
clock-names = "pcie", "pcie_bus", "pcie_phy";
-   status = "disabled";
};
 
pmu {

Here are the resultant binaries you can copy to your bootloader
partition, for convenience:

http://ce.gl/imx6q-cubox-i.dtb
http://ce.gl/imx6dl-cubox-i.dtb

Replacing these dtbs indeed causes my _match() function to succeed where
it hadn't before.

Finally, here is the (very much in-progress) driver:

Index: sys/arch/armv7/conf/GENERIC
===
RCS file: /cvs/src/sys/arch/armv7/conf/GENERIC,v
retrieving revision 1.31
diff -u -p -r1.31 GENERIC
--- sys/arch/armv7/conf/GENERIC 12 Jul 2016 19:17:49 -  1.31
+++ sys/arch/armv7/conf/GENERIC 14 Jul 2016 06:53:44 -
@@ -53,6 +53,7 @@ imxesdhc* at fdt? # SDHC controller
 sdmmc* at imxesdhc?# SD/MMC bus
 imxahci*   at fdt? # AHCI/SATA
 imxehci*   at fdt? # EHCI
+imxpcie*   at fdt? # PCI-E
 usb*   at imxehci?
 
 # OMAP3xxx/OMAP4xxx SoC
Index: sys/arch/armv7/imx/files.imx
===
RCS file: /cvs/src/sys/arch/armv7/imx/files.imx,v
retrieving revision 1.15
diff -u -p -r1.15 files.imx
--- sys/arch/armv7/imx/files.imx12 Jul 2016 19:17:49 -  1.15
+++ sys/arch/armv7/imx/files.imx14 Jul 2016 06:53:44 -
@@ -51,3 +51,7 @@ file  arch/armv7/imx/imxesdhc.c   imxesdhc
 device imxahci: scsi, atascsi
 attach imxahci at fdt
 file   arch/armv7/imx/imxahci.cimxahci
+
+device imxpcie
+attach imxpcie at fdt
+file   arch/armv7/imx/imxpcie.cimxpcie
Index: sys/arch/armv7/imx/imxpcie.c
===
RCS file: sys/arch/armv7/imx/imxpcie.c
diff -N sys/arch/armv7/imx/imxpcie.c
--- /dev/null   1 Jan 1970 00:00:00 -
+++ sys/arch/armv7/imx/imxpcie.c14 Jul 2016 06:53:44 -
@@ -0,0 +1,77 @@
+/*
+ * Copyright (c) 2016 Ian Sutton 
+ *
+ * Permission to use, copy, modify, and distribute this software for any
+ * purpose with or without fee is hereby granted, provided that the above
+ * copyright notice and this permission notice appear in all copies.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES
+ * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF
+ * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR
+ * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
+ * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
+ * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF
+ * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
+ */
+
+#include 
+#include 
+#include 
+
+#include 
+#include 
+
+#include 
+#include 
+
+#include "imxpcievar.h"
+
+int  imxpcie_match(struct device *, void *, void *);
+void