Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Bjorn Andersson]

On Wed 25 May 04:13 PDT 2016, Stanimir Varbanov wrote:

> On 03/29/2016 06:37 AM, Bjorn Andersson wrote:
> > From: Bjorn Andersson 
> > 
> > This introduces the peripheral image loader, for loading WCNSS firmware
> > and boot the core on e.g. MSM8974. The firmware is verified and booted
> > with the help of the Peripheral Authentication System (PAS) in
> > TrustZone.
> > 
> > Signed-off-by: Bjorn Andersson 
> > Signed-off-by: Bjorn Andersson 
> > ---
> > 
> > Changes since v1:
> > - Split iris definition into separate driver/dt-node
> > - Move constants from DT to code
> > - Make stop-state and some of interrupts optional to properly work on 8064
> > - Cleaned up and made mdt loader support relocation, which is needed on 
> > 8016.
> > 
> >  drivers/remoteproc/Kconfig   |  12 +
> >  drivers/remoteproc/Makefile  |   2 +
> >  drivers/remoteproc/qcom_mdt_loader.c | 172 +++
> >  drivers/remoteproc/qcom_mdt_loader.h |   7 +
> >  drivers/remoteproc/qcom_wcnss.c  | 579 
> > +++
> >  drivers/remoteproc/qcom_wcnss.h  |  22 ++
> >  drivers/remoteproc/qcom_wcnss_iris.c | 185 +++
> >  7 files changed, 979 insertions(+)
> >  create mode 100644 drivers/remoteproc/qcom_mdt_loader.c
> >  create mode 100644 drivers/remoteproc/qcom_mdt_loader.h
> >  create mode 100644 drivers/remoteproc/qcom_wcnss.c
> >  create mode 100644 drivers/remoteproc/qcom_wcnss.h
> >  create mode 100644 drivers/remoteproc/qcom_wcnss_iris.c
> > 
> 
> 
> 
> > +
> > +static int wcnss_remove(struct platform_device *pdev)
> > +{
> > +   struct qcom_wcnss *wcnss = platform_get_drvdata(pdev);
> > +
> > +   of_platform_depopulate(>dev);
> > +
> > +   qcom_smem_state_put(wcnss->state);
> > +   rproc_put(wcnss->rproc);
> 
> might be need to call rproc_del() too ?
> 

You're correct. Thanks for having a look!

Regards,
Bjorn

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Bjorn Andersson]

On Wed 25 May 04:13 PDT 2016, Stanimir Varbanov wrote:

> On 03/29/2016 06:37 AM, Bjorn Andersson wrote:
> > From: Bjorn Andersson 
> > 
> > This introduces the peripheral image loader, for loading WCNSS firmware
> > and boot the core on e.g. MSM8974. The firmware is verified and booted
> > with the help of the Peripheral Authentication System (PAS) in
> > TrustZone.
> > 
> > Signed-off-by: Bjorn Andersson 
> > Signed-off-by: Bjorn Andersson 
> > ---
> > 
> > Changes since v1:
> > - Split iris definition into separate driver/dt-node
> > - Move constants from DT to code
> > - Make stop-state and some of interrupts optional to properly work on 8064
> > - Cleaned up and made mdt loader support relocation, which is needed on 
> > 8016.
> > 
> >  drivers/remoteproc/Kconfig   |  12 +
> >  drivers/remoteproc/Makefile  |   2 +
> >  drivers/remoteproc/qcom_mdt_loader.c | 172 +++
> >  drivers/remoteproc/qcom_mdt_loader.h |   7 +
> >  drivers/remoteproc/qcom_wcnss.c  | 579 
> > +++
> >  drivers/remoteproc/qcom_wcnss.h  |  22 ++
> >  drivers/remoteproc/qcom_wcnss_iris.c | 185 +++
> >  7 files changed, 979 insertions(+)
> >  create mode 100644 drivers/remoteproc/qcom_mdt_loader.c
> >  create mode 100644 drivers/remoteproc/qcom_mdt_loader.h
> >  create mode 100644 drivers/remoteproc/qcom_wcnss.c
> >  create mode 100644 drivers/remoteproc/qcom_wcnss.h
> >  create mode 100644 drivers/remoteproc/qcom_wcnss_iris.c
> > 
> 
> 
> 
> > +
> > +static int wcnss_remove(struct platform_device *pdev)
> > +{
> > +   struct qcom_wcnss *wcnss = platform_get_drvdata(pdev);
> > +
> > +   of_platform_depopulate(>dev);
> > +
> > +   qcom_smem_state_put(wcnss->state);
> > +   rproc_put(wcnss->rproc);
> 
> might be need to call rproc_del() too ?
> 

You're correct. Thanks for having a look!

Regards,
Bjorn

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Stanimir Varbanov]

On 03/29/2016 06:37 AM, Bjorn Andersson wrote:
> From: Bjorn Andersson 
> 
> This introduces the peripheral image loader, for loading WCNSS firmware
> and boot the core on e.g. MSM8974. The firmware is verified and booted
> with the help of the Peripheral Authentication System (PAS) in
> TrustZone.
> 
> Signed-off-by: Bjorn Andersson 
> Signed-off-by: Bjorn Andersson 
> ---
> 
> Changes since v1:
> - Split iris definition into separate driver/dt-node
> - Move constants from DT to code
> - Make stop-state and some of interrupts optional to properly work on 8064
> - Cleaned up and made mdt loader support relocation, which is needed on 8016.
> 
>  drivers/remoteproc/Kconfig   |  12 +
>  drivers/remoteproc/Makefile  |   2 +
>  drivers/remoteproc/qcom_mdt_loader.c | 172 +++
>  drivers/remoteproc/qcom_mdt_loader.h |   7 +
>  drivers/remoteproc/qcom_wcnss.c  | 579 
> +++
>  drivers/remoteproc/qcom_wcnss.h  |  22 ++
>  drivers/remoteproc/qcom_wcnss_iris.c | 185 +++
>  7 files changed, 979 insertions(+)
>  create mode 100644 drivers/remoteproc/qcom_mdt_loader.c
>  create mode 100644 drivers/remoteproc/qcom_mdt_loader.h
>  create mode 100644 drivers/remoteproc/qcom_wcnss.c
>  create mode 100644 drivers/remoteproc/qcom_wcnss.h
>  create mode 100644 drivers/remoteproc/qcom_wcnss_iris.c
> 



> +
> +static int wcnss_remove(struct platform_device *pdev)
> +{
> + struct qcom_wcnss *wcnss = platform_get_drvdata(pdev);
> +
> + of_platform_depopulate(>dev);
> +
> + qcom_smem_state_put(wcnss->state);
> + rproc_put(wcnss->rproc);

might be need to call rproc_del() too ?

> +
> + return 0;
> +}
> +



-- 
regards,
Stan

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Stanimir Varbanov]

On 03/29/2016 06:37 AM, Bjorn Andersson wrote:
> From: Bjorn Andersson 
> 
> This introduces the peripheral image loader, for loading WCNSS firmware
> and boot the core on e.g. MSM8974. The firmware is verified and booted
> with the help of the Peripheral Authentication System (PAS) in
> TrustZone.
> 
> Signed-off-by: Bjorn Andersson 
> Signed-off-by: Bjorn Andersson 
> ---
> 
> Changes since v1:
> - Split iris definition into separate driver/dt-node
> - Move constants from DT to code
> - Make stop-state and some of interrupts optional to properly work on 8064
> - Cleaned up and made mdt loader support relocation, which is needed on 8016.
> 
>  drivers/remoteproc/Kconfig   |  12 +
>  drivers/remoteproc/Makefile  |   2 +
>  drivers/remoteproc/qcom_mdt_loader.c | 172 +++
>  drivers/remoteproc/qcom_mdt_loader.h |   7 +
>  drivers/remoteproc/qcom_wcnss.c  | 579 
> +++
>  drivers/remoteproc/qcom_wcnss.h  |  22 ++
>  drivers/remoteproc/qcom_wcnss_iris.c | 185 +++
>  7 files changed, 979 insertions(+)
>  create mode 100644 drivers/remoteproc/qcom_mdt_loader.c
>  create mode 100644 drivers/remoteproc/qcom_mdt_loader.h
>  create mode 100644 drivers/remoteproc/qcom_wcnss.c
>  create mode 100644 drivers/remoteproc/qcom_wcnss.h
>  create mode 100644 drivers/remoteproc/qcom_wcnss_iris.c
> 



> +
> +static int wcnss_remove(struct platform_device *pdev)
> +{
> + struct qcom_wcnss *wcnss = platform_get_drvdata(pdev);
> +
> + of_platform_depopulate(>dev);
> +
> + qcom_smem_state_put(wcnss->state);
> + rproc_put(wcnss->rproc);

might be need to call rproc_del() too ?

> +
> + return 0;
> +}
> +



-- 
regards,
Stan

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Bjorn Andersson]

On Fri 15 Apr 13:17 PDT 2016, John Stultz wrote:

> On Mon, Mar 28, 2016 at 8:37 PM, Bjorn Andersson
>  wrote:
> > From: Bjorn Andersson 
> >
> > This introduces the peripheral image loader, for loading WCNSS firmware
> > and boot the core on e.g. MSM8974. The firmware is verified and booted
> > with the help of the Peripheral Authentication System (PAS) in
> > TrustZone.
> >
> > Signed-off-by: Bjorn Andersson 
> > Signed-off-by: Bjorn Andersson 
> > ---
> >
> > Changes since v1:
> > - Split iris definition into separate driver/dt-node
> > - Move constants from DT to code
> > - Make stop-state and some of interrupts optional to properly work on 8064
> > - Cleaned up and made mdt loader support relocation, which is needed on 
> > 8016.
> 
> Hey Bjorn,
> As you know,  I've been successfully using this patchset along
> with a number of other patches in your trees to get wifi working on
> the 2013 Nexus7. So for that, this can get a Tested-by: John Stultz
>  :)
> 

Thanks!

> Though while I don't have much feedback on this specific driver, I am
> a little curious about how the bigger integrated picture should look.
> 
> Currently, after bootup, one must "echo start >
> /sys/kernel/debug/remoteproc/remoteproc0/state" to actually boot the
> remote processor.
> 
> One issue is that if I try to integrate that line into some of the
> bootup scripts, the system will hard hang. No panic, no OOPs, no
> watchdog reboot, just a full device lockup. So it seems like there
> needs to be some checks to ensure that whatever clks or otherhardware
> is needed are up and running.
> 

I do see a similar problem on 8974, if I enable "clock scaling" on the
RPM. My guess here is that while booting the WCNSS firmware it releases
some clock that we have not voted for, which makes the RPM pull the
carpet underneath us.

> Second, after booting when I do "echo start..." manually, on occasion
> I run into the case where while we're waiting for the firmware to
> finish loading and the remote proc to come up, wpa_supplicant kicks in
> and starts the wcnss driver, which tries to load the configuration
> firmware before the remoteproc is all the way up. This fails, and then
> usually a few seconds later there's a bad pointer traversal that
> Oopses the machine (dmesg log below)
> 

There are two cases I've seen this:
1) When the WLAN driver starts communicating with the WLAN part of the
firmware before the WCNSS part has informed us about it being fully
booted (and have accepted the NV).

2) When not being patient enough on the first message sent (after
passing #1).


I redesigned the solution to not probe the wifi (and bt) drivers before
we get the indication from the wcnss firmware, I believe that's what
you're running. So #1 should not be causing this.

I have not seen any additional handling for #2 in the downstream driver,
so perhaps we should just bump the timeout and accept that it's going to
take some time for the firmware to get ready. But further archaeological
efforts are needed to conclude this.

> This is clearly racy, and I wonder if the starting of the remoteproc
> is something that should be done by the wcnss driver which depends on
> it?  Though I'm not sure how this would be integrated.
> 

The race should be handled regardless.

But the question of who actually triggers the booting of the wifi
system is still an open one. All executing parts of the wcnss and wifi
driver follows the life cycle of the remote services, i.e. follows the
remoteproc life cycle - so they can't request the system to boot.

We could design it so that the life cycle follows module_init/exit of
the wcnss driver, but that's clunky and as we don't have the firmware
available until later during boot this requires us to have these as
modules (or figure out how to get async firmware loading that waits for
the partition to be mounted).


In the downstream kernel we can see a few different solutions:
- The wireless subsystem comes to life when you touch /dev/wlan, and I
  don't think you can shut it down.
- The application DSP has followed a few different models, one being
  tied 1:1 with a kernel module for the specific purpose of booting and
  stopping the core.
- The video coprocessor, follows the life cycle of the v4l driver - so
  that should be fine - except for the firmware not being available at
  probe time, so we're back at loading from a future file system.
- The modem is booted when a certain QMI service is being looked for.


In the Android case we could use the HALs and some reference counting to
keep the other CPUs running, but in a normal Linux system we don't have
something like this.

So this is still an open issue, that we have to spend more time on.

> thanks
> -john
> 
> 
> "echo start..." happened here...
> 
> [   46.719340]  remoteproc0: powering up 3204000.wcnss-rproc
> [   46.719486]  

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Bjorn Andersson]

On Fri 15 Apr 13:17 PDT 2016, John Stultz wrote:

> On Mon, Mar 28, 2016 at 8:37 PM, Bjorn Andersson
>  wrote:
> > From: Bjorn Andersson 
> >
> > This introduces the peripheral image loader, for loading WCNSS firmware
> > and boot the core on e.g. MSM8974. The firmware is verified and booted
> > with the help of the Peripheral Authentication System (PAS) in
> > TrustZone.
> >
> > Signed-off-by: Bjorn Andersson 
> > Signed-off-by: Bjorn Andersson 
> > ---
> >
> > Changes since v1:
> > - Split iris definition into separate driver/dt-node
> > - Move constants from DT to code
> > - Make stop-state and some of interrupts optional to properly work on 8064
> > - Cleaned up and made mdt loader support relocation, which is needed on 
> > 8016.
> 
> Hey Bjorn,
> As you know,  I've been successfully using this patchset along
> with a number of other patches in your trees to get wifi working on
> the 2013 Nexus7. So for that, this can get a Tested-by: John Stultz
>  :)
> 

Thanks!

> Though while I don't have much feedback on this specific driver, I am
> a little curious about how the bigger integrated picture should look.
> 
> Currently, after bootup, one must "echo start >
> /sys/kernel/debug/remoteproc/remoteproc0/state" to actually boot the
> remote processor.
> 
> One issue is that if I try to integrate that line into some of the
> bootup scripts, the system will hard hang. No panic, no OOPs, no
> watchdog reboot, just a full device lockup. So it seems like there
> needs to be some checks to ensure that whatever clks or otherhardware
> is needed are up and running.
> 

I do see a similar problem on 8974, if I enable "clock scaling" on the
RPM. My guess here is that while booting the WCNSS firmware it releases
some clock that we have not voted for, which makes the RPM pull the
carpet underneath us.

> Second, after booting when I do "echo start..." manually, on occasion
> I run into the case where while we're waiting for the firmware to
> finish loading and the remote proc to come up, wpa_supplicant kicks in
> and starts the wcnss driver, which tries to load the configuration
> firmware before the remoteproc is all the way up. This fails, and then
> usually a few seconds later there's a bad pointer traversal that
> Oopses the machine (dmesg log below)
> 

There are two cases I've seen this:
1) When the WLAN driver starts communicating with the WLAN part of the
firmware before the WCNSS part has informed us about it being fully
booted (and have accepted the NV).

2) When not being patient enough on the first message sent (after
passing #1).


I redesigned the solution to not probe the wifi (and bt) drivers before
we get the indication from the wcnss firmware, I believe that's what
you're running. So #1 should not be causing this.

I have not seen any additional handling for #2 in the downstream driver,
so perhaps we should just bump the timeout and accept that it's going to
take some time for the firmware to get ready. But further archaeological
efforts are needed to conclude this.

> This is clearly racy, and I wonder if the starting of the remoteproc
> is something that should be done by the wcnss driver which depends on
> it?  Though I'm not sure how this would be integrated.
> 

The race should be handled regardless.

But the question of who actually triggers the booting of the wifi
system is still an open one. All executing parts of the wcnss and wifi
driver follows the life cycle of the remote services, i.e. follows the
remoteproc life cycle - so they can't request the system to boot.

We could design it so that the life cycle follows module_init/exit of
the wcnss driver, but that's clunky and as we don't have the firmware
available until later during boot this requires us to have these as
modules (or figure out how to get async firmware loading that waits for
the partition to be mounted).


In the downstream kernel we can see a few different solutions:
- The wireless subsystem comes to life when you touch /dev/wlan, and I
  don't think you can shut it down.
- The application DSP has followed a few different models, one being
  tied 1:1 with a kernel module for the specific purpose of booting and
  stopping the core.
- The video coprocessor, follows the life cycle of the v4l driver - so
  that should be fine - except for the firmware not being available at
  probe time, so we're back at loading from a future file system.
- The modem is booted when a certain QMI service is being looked for.


In the Android case we could use the HALs and some reference counting to
keep the other CPUs running, but in a normal Linux system we don't have
something like this.

So this is still an open issue, that we have to spend more time on.

> thanks
> -john
> 
> 
> "echo start..." happened here...
> 
> [   46.719340]  remoteproc0: powering up 3204000.wcnss-rproc
> [   46.719486]  remoteproc0: Booting fw image wcnss.mdt, size 6804
> [   47.307160] qcom_wcnss_ctrl riva.wcnss: WCNSS Version 1.4 1.2
> [   47.321853] wcn36xx: 

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[John Stultz]

On Mon, Mar 28, 2016 at 8:37 PM, Bjorn Andersson
 wrote:
> From: Bjorn Andersson 
>
> This introduces the peripheral image loader, for loading WCNSS firmware
> and boot the core on e.g. MSM8974. The firmware is verified and booted
> with the help of the Peripheral Authentication System (PAS) in
> TrustZone.
>
> Signed-off-by: Bjorn Andersson 
> Signed-off-by: Bjorn Andersson 
> ---
>
> Changes since v1:
> - Split iris definition into separate driver/dt-node
> - Move constants from DT to code
> - Make stop-state and some of interrupts optional to properly work on 8064
> - Cleaned up and made mdt loader support relocation, which is needed on 8016.

Hey Bjorn,
As you know,  I've been successfully using this patchset along
with a number of other patches in your trees to get wifi working on
the 2013 Nexus7. So for that, this can get a Tested-by: John Stultz
 :)

Though while I don't have much feedback on this specific driver, I am
a little curious about how the bigger integrated picture should look.

Currently, after bootup, one must "echo start >
/sys/kernel/debug/remoteproc/remoteproc0/state" to actually boot the
remote processor.

One issue is that if I try to integrate that line into some of the
bootup scripts, the system will hard hang. No panic, no OOPs, no
watchdog reboot, just a full device lockup. So it seems like there
needs to be some checks to ensure that whatever clks or otherhardware
is needed are up and running.

Second, after booting when I do "echo start..." manually, on occasion
I run into the case where while we're waiting for the firmware to
finish loading and the remote proc to come up, wpa_supplicant kicks in
and starts the wcnss driver, which tries to load the configuration
firmware before the remoteproc is all the way up. This fails, and then
usually a few seconds later there's a bad pointer traversal that
Oopses the machine (dmesg log below)

This is clearly racy, and I wonder if the starting of the remoteproc
is something that should be done by the wcnss driver which depends on
it?  Though I'm not sure how this would be integrated.

thanks
-john


"echo start..." happened here...

[   46.719340]  remoteproc0: powering up 3204000.wcnss-rproc
[   46.719486]  remoteproc0: Booting fw image wcnss.mdt, size 6804
[   47.307160] qcom_wcnss_ctrl riva.wcnss: WCNSS Version 1.4 1.2
[   47.321853] wcn36xx: mac address: 18:00:2d:88:9c:a9

But, before loading is finished, wpa_supplicant starts up...

[   47.403815] init: Starting service 'wpa_supplicant'...
[   47.749631] wcn36xx smd:riva@6:wcnss:wifi: loading
/system/vendor/firmware/wlan/prima/WCNSS_qcom_wlan_nv.bin failed with
error -13
[   47.749824] wcn36xx smd:riva@6:wcnss:wifi: Direct firmware load for
wlan/prima/WCNSS_qcom_wlan_nv.bin failed with error -2
[   47.749841] wcn36xx smd:riva@6:wcnss:wifi: Falling back to user helper

(Note, this firmware load error above happens normally, and the
userhelper usually has to save the day, this is probably a separate
issue with the wcn36xx patches I'm using, and not an issue  with the
remoteproc code)

[   48.246701] wcn36xx: ERROR Timeout! No SMD response in 500ms
[   48.246752] wcn36xx: ERROR Failed to push NV to chip
[   48.268973] init: Service 'wpa_supplicant' (pid 1170) exited with status 255
[   51.858442]  remoteproc0: remote processor 3204000.wcnss-rproc is now up
[   67.543147] init: Starting service 'wpa_supplicant'...
[   67.877434] wcn36xx: ERROR hal_load_nv response failed err=5
[   67.877443] wcn36xx: ERROR Failed to push NV to chip
[   67.891921] init: Service 'wpa_supplicant' (pid 1175) exited with status 255
[   87.682962] Unable to handle kernel NULL pointer dereference at
virtual address 0038
[   87.683324] pgd = e7d2c000
[   87.691595] [0038] *pgd=
[   87.697486] Internal error: Oops: 5 [#1] PREEMPT SMP ARM
[   87.697658] CPU: 3 PID: 159 Comm: lmkd Not tainted
4.6.0-rc3-00095-ga08f5eb #1252
[   87.703041] Hardware name: Qualcomm (Flattened Device Tree)
[   87.710413] task: e7c69a00 ti: e7d3 task.ti: e7d3
[   87.715798] PC is at kmem_cache_alloc+0x80/0x234
[   87.721347] LR is at kmem_cache_alloc+0x40/0x234
[   87.726037] pc : []lr : []psr: 200f0013
[   87.726037] sp : e7d31f20  ip : 0001  fp : 7f5c5004
[   87.730679] r10: 0001025f  r9 : e7d3  r8 : c0e04538
[   87.741832] r7 : c0425d30  r6 : 024000c0  r5 : c0201b80  r4 : 0038
[   87.747047] r3 :   r2 : 28665000  r1 : 0001025f  r0 : 
[   87.753653] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   87.760159] Control: 10c5787d  Table: a7d2c06a  DAC: 0051
[   87.767358] Process lmkd (pid: 159, stack limit = 0xe7d30210)
[   87.773085] Stack: (0xe7d31f20 to 0xe7d32000)
[   87.778832] 1f20: e5c92488  00020001 bea9ea04 
 c0e04594 e7d3
[   87.783195] 1f40:  c0425d30 00020001 

Re: [PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[John Stultz]

On Mon, Mar 28, 2016 at 8:37 PM, Bjorn Andersson
 wrote:
> From: Bjorn Andersson 
>
> This introduces the peripheral image loader, for loading WCNSS firmware
> and boot the core on e.g. MSM8974. The firmware is verified and booted
> with the help of the Peripheral Authentication System (PAS) in
> TrustZone.
>
> Signed-off-by: Bjorn Andersson 
> Signed-off-by: Bjorn Andersson 
> ---
>
> Changes since v1:
> - Split iris definition into separate driver/dt-node
> - Move constants from DT to code
> - Make stop-state and some of interrupts optional to properly work on 8064
> - Cleaned up and made mdt loader support relocation, which is needed on 8016.

Hey Bjorn,
As you know,  I've been successfully using this patchset along
with a number of other patches in your trees to get wifi working on
the 2013 Nexus7. So for that, this can get a Tested-by: John Stultz
 :)

Though while I don't have much feedback on this specific driver, I am
a little curious about how the bigger integrated picture should look.

Currently, after bootup, one must "echo start >
/sys/kernel/debug/remoteproc/remoteproc0/state" to actually boot the
remote processor.

One issue is that if I try to integrate that line into some of the
bootup scripts, the system will hard hang. No panic, no OOPs, no
watchdog reboot, just a full device lockup. So it seems like there
needs to be some checks to ensure that whatever clks or otherhardware
is needed are up and running.

Second, after booting when I do "echo start..." manually, on occasion
I run into the case where while we're waiting for the firmware to
finish loading and the remote proc to come up, wpa_supplicant kicks in
and starts the wcnss driver, which tries to load the configuration
firmware before the remoteproc is all the way up. This fails, and then
usually a few seconds later there's a bad pointer traversal that
Oopses the machine (dmesg log below)

This is clearly racy, and I wonder if the starting of the remoteproc
is something that should be done by the wcnss driver which depends on
it?  Though I'm not sure how this would be integrated.

thanks
-john


"echo start..." happened here...

[   46.719340]  remoteproc0: powering up 3204000.wcnss-rproc
[   46.719486]  remoteproc0: Booting fw image wcnss.mdt, size 6804
[   47.307160] qcom_wcnss_ctrl riva.wcnss: WCNSS Version 1.4 1.2
[   47.321853] wcn36xx: mac address: 18:00:2d:88:9c:a9

But, before loading is finished, wpa_supplicant starts up...

[   47.403815] init: Starting service 'wpa_supplicant'...
[   47.749631] wcn36xx smd:riva@6:wcnss:wifi: loading
/system/vendor/firmware/wlan/prima/WCNSS_qcom_wlan_nv.bin failed with
error -13
[   47.749824] wcn36xx smd:riva@6:wcnss:wifi: Direct firmware load for
wlan/prima/WCNSS_qcom_wlan_nv.bin failed with error -2
[   47.749841] wcn36xx smd:riva@6:wcnss:wifi: Falling back to user helper

(Note, this firmware load error above happens normally, and the
userhelper usually has to save the day, this is probably a separate
issue with the wcn36xx patches I'm using, and not an issue  with the
remoteproc code)

[   48.246701] wcn36xx: ERROR Timeout! No SMD response in 500ms
[   48.246752] wcn36xx: ERROR Failed to push NV to chip
[   48.268973] init: Service 'wpa_supplicant' (pid 1170) exited with status 255
[   51.858442]  remoteproc0: remote processor 3204000.wcnss-rproc is now up
[   67.543147] init: Starting service 'wpa_supplicant'...
[   67.877434] wcn36xx: ERROR hal_load_nv response failed err=5
[   67.877443] wcn36xx: ERROR Failed to push NV to chip
[   67.891921] init: Service 'wpa_supplicant' (pid 1175) exited with status 255
[   87.682962] Unable to handle kernel NULL pointer dereference at
virtual address 0038
[   87.683324] pgd = e7d2c000
[   87.691595] [0038] *pgd=
[   87.697486] Internal error: Oops: 5 [#1] PREEMPT SMP ARM
[   87.697658] CPU: 3 PID: 159 Comm: lmkd Not tainted
4.6.0-rc3-00095-ga08f5eb #1252
[   87.703041] Hardware name: Qualcomm (Flattened Device Tree)
[   87.710413] task: e7c69a00 ti: e7d3 task.ti: e7d3
[   87.715798] PC is at kmem_cache_alloc+0x80/0x234
[   87.721347] LR is at kmem_cache_alloc+0x40/0x234
[   87.726037] pc : []lr : []psr: 200f0013
[   87.726037] sp : e7d31f20  ip : 0001  fp : 7f5c5004
[   87.730679] r10: 0001025f  r9 : e7d3  r8 : c0e04538
[   87.741832] r7 : c0425d30  r6 : 024000c0  r5 : c0201b80  r4 : 0038
[   87.747047] r3 :   r2 : 28665000  r1 : 0001025f  r0 : 
[   87.753653] Flags: nzCv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
[   87.760159] Control: 10c5787d  Table: a7d2c06a  DAC: 0051
[   87.767358] Process lmkd (pid: 159, stack limit = 0xe7d30210)
[   87.773085] Stack: (0xe7d31f20 to 0xe7d32000)
[   87.778832] 1f20: e5c92488  00020001 bea9ea04 
 c0e04594 e7d3
[   87.783195] 1f40:  c0425d30 00020001 0002 ff9c
0142 c03081c4 e7d3
[   87.791355] 1f60:  c041bb94  e5c92480 e5c92480
00020001 bea9 0002
[   

[PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Bjorn Andersson]

From: Bjorn Andersson 

This introduces the peripheral image loader, for loading WCNSS firmware
and boot the core on e.g. MSM8974. The firmware is verified and booted
with the help of the Peripheral Authentication System (PAS) in
TrustZone.

Signed-off-by: Bjorn Andersson 
Signed-off-by: Bjorn Andersson 
---

Changes since v1:
- Split iris definition into separate driver/dt-node
- Move constants from DT to code
- Make stop-state and some of interrupts optional to properly work on 8064
- Cleaned up and made mdt loader support relocation, which is needed on 8016.

 drivers/remoteproc/Kconfig   |  12 +
 drivers/remoteproc/Makefile  |   2 +
 drivers/remoteproc/qcom_mdt_loader.c | 172 +++
 drivers/remoteproc/qcom_mdt_loader.h |   7 +
 drivers/remoteproc/qcom_wcnss.c  | 579 +++
 drivers/remoteproc/qcom_wcnss.h  |  22 ++
 drivers/remoteproc/qcom_wcnss_iris.c | 185 +++
 7 files changed, 979 insertions(+)
 create mode 100644 drivers/remoteproc/qcom_mdt_loader.c
 create mode 100644 drivers/remoteproc/qcom_mdt_loader.h
 create mode 100644 drivers/remoteproc/qcom_wcnss.c
 create mode 100644 drivers/remoteproc/qcom_wcnss.h
 create mode 100644 drivers/remoteproc/qcom_wcnss_iris.c

diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig
index 72e97d7a5209..7290c46fff9d 100644
--- a/drivers/remoteproc/Kconfig
+++ b/drivers/remoteproc/Kconfig
@@ -86,4 +86,16 @@ config ST_REMOTEPROC
  processor framework.
  This can be either built-in or a loadable module.
 
+config QCOM_MDT_LOADER
+   tristate
+
+config QCOM_WCNSS_PIL
+   tristate "Qualcomm WCNSS Peripheral Image Loader"
+   depends on OF && ARCH_QCOM
+   select REMOTEPROC
+   select QCOM_MDT_LOADER
+   select QCOM_SCM
+   help
+ Peripherial Image Loader for the WCNSS block.
+
 endmenu
diff --git a/drivers/remoteproc/Makefile b/drivers/remoteproc/Makefile
index 279cb2edc880..97e6ddbe17ea 100644
--- a/drivers/remoteproc/Makefile
+++ b/drivers/remoteproc/Makefile
@@ -12,3 +12,5 @@ obj-$(CONFIG_STE_MODEM_RPROC) += ste_modem_rproc.o
 obj-$(CONFIG_WKUP_M3_RPROC)+= wkup_m3_rproc.o
 obj-$(CONFIG_DA8XX_REMOTEPROC) += da8xx_remoteproc.o
 obj-$(CONFIG_ST_REMOTEPROC)+= st_remoteproc.o
+obj-$(CONFIG_QCOM_MDT_LOADER)  += qcom_mdt_loader.o
+obj-$(CONFIG_QCOM_WCNSS_PIL)   += qcom_wcnss.o qcom_wcnss_iris.o
diff --git a/drivers/remoteproc/qcom_mdt_loader.c 
b/drivers/remoteproc/qcom_mdt_loader.c
new file mode 100644
index ..a8195093d640
--- /dev/null
+++ b/drivers/remoteproc/qcom_mdt_loader.c
@@ -0,0 +1,172 @@
+/*
+ * Qualcomm Peripheral Image Loader
+ *
+ * Copyright (C) 2016 Linaro Ltd
+ * Copyright (C) 2015 Sony Mobile Communications Inc
+ * Copyright (c) 2012-2013, The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "remoteproc_internal.h"
+
+#define QCOM_MDT_TYPE_MASK (7 << 24)
+#define QCOM_MDT_TYPE_HASH (2 << 24)
+#define QCOM_MDT_RELOCATABLE   BIT(27)
+
+/**
+ * qcom_mdt_find_rsc_table() - provide dummy resource table for remoteproc
+ * @rproc: remoteproc handle
+ * @fw:firmware header
+ * @tablesz:   outgoing size of the table
+ *
+ * Returns a dummy table.
+ */
+struct resource_table *qcom_mdt_find_rsc_table(struct rproc *rproc,
+  const struct firmware *fw,
+  int *tablesz)
+{
+   static struct resource_table table = { .ver = 1, };
+
+   *tablesz = sizeof(table);
+   return 
+}
+EXPORT_SYMBOL_GPL(qcom_mdt_find_rsc_table);
+
+/**
+ * qcom_mdt_load() - load the firmware which header is defined in fw
+ * @rproc: rproc handle
+ * @pas_id:PAS identifier to load this firmware into
+ * @fw:frimware object for the header
+ * @mem_phys:  physical address of reserved memory region for the firmware
+ * @mem_region:pointer to a mapping of the reserved memory region
+ * @mem_size:  size of the reserved memory region
+ *
+ * Returns 0 on success, negative errno otherwise.
+ */
+int qcom_mdt_load(struct rproc *rproc,
+ unsigned int pas_id,
+ const struct firmware *fw,
+ phys_addr_t mem_phys,
+ void *mem_region,
+ size_t mem_size)
+{
+   

[PATCH v2 4/9] remoteproc: Introduce Qualcomm WCNSS firmware loader

[Bjorn Andersson]

From: Bjorn Andersson 

This introduces the peripheral image loader, for loading WCNSS firmware
and boot the core on e.g. MSM8974. The firmware is verified and booted
with the help of the Peripheral Authentication System (PAS) in
TrustZone.

Signed-off-by: Bjorn Andersson 
Signed-off-by: Bjorn Andersson 
---

Changes since v1:
- Split iris definition into separate driver/dt-node
- Move constants from DT to code
- Make stop-state and some of interrupts optional to properly work on 8064
- Cleaned up and made mdt loader support relocation, which is needed on 8016.

 drivers/remoteproc/Kconfig   |  12 +
 drivers/remoteproc/Makefile  |   2 +
 drivers/remoteproc/qcom_mdt_loader.c | 172 +++
 drivers/remoteproc/qcom_mdt_loader.h |   7 +
 drivers/remoteproc/qcom_wcnss.c  | 579 +++
 drivers/remoteproc/qcom_wcnss.h  |  22 ++
 drivers/remoteproc/qcom_wcnss_iris.c | 185 +++
 7 files changed, 979 insertions(+)
 create mode 100644 drivers/remoteproc/qcom_mdt_loader.c
 create mode 100644 drivers/remoteproc/qcom_mdt_loader.h
 create mode 100644 drivers/remoteproc/qcom_wcnss.c
 create mode 100644 drivers/remoteproc/qcom_wcnss.h
 create mode 100644 drivers/remoteproc/qcom_wcnss_iris.c

diff --git a/drivers/remoteproc/Kconfig b/drivers/remoteproc/Kconfig
index 72e97d7a5209..7290c46fff9d 100644
--- a/drivers/remoteproc/Kconfig
+++ b/drivers/remoteproc/Kconfig
@@ -86,4 +86,16 @@ config ST_REMOTEPROC
  processor framework.
  This can be either built-in or a loadable module.
 
+config QCOM_MDT_LOADER
+   tristate
+
+config QCOM_WCNSS_PIL
+   tristate "Qualcomm WCNSS Peripheral Image Loader"
+   depends on OF && ARCH_QCOM
+   select REMOTEPROC
+   select QCOM_MDT_LOADER
+   select QCOM_SCM
+   help
+ Peripherial Image Loader for the WCNSS block.
+
 endmenu
diff --git a/drivers/remoteproc/Makefile b/drivers/remoteproc/Makefile
index 279cb2edc880..97e6ddbe17ea 100644
--- a/drivers/remoteproc/Makefile
+++ b/drivers/remoteproc/Makefile
@@ -12,3 +12,5 @@ obj-$(CONFIG_STE_MODEM_RPROC) += ste_modem_rproc.o
 obj-$(CONFIG_WKUP_M3_RPROC)+= wkup_m3_rproc.o
 obj-$(CONFIG_DA8XX_REMOTEPROC) += da8xx_remoteproc.o
 obj-$(CONFIG_ST_REMOTEPROC)+= st_remoteproc.o
+obj-$(CONFIG_QCOM_MDT_LOADER)  += qcom_mdt_loader.o
+obj-$(CONFIG_QCOM_WCNSS_PIL)   += qcom_wcnss.o qcom_wcnss_iris.o
diff --git a/drivers/remoteproc/qcom_mdt_loader.c 
b/drivers/remoteproc/qcom_mdt_loader.c
new file mode 100644
index ..a8195093d640
--- /dev/null
+++ b/drivers/remoteproc/qcom_mdt_loader.c
@@ -0,0 +1,172 @@
+/*
+ * Qualcomm Peripheral Image Loader
+ *
+ * Copyright (C) 2016 Linaro Ltd
+ * Copyright (C) 2015 Sony Mobile Communications Inc
+ * Copyright (c) 2012-2013, The Linux Foundation. All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+#include "remoteproc_internal.h"
+
+#define QCOM_MDT_TYPE_MASK (7 << 24)
+#define QCOM_MDT_TYPE_HASH (2 << 24)
+#define QCOM_MDT_RELOCATABLE   BIT(27)
+
+/**
+ * qcom_mdt_find_rsc_table() - provide dummy resource table for remoteproc
+ * @rproc: remoteproc handle
+ * @fw:firmware header
+ * @tablesz:   outgoing size of the table
+ *
+ * Returns a dummy table.
+ */
+struct resource_table *qcom_mdt_find_rsc_table(struct rproc *rproc,
+  const struct firmware *fw,
+  int *tablesz)
+{
+   static struct resource_table table = { .ver = 1, };
+
+   *tablesz = sizeof(table);
+   return 
+}
+EXPORT_SYMBOL_GPL(qcom_mdt_find_rsc_table);
+
+/**
+ * qcom_mdt_load() - load the firmware which header is defined in fw
+ * @rproc: rproc handle
+ * @pas_id:PAS identifier to load this firmware into
+ * @fw:frimware object for the header
+ * @mem_phys:  physical address of reserved memory region for the firmware
+ * @mem_region:pointer to a mapping of the reserved memory region
+ * @mem_size:  size of the reserved memory region
+ *
+ * Returns 0 on success, negative errno otherwise.
+ */
+int qcom_mdt_load(struct rproc *rproc,
+ unsigned int pas_id,
+ const struct firmware *fw,
+ phys_addr_t mem_phys,
+ void *mem_region,
+ size_t mem_size)
+{
+   const struct elf32_phdr *phdrs;
+   const struct elf32_phdr *phdr;
+   const struct