Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Fri, Jul 1, 2022 at 12:13 PM Saravana Kannan wrote: > > On Fri, Jul 1, 2022 at 8:08 AM Sudeep Holla wrote: > > > > Hi, Saravana, > > > > On Fri, Jul 01, 2022 at 01:26:12AM -0700, Saravana Kannan wrote: > > > > [...] > > > > > Can you check if this hack helps? If so, then I can think about > > > whether we can pick it up without breaking everything else. Copy-paste > > > tab mess up warning. > > > > Sorry for jumping in late and not even sure if this is right thread. > > I have not bisected anything yet, but I am seeing issues on my Juno R2 > > with SCMI enabled power domains and Coresight AMBA devices. > > > > OF: amba_device_add() failed (-19) for /etf@2001 > > OF: amba_device_add() failed (-19) for /tpiu@2003 > > OF: amba_device_add() failed (-19) for /funnel@2004 > > OF: amba_device_add() failed (-19) for /etr@2007 > > OF: amba_device_add() failed (-19) for /stm@2010 > > OF: amba_device_add() failed (-19) for /replicator@2012 > > OF: amba_device_add() failed (-19) for /cpu-debug@2201 > > OF: amba_device_add() failed (-19) for /etm@2204 > > OF: amba_device_add() failed (-19) for /cti@2202 > > OF: amba_device_add() failed (-19) for /funnel@220c > > OF: amba_device_add() failed (-19) for /cpu-debug@2211 > > OF: amba_device_add() failed (-19) for /etm@2214 > > OF: amba_device_add() failed (-19) for /cti@2212 > > OF: amba_device_add() failed (-19) for /cpu-debug@2301 > > OF: amba_device_add() failed (-19) for /etm@2304 > > OF: amba_device_add() failed (-19) for /cti@2302 > > OF: amba_device_add() failed (-19) for /funnel@230c > > OF: amba_device_add() failed (-19) for /cpu-debug@2311 > > OF: amba_device_add() failed (-19) for /etm@2314 > > OF: amba_device_add() failed (-19) for /cti@2312 > > OF: amba_device_add() failed (-19) for /cpu-debug@2321 > > OF: amba_device_add() failed (-19) for /etm@2324 > > OF: amba_device_add() failed (-19) for /cti@2322 > > OF: amba_device_add() failed (-19) for /cpu-debug@2331 > > OF: amba_device_add() failed (-19) for /etm@2334 > > OF: amba_device_add() failed (-19) for /cti@2332 > > OF: amba_device_add() failed (-19) for /cti@2002 > > OF: amba_device_add() failed (-19) for /cti@2011 > > OF: amba_device_add() failed (-19) for /funnel@2013 > > OF: amba_device_add() failed (-19) for /etf@2014 > > OF: amba_device_add() failed (-19) for /funnel@2015 > > OF: amba_device_add() failed (-19) for /cti@2016 > > > > These are working fine with deferred probe in the mainline. > > I tried the hack you have suggested here(rather Tony's version), > > Thanks for trying that. > > > also > > tried with fw_devlink=0 and fw_devlink=1 > > 0 and 1 aren't valid input to fw_devlink. But yeah, I don't expect > disabling it to make anything better. > > > && fw_devlink.strict=0 > > No change in the behaviour. > > > > The DTS are in arch/arm64/boot/dts/arm/juno-*-scmi.dts and there > > coresight devices are mostly in juno-cs-r1r2.dtsi > > Thanks > > > Let me know if there is anything obvious or you want me to bisect which > > means I need more time. I can do that next week. > > I'll let you know once I poke at the DTS. We need to figure out why > fw_devlink wasn't blocking these from getting to the error (same as in > Tony's case). But since these are amba devices, I think I have some > guesses. > > This is an old series that had some issues in some cases and I haven't > gotten around to looking at it. You can give that a shot if you can > apply it to a recent tree. > https://lore.kernel.org/lkml/20210304195101.3843496-1-sarava...@google.com/ I rebased it to driver-core-next and tested the patch (for correctness, not with your issue though). I'm fairly sure it should help with your issue. Can you give it a shot please? https://lore.kernel.org/lkml/20220705083934.3974140-1-sarava...@google.com/T/#u -Saravana > > After looking at that old patch again, I think I know what's going on. > For normal devices, the pm domain attach happens AFTER the device is > added and fw_devlink has had a chance to set up device links. And if > the suppliers aren't ready, really_probe() won't get as far as > dev_pm_domain_attach(). But for amba, the clock and pm domain > suppliers are "grabbed" before adding the device. > > So with that old patch + always returning -EPROBE_DEFER in > amba_device_add() if amba_read_periphid() fails should fix your issue. > > -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: (EXT) Re: (EXT) Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Mon, Jul 4, 2022 at 12:07 AM Alexander Stein
wrote:
>
> Am Freitag, 1. Juli 2022, 09:02:22 CEST schrieb Saravana Kannan:
> > On Thu, Jun 30, 2022 at 11:02 PM Alexander Stein
> >
> > wrote:
> > > Hi Saravana,
> > >
> > > Am Freitag, 1. Juli 2022, 02:37:14 CEST schrieb Saravana Kannan:
> > > > On Thu, Jun 23, 2022 at 5:08 AM Alexander Stein
> > > >
> > > > wrote:
> > > > > Hi,
> > > > >
> > > > > Am Dienstag, 21. Juni 2022, 09:28:43 CEST schrieb Tony Lindgren:
> > > > > > Hi,
> > > > > >
> > > > > > * Saravana Kannan [700101 02:00]:
> > > > > > > Now that fw_devlink=on by default and fw_devlink supports
> > > > > > > "power-domains" property, the execution will never get to the
> > > > > > > point
> > > > > > > where driver_deferred_probe_check_state() is called before the
> > > > > > > supplier
> > > > > > > has probed successfully or before deferred probe timeout has
> > > > > > > expired.
> > > > > > >
> > > > > > > So, delete the call and replace it with -ENODEV.
> > > > > >
> > > > > > Looks like this causes omaps to not boot in Linux next. With this
> > > > > > simple-pm-bus fails to probe initially as the power-domain is not
> > > > > > yet available. On platform_probe() genpd_get_from_provider() returns
> > > > > > -ENOENT.
> > > > > >
> > > > > > Seems like other stuff is potentially broken too, any ideas on
> > > > > > how to fix this?
> > > > >
> > > > > I think I'm hit by this as well, although I do not get a lockup.
> > > > > In my case I'm using
> > > > > arch/arm64/boot/dts/freescale/imx8mq-tqma8mq-mba8mx.dts and probing of
> > > > > 3832.blk-ctrl fails as the power-domain is not (yet) registed.
> > > >
> > > > Ok, took a look.
> > > >
> > > > The problem is that there are two drivers for the same device and they
> > > > both initialize this device.
> > > >
> > > > gpc: gpc@303a {
> > > >
> > > > compatible = "fsl,imx8mq-gpc";
> > > >
> > > > }
> > > >
> > > > $ git grep -l "fsl,imx7d-gpc" -- drivers/
> > > > drivers/irqchip/irq-imx-gpcv2.c
> > > > drivers/soc/imx/gpcv2.c
> > > >
> > > > IMHO, this is a bad/broken design.
> > > >
> > > > So what's happening is that fw_devlink will block the probe of
> > > > 3832.blk-ctrl until 303a.gpc is initialized. And it stops
> > > > blocking the probe of 3832.blk-ctrl as soon as the first driver
> > > > initializes the device. In this case, it's the irqchip driver.
> > > >
> > > > I'd recommend combining these drivers into one. Something like the
> > > > patch I'm attaching (sorry for the attachment, copy-paste is mangling
> > > > the tabs). Can you give it a shot please?
> > >
> > > I tried this patch and it delayed the driver initialization (those of UART
> > > as
> > > well BTW). Unfortunately the driver fails the same way:
> > Thanks for testing the patch!
> >
> > > > [1.125253] imx8m-blk-ctrl 3832.blk-ctrl: error -ENODEV: failed
> > > > to
> > >
> > > attach power domain "bus"
> > >
> > > More than that it even introduced some more errors:
> > > > [0.008160] irq: no irq domain found for gpc@303a !
> >
> > So the idea behind my change was that as long as the irqchip isn't the
> > root of the irqdomain (might be using the terms incorrectly) like the
> > gic, you can make it a platform driver. And I was trying to hack up a
> > patch that's the equivalent of platform_irqchip_probe() (which just
> > ends up eventually calling the callback you use in IRQCHIP_DECLARE().
> > I probably made some mistake in the quick hack that I'm sure if
> > fixable.
> >
> > > > [0.013251] Failed to map interrupt for
> > > > /soc@0/bus@3040/timer@306a
> >
> > However, this timer driver also uses TIMER_OF_DECLARE() which can't
> > handle failure to get the IRQ (because it's can't -EPROBE_DEFER). So,
> > this means, the timer driver inturn needs to be converted to a
> > platform driver if it's supposed to work with the IRQCHIP_DECLARE()
> > being converted to a platform driver.
> >
> > But that's a can of worms not worth opening. But then I remembered
> > this simpler workaround will work and it is pretty much a variant of
> > the workaround that's already in the gpc's irqchip driver to allow two
> > drivers to probe the same device (people really should stop doing
> > that).
> >
> > Can you drop my previous hack patch and try this instead please? I'm
> > 99% sure this will work.
> >
> > diff --git a/drivers/irqchip/irq-imx-gpcv2.c
> > b/drivers/irqchip/irq-imx-gpcv2.c index b9c22f764b4d..8a0e82067924 100644
> > --- a/drivers/irqchip/irq-imx-gpcv2.c
> > +++ b/drivers/irqchip/irq-imx-gpcv2.c
> > @@ -283,6 +283,7 @@ static int __init imx_gpcv2_irqchip_init(struct
> > device_node *node,
> > * later the GPC power domain driver will not be skipped.
> > */
> > of_node_clear_flag(node, OF_POPULATED);
> > + fwnode_dev_initialized(domain->fwnode, false);
> > return 0;
> > }
>
> Just to be sure here, I tried this patch on top of next-20220701 but
> unfortunately this doesn't
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Fri, Jul 1, 2022 at 8:08 AM Sudeep Holla wrote: > > Hi, Saravana, > > On Fri, Jul 01, 2022 at 01:26:12AM -0700, Saravana Kannan wrote: > > [...] > > > Can you check if this hack helps? If so, then I can think about > > whether we can pick it up without breaking everything else. Copy-paste > > tab mess up warning. > > Sorry for jumping in late and not even sure if this is right thread. > I have not bisected anything yet, but I am seeing issues on my Juno R2 > with SCMI enabled power domains and Coresight AMBA devices. > > OF: amba_device_add() failed (-19) for /etf@2001 > OF: amba_device_add() failed (-19) for /tpiu@2003 > OF: amba_device_add() failed (-19) for /funnel@2004 > OF: amba_device_add() failed (-19) for /etr@2007 > OF: amba_device_add() failed (-19) for /stm@2010 > OF: amba_device_add() failed (-19) for /replicator@2012 > OF: amba_device_add() failed (-19) for /cpu-debug@2201 > OF: amba_device_add() failed (-19) for /etm@2204 > OF: amba_device_add() failed (-19) for /cti@2202 > OF: amba_device_add() failed (-19) for /funnel@220c > OF: amba_device_add() failed (-19) for /cpu-debug@2211 > OF: amba_device_add() failed (-19) for /etm@2214 > OF: amba_device_add() failed (-19) for /cti@2212 > OF: amba_device_add() failed (-19) for /cpu-debug@2301 > OF: amba_device_add() failed (-19) for /etm@2304 > OF: amba_device_add() failed (-19) for /cti@2302 > OF: amba_device_add() failed (-19) for /funnel@230c > OF: amba_device_add() failed (-19) for /cpu-debug@2311 > OF: amba_device_add() failed (-19) for /etm@2314 > OF: amba_device_add() failed (-19) for /cti@2312 > OF: amba_device_add() failed (-19) for /cpu-debug@2321 > OF: amba_device_add() failed (-19) for /etm@2324 > OF: amba_device_add() failed (-19) for /cti@2322 > OF: amba_device_add() failed (-19) for /cpu-debug@2331 > OF: amba_device_add() failed (-19) for /etm@2334 > OF: amba_device_add() failed (-19) for /cti@2332 > OF: amba_device_add() failed (-19) for /cti@2002 > OF: amba_device_add() failed (-19) for /cti@2011 > OF: amba_device_add() failed (-19) for /funnel@2013 > OF: amba_device_add() failed (-19) for /etf@2014 > OF: amba_device_add() failed (-19) for /funnel@2015 > OF: amba_device_add() failed (-19) for /cti@2016 > > These are working fine with deferred probe in the mainline. > I tried the hack you have suggested here(rather Tony's version), Thanks for trying that. > also > tried with fw_devlink=0 and fw_devlink=1 0 and 1 aren't valid input to fw_devlink. But yeah, I don't expect disabling it to make anything better. > && fw_devlink.strict=0 > No change in the behaviour. > > The DTS are in arch/arm64/boot/dts/arm/juno-*-scmi.dts and there > coresight devices are mostly in juno-cs-r1r2.dtsi Thanks > Let me know if there is anything obvious or you want me to bisect which > means I need more time. I can do that next week. I'll let you know once I poke at the DTS. We need to figure out why fw_devlink wasn't blocking these from getting to the error (same as in Tony's case). But since these are amba devices, I think I have some guesses. This is an old series that had some issues in some cases and I haven't gotten around to looking at it. You can give that a shot if you can apply it to a recent tree. https://lore.kernel.org/lkml/20210304195101.3843496-1-sarava...@google.com/ After looking at that old patch again, I think I know what's going on. For normal devices, the pm domain attach happens AFTER the device is added and fw_devlink has had a chance to set up device links. And if the suppliers aren't ready, really_probe() won't get as far as dev_pm_domain_attach(). But for amba, the clock and pm domain suppliers are "grabbed" before adding the device. So with that old patch + always returning -EPROBE_DEFER in amba_device_add() if amba_read_periphid() fails should fix your issue. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Fri, Jul 1, 2022 at 1:10 AM Saravana Kannan wrote:
>
> On Thu, Jun 30, 2022 at 11:12 PM Tony Lindgren wrote:
> >
> > * Tony Lindgren [220701 08:33]:
> > > * Saravana Kannan [220630 23:25]:
> > > > On Thu, Jun 30, 2022 at 4:26 PM Rob Herring wrote:
> > > > >
> > > > > On Thu, Jun 30, 2022 at 5:11 PM Saravana Kannan
> > > > > wrote:
> > > > > >
> > > > > > On Mon, Jun 27, 2022 at 2:10 AM Tony Lindgren
> > > > > > wrote:
> > > > > > >
> > > > > > > * Saravana Kannan [220623 08:17]:
> > > > > > > > On Thu, Jun 23, 2022 at 12:01 AM Tony Lindgren
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > * Saravana Kannan [220622 19:05]:
> > > > > > > > > > On Tue, Jun 21, 2022 at 9:59 PM Tony Lindgren
> > > > > > > > > > wrote:
> > > > > > > > > > > This issue is no directly related fw_devlink. It is a
> > > > > > > > > > > side effect of
> > > > > > > > > > > removing driver_deferred_probe_check_state(). We no
> > > > > > > > > > > longer return
> > > > > > > > > > > -EPROBE_DEFER at the end of
> > > > > > > > > > > driver_deferred_probe_check_state().
> > > > > > > > > >
> > > > > > > > > > Yes, I understand the issue. But
> > > > > > > > > > driver_deferred_probe_check_state()
> > > > > > > > > > was deleted because fw_devlink=on should have short
> > > > > > > > > > circuited the
> > > > > > > > > > probe attempt with an -EPROBE_DEFER before reaching the
> > > > > > > > > > bus/driver
> > > > > > > > > > probe function and hitting this -ENOENT failure. That's why
> > > > > > > > > > I was
> > > > > > > > > > asking the other questions.
> > > > > > > > >
> > > > > > > > > OK. So where is the -EPROBE_DEFER supposed to happen without
> > > > > > > > > driver_deferred_probe_check_state() then?
> > > > > > > >
> > > > > > > > device_links_check_suppliers() call inside really_probe() would
> > > > > > > > short
> > > > > > > > circuit and return an -EPROBE_DEFER if the device links are
> > > > > > > > created as
> > > > > > > > expected.
> > > > > > >
> > > > > > > OK
> > > > > > >
> > > > > > > > > Hmm so I'm not seeing any supplier for the top level ocp
> > > > > > > > > device in
> > > > > > > > > the booting case without your patches. I see the suppliers
> > > > > > > > > for the
> > > > > > > > > ocp child device instances only.
> > > > > > > >
> > > > > > > > Hmmm... this is strange (that the device link isn't there), but
> > > > > > > > this
> > > > > > > > is what I suspected.
> > > > > > >
> > > > > > > Yup, maybe it's because of the supplier being a device in the
> > > > > > > child
> > > > > > > interconnect for the ocp.
> > > > > >
> > > > > > Ugh... yeah, this is why the normal (not SYNC_STATE_ONLY) device
> > > > > > link
> > > > > > isn't being created.
> > > > > >
> > > > > > So the aggregated view is something like (I had to set tabs = 4
> > > > > > space
> > > > > > to fit it within 80 cols):
> > > > > >
> > > > > > ocp: ocp { <= Consumer
> > > > > > compatible = "simple-pm-bus";
> > > > > > power-domains = <_per>; <=== Supplier ref
> > > > > >
> > > > > > l4_wkup: interconnect@44c0 {
> > > > > > compatible = "ti,am33xx-l4-wkup", "simple-pm-bus";
> > > > > >
> > > > > > segment@20 { /* 0x44e0 */
> > > > > > compatible = "simple-pm-bus";
> > > > > >
> > > > > > target-module@0 { /* 0x44e0, ap 8 58.0 */
> > > > > > compatible = "ti,sysc-omap4", "ti,sysc";
> > > > > >
> > > > > > prcm: prcm@0 {
> > > > > > compatible = "ti,am3-prcm", "simple-bus";
> > > > > >
> > > > > > prm_per: prm@c00 { <= Actual
> > > > > > Supplier
> > > > > > compatible = "ti,am3-prm-inst",
> > > > > > "ti,omap-prm-inst";
> > > > > > };
> > > > > > };
> > > > > > };
> > > > > > };
> > > > > > };
> > > > > > };
> > > > > >
> > > > > > The power-domain supplier is the great-great-great-grand-child of
> > > > > > the
> > > > > > consumer. It's not clear to me how this is valid. What does it even
> > > > > > mean?
> > > > > >
> > > > > > Rob, is this considered a valid DT?
> > > > >
> > > > > Valid DT for broken h/w.
> > > >
> > > > I'm not sure even in that case it's valid. When the parent device is
> > > > in reset (when the SoC is coming out of reset), there's no way the
> > > > descendant is functional. And if the descendant is not functional, how
> > > > is the parent device powered up? This just feels like an incorrect
> > > > representation of the real h/w.
> > >
> > > It should be correct representation based on scanning the interconnects
> > > and looking at the documentation. Some interconnect parts are wired
> > > always-on and some interconnect instances may be dual-mapped.
>
> Thanks for helping to debug this. Appreciate it.
>
>
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Thu, Jun 30, 2022 at 11:12 PM Tony Lindgren wrote:
>
> * Tony Lindgren [220701 08:33]:
> > * Saravana Kannan [220630 23:25]:
> > > On Thu, Jun 30, 2022 at 4:26 PM Rob Herring wrote:
> > > >
> > > > On Thu, Jun 30, 2022 at 5:11 PM Saravana Kannan
> > > > wrote:
> > > > >
> > > > > On Mon, Jun 27, 2022 at 2:10 AM Tony Lindgren
> > > > > wrote:
> > > > > >
> > > > > > * Saravana Kannan [220623 08:17]:
> > > > > > > On Thu, Jun 23, 2022 at 12:01 AM Tony Lindgren
> > > > > > > wrote:
> > > > > > > >
> > > > > > > > * Saravana Kannan [220622 19:05]:
> > > > > > > > > On Tue, Jun 21, 2022 at 9:59 PM Tony Lindgren
> > > > > > > > > wrote:
> > > > > > > > > > This issue is no directly related fw_devlink. It is a side
> > > > > > > > > > effect of
> > > > > > > > > > removing driver_deferred_probe_check_state(). We no longer
> > > > > > > > > > return
> > > > > > > > > > -EPROBE_DEFER at the end of
> > > > > > > > > > driver_deferred_probe_check_state().
> > > > > > > > >
> > > > > > > > > Yes, I understand the issue. But
> > > > > > > > > driver_deferred_probe_check_state()
> > > > > > > > > was deleted because fw_devlink=on should have short circuited
> > > > > > > > > the
> > > > > > > > > probe attempt with an -EPROBE_DEFER before reaching the
> > > > > > > > > bus/driver
> > > > > > > > > probe function and hitting this -ENOENT failure. That's why I
> > > > > > > > > was
> > > > > > > > > asking the other questions.
> > > > > > > >
> > > > > > > > OK. So where is the -EPROBE_DEFER supposed to happen without
> > > > > > > > driver_deferred_probe_check_state() then?
> > > > > > >
> > > > > > > device_links_check_suppliers() call inside really_probe() would
> > > > > > > short
> > > > > > > circuit and return an -EPROBE_DEFER if the device links are
> > > > > > > created as
> > > > > > > expected.
> > > > > >
> > > > > > OK
> > > > > >
> > > > > > > > Hmm so I'm not seeing any supplier for the top level ocp device
> > > > > > > > in
> > > > > > > > the booting case without your patches. I see the suppliers for
> > > > > > > > the
> > > > > > > > ocp child device instances only.
> > > > > > >
> > > > > > > Hmmm... this is strange (that the device link isn't there), but
> > > > > > > this
> > > > > > > is what I suspected.
> > > > > >
> > > > > > Yup, maybe it's because of the supplier being a device in the child
> > > > > > interconnect for the ocp.
> > > > >
> > > > > Ugh... yeah, this is why the normal (not SYNC_STATE_ONLY) device link
> > > > > isn't being created.
> > > > >
> > > > > So the aggregated view is something like (I had to set tabs = 4 space
> > > > > to fit it within 80 cols):
> > > > >
> > > > > ocp: ocp { <= Consumer
> > > > > compatible = "simple-pm-bus";
> > > > > power-domains = <_per>; <=== Supplier ref
> > > > >
> > > > > l4_wkup: interconnect@44c0 {
> > > > > compatible = "ti,am33xx-l4-wkup", "simple-pm-bus";
> > > > >
> > > > > segment@20 { /* 0x44e0 */
> > > > > compatible = "simple-pm-bus";
> > > > >
> > > > > target-module@0 { /* 0x44e0, ap 8 58.0 */
> > > > > compatible = "ti,sysc-omap4", "ti,sysc";
> > > > >
> > > > > prcm: prcm@0 {
> > > > > compatible = "ti,am3-prcm", "simple-bus";
> > > > >
> > > > > prm_per: prm@c00 { <= Actual Supplier
> > > > > compatible = "ti,am3-prm-inst",
> > > > > "ti,omap-prm-inst";
> > > > > };
> > > > > };
> > > > > };
> > > > > };
> > > > > };
> > > > > };
> > > > >
> > > > > The power-domain supplier is the great-great-great-grand-child of the
> > > > > consumer. It's not clear to me how this is valid. What does it even
> > > > > mean?
> > > > >
> > > > > Rob, is this considered a valid DT?
> > > >
> > > > Valid DT for broken h/w.
> > >
> > > I'm not sure even in that case it's valid. When the parent device is
> > > in reset (when the SoC is coming out of reset), there's no way the
> > > descendant is functional. And if the descendant is not functional, how
> > > is the parent device powered up? This just feels like an incorrect
> > > representation of the real h/w.
> >
> > It should be correct representation based on scanning the interconnects
> > and looking at the documentation. Some interconnect parts are wired
> > always-on and some interconnect instances may be dual-mapped.
Thanks for helping to debug this. Appreciate it.
> >
> > We have a quirk to probe prm/prcm first with pdata_quirks_init_clocks().
:'(
I checked out the code. These prm devices just get populated with NULL
as the parent. So they are effectively top level devices from the
perspective of driver core.
> > Maybe that also now fails in addition to the top level interconnect
> > probing no longer
Re: (EXT) Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Thu, Jun 30, 2022 at 11:02 PM Alexander Stein
wrote:
>
> Hi Saravana,
>
> Am Freitag, 1. Juli 2022, 02:37:14 CEST schrieb Saravana Kannan:
> > On Thu, Jun 23, 2022 at 5:08 AM Alexander Stein
> >
> > wrote:
> > > Hi,
> > >
> > > Am Dienstag, 21. Juni 2022, 09:28:43 CEST schrieb Tony Lindgren:
> > > > Hi,
> > > >
> > > > * Saravana Kannan [700101 02:00]:
> > > > > Now that fw_devlink=on by default and fw_devlink supports
> > > > > "power-domains" property, the execution will never get to the point
> > > > > where driver_deferred_probe_check_state() is called before the
> > > > > supplier
> > > > > has probed successfully or before deferred probe timeout has expired.
> > > > >
> > > > > So, delete the call and replace it with -ENODEV.
> > > >
> > > > Looks like this causes omaps to not boot in Linux next. With this
> > > > simple-pm-bus fails to probe initially as the power-domain is not
> > > > yet available. On platform_probe() genpd_get_from_provider() returns
> > > > -ENOENT.
> > > >
> > > > Seems like other stuff is potentially broken too, any ideas on
> > > > how to fix this?
> > >
> > > I think I'm hit by this as well, although I do not get a lockup.
> > > In my case I'm using
> > > arch/arm64/boot/dts/freescale/imx8mq-tqma8mq-mba8mx.dts and probing of
> > > 3832.blk-ctrl fails as the power-domain is not (yet) registed.
> >
> > Ok, took a look.
> >
> > The problem is that there are two drivers for the same device and they
> > both initialize this device.
> >
> > gpc: gpc@303a {
> > compatible = "fsl,imx8mq-gpc";
> > }
> >
> > $ git grep -l "fsl,imx7d-gpc" -- drivers/
> > drivers/irqchip/irq-imx-gpcv2.c
> > drivers/soc/imx/gpcv2.c
> >
> > IMHO, this is a bad/broken design.
> >
> > So what's happening is that fw_devlink will block the probe of
> > 3832.blk-ctrl until 303a.gpc is initialized. And it stops
> > blocking the probe of 3832.blk-ctrl as soon as the first driver
> > initializes the device. In this case, it's the irqchip driver.
> >
> > I'd recommend combining these drivers into one. Something like the
> > patch I'm attaching (sorry for the attachment, copy-paste is mangling
> > the tabs). Can you give it a shot please?
>
> I tried this patch and it delayed the driver initialization (those of UART as
> well BTW). Unfortunately the driver fails the same way:
Thanks for testing the patch!
> > [1.125253] imx8m-blk-ctrl 3832.blk-ctrl: error -ENODEV: failed to
> attach power domain "bus"
>
> More than that it even introduced some more errors:
> > [0.008160] irq: no irq domain found for gpc@303a !
So the idea behind my change was that as long as the irqchip isn't the
root of the irqdomain (might be using the terms incorrectly) like the
gic, you can make it a platform driver. And I was trying to hack up a
patch that's the equivalent of platform_irqchip_probe() (which just
ends up eventually calling the callback you use in IRQCHIP_DECLARE().
I probably made some mistake in the quick hack that I'm sure if
fixable.
> > [0.013251] Failed to map interrupt for
> > /soc@0/bus@3040/timer@306a
However, this timer driver also uses TIMER_OF_DECLARE() which can't
handle failure to get the IRQ (because it's can't -EPROBE_DEFER). So,
this means, the timer driver inturn needs to be converted to a
platform driver if it's supposed to work with the IRQCHIP_DECLARE()
being converted to a platform driver.
But that's a can of worms not worth opening. But then I remembered
this simpler workaround will work and it is pretty much a variant of
the workaround that's already in the gpc's irqchip driver to allow two
drivers to probe the same device (people really should stop doing
that).
Can you drop my previous hack patch and try this instead please? I'm
99% sure this will work.
diff --git a/drivers/irqchip/irq-imx-gpcv2.c b/drivers/irqchip/irq-imx-gpcv2.c
index b9c22f764b4d..8a0e82067924 100644
--- a/drivers/irqchip/irq-imx-gpcv2.c
+++ b/drivers/irqchip/irq-imx-gpcv2.c
@@ -283,6 +283,7 @@ static int __init imx_gpcv2_irqchip_init(struct
device_node *node,
* later the GPC power domain driver will not be skipped.
*/
of_node_clear_flag(node, OF_POPULATED);
+ fwnode_dev_initialized(domain->fwnode, false);
return 0;
}
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 2/2] serial: Set probe_no_timeout for all DT based drivers
With commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by
default") the probing of TTY consoles could get delayed if they have
optional suppliers that are listed in DT, but those suppliers don't
probe by the time kernel boot finishes. The console devices will probe
eventually after driver_probe_timeout expires.
However, since consoles are often used for debugging kernel issues, it
does not make sense to delay their probe. So, set the newly added
probe_no_timeout flag for all serial drivers that at DT based. This way,
fw_devlink will know not to delay the probing of the consoles past
kernel boot.
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Reported-by: Fabio Estevam
Reported-by: Ahmad Fatoum
Tested-by: Fabio Estevam
Signed-off-by: Saravana Kannan
---
drivers/tty/ehv_bytechan.c | 1 +
drivers/tty/goldfish.c | 1 +
drivers/tty/hvc/hvc_opal.c | 1 +
drivers/tty/serial/8250/8250_aspeed_vuart.c | 1 +
drivers/tty/serial/8250/8250_bcm2835aux.c | 1 +
drivers/tty/serial/8250/8250_bcm7271.c | 1 +
drivers/tty/serial/8250/8250_dw.c | 1 +
drivers/tty/serial/8250/8250_em.c | 1 +
drivers/tty/serial/8250/8250_ingenic.c | 1 +
drivers/tty/serial/8250/8250_lpc18xx.c | 1 +
drivers/tty/serial/8250/8250_mtk.c | 1 +
drivers/tty/serial/8250/8250_of.c | 1 +
drivers/tty/serial/8250/8250_omap.c | 1 +
drivers/tty/serial/8250/8250_pxa.c | 1 +
drivers/tty/serial/8250/8250_tegra.c| 1 +
drivers/tty/serial/8250/8250_uniphier.c | 1 +
drivers/tty/serial/altera_jtaguart.c| 1 +
drivers/tty/serial/altera_uart.c| 1 +
drivers/tty/serial/amba-pl011.c | 1 +
drivers/tty/serial/apbuart.c| 1 +
drivers/tty/serial/ar933x_uart.c| 1 +
drivers/tty/serial/arc_uart.c | 1 +
drivers/tty/serial/atmel_serial.c | 1 +
drivers/tty/serial/bcm63xx_uart.c | 1 +
drivers/tty/serial/clps711x.c | 1 +
drivers/tty/serial/cpm_uart/cpm_uart_core.c | 1 +
drivers/tty/serial/digicolor-usart.c| 1 +
drivers/tty/serial/fsl_linflexuart.c| 1 +
drivers/tty/serial/fsl_lpuart.c | 1 +
drivers/tty/serial/imx.c| 1 +
drivers/tty/serial/lantiq.c | 1 +
drivers/tty/serial/liteuart.c | 1 +
drivers/tty/serial/lpc32xx_hs.c | 1 +
drivers/tty/serial/max310x.c| 1 +
drivers/tty/serial/meson_uart.c | 1 +
drivers/tty/serial/milbeaut_usio.c | 1 +
drivers/tty/serial/mpc52xx_uart.c | 1 +
drivers/tty/serial/mps2-uart.c | 1 +
drivers/tty/serial/msm_serial.c | 1 +
drivers/tty/serial/mvebu-uart.c | 1 +
drivers/tty/serial/mxs-auart.c | 1 +
drivers/tty/serial/omap-serial.c| 1 +
drivers/tty/serial/owl-uart.c | 1 +
drivers/tty/serial/pic32_uart.c | 1 +
drivers/tty/serial/pmac_zilog.c | 1 +
drivers/tty/serial/pxa.c| 1 +
drivers/tty/serial/qcom_geni_serial.c | 1 +
drivers/tty/serial/rda-uart.c | 1 +
drivers/tty/serial/samsung_tty.c| 1 +
drivers/tty/serial/sc16is7xx.c | 1 +
drivers/tty/serial/serial-tegra.c | 1 +
drivers/tty/serial/sh-sci.c | 1 +
drivers/tty/serial/sifive.c | 1 +
drivers/tty/serial/sprd_serial.c| 1 +
drivers/tty/serial/st-asc.c | 1 +
drivers/tty/serial/stm32-usart.c| 1 +
drivers/tty/serial/sunhv.c | 1 +
drivers/tty/serial/sunplus-uart.c | 1 +
drivers/tty/serial/sunsab.c | 1 +
drivers/tty/serial/sunsu.c | 1 +
drivers/tty/serial/sunzilog.c | 1 +
drivers/tty/serial/tegra-tcu.c | 1 +
drivers/tty/serial/uartlite.c | 1 +
drivers/tty/serial/ucc_uart.c | 1 +
drivers/tty/serial/vt8500_serial.c | 1 +
drivers/tty/serial/xilinx_uartps.c | 1 +
66 files changed, 66 insertions(+)
diff --git a/drivers/tty/ehv_bytechan.c b/drivers/tty/ehv_bytechan.c
index 19d32cb6af84..6de710da99be 100644
--- a/drivers/tty/ehv_bytechan.c
+++ b/drivers/tty/ehv_bytechan.c
@@ -739,6 +739,7 @@ static struct platform_driver ehv_bc_tty_driver = {
.driver = {
.name = "ehv-bc",
.of_match_table = ehv_bc_tty_of_ids,
+ .probe_no_timeout = true,
.suppress_bind_attrs = true,
},
.probe = ehv_bc_tty_probe,
diff --git a/drivers/tty/goldfish.c b/drivers/tty/goldfish.c
index c7968aecd870..f9760598c836 100644
--- a/drivers/tty/goldfish.c
+++ b/drivers/tty/goldfish.c
@@ -474,6 +474,7 @@ static struct platform_driver goldfish_tty_platform_driver
= {
[PATCH v2 0/2] Fix console probe delay when stdout-path isn't set
These patches are on top of driver-core-next. Even if stdout-path isn't set in DT, this patch should take console probe times back to how they were before the deferred_probe_timeout clean up series[1]. v1->v2: - Fixed the accidental change that Tobias pointed out. - Added Tested-by tag [1] - https://lore.kernel.org/lkml/20220601070707.3946847-1-sarava...@google.com/ -Saravana cc: Rob Herring cc: sascha hauer cc: peng fan cc: kevin hilman cc: ulf hansson cc: len brown cc: pavel machek cc: joerg roedel cc: will deacon cc: andrew lunn cc: heiner kallweit cc: russell king cc: "david s. miller" cc: eric dumazet cc: jakub kicinski cc: paolo abeni cc: linus walleij cc: hideaki yoshifuji cc: david ahern cc: kernel-t...@android.com cc: linux-ker...@vger.kernel.org cc: linux...@vger.kernel.org cc: iommu@lists.linux-foundation.org cc: net...@vger.kernel.org cc: linux-g...@vger.kernel.org Cc: ker...@pengutronix.de Saravana Kannan (2): driver core: Add probe_no_timeout flag for drivers serial: Set probe_no_timeout for all DT based drivers drivers/base/base.h | 1 + drivers/base/core.c | 7 +++ drivers/base/dd.c | 3 +++ drivers/tty/ehv_bytechan.c | 1 + drivers/tty/goldfish.c | 1 + drivers/tty/hvc/hvc_opal.c | 1 + drivers/tty/serial/8250/8250_aspeed_vuart.c | 1 + drivers/tty/serial/8250/8250_bcm2835aux.c | 1 + drivers/tty/serial/8250/8250_bcm7271.c | 1 + drivers/tty/serial/8250/8250_dw.c | 1 + drivers/tty/serial/8250/8250_em.c | 1 + drivers/tty/serial/8250/8250_ingenic.c | 1 + drivers/tty/serial/8250/8250_lpc18xx.c | 1 + drivers/tty/serial/8250/8250_mtk.c | 1 + drivers/tty/serial/8250/8250_of.c | 1 + drivers/tty/serial/8250/8250_omap.c | 1 + drivers/tty/serial/8250/8250_pxa.c | 1 + drivers/tty/serial/8250/8250_tegra.c| 1 + drivers/tty/serial/8250/8250_uniphier.c | 1 + drivers/tty/serial/altera_jtaguart.c| 1 + drivers/tty/serial/altera_uart.c| 1 + drivers/tty/serial/amba-pl011.c | 1 + drivers/tty/serial/apbuart.c| 1 + drivers/tty/serial/ar933x_uart.c| 1 + drivers/tty/serial/arc_uart.c | 1 + drivers/tty/serial/atmel_serial.c | 1 + drivers/tty/serial/bcm63xx_uart.c | 1 + drivers/tty/serial/clps711x.c | 1 + drivers/tty/serial/cpm_uart/cpm_uart_core.c | 1 + drivers/tty/serial/digicolor-usart.c| 1 + drivers/tty/serial/fsl_linflexuart.c| 1 + drivers/tty/serial/fsl_lpuart.c | 1 + drivers/tty/serial/imx.c| 1 + drivers/tty/serial/lantiq.c | 1 + drivers/tty/serial/liteuart.c | 1 + drivers/tty/serial/lpc32xx_hs.c | 1 + drivers/tty/serial/max310x.c| 1 + drivers/tty/serial/meson_uart.c | 1 + drivers/tty/serial/milbeaut_usio.c | 1 + drivers/tty/serial/mpc52xx_uart.c | 1 + drivers/tty/serial/mps2-uart.c | 1 + drivers/tty/serial/msm_serial.c | 1 + drivers/tty/serial/mvebu-uart.c | 1 + drivers/tty/serial/mxs-auart.c | 1 + drivers/tty/serial/omap-serial.c| 1 + drivers/tty/serial/owl-uart.c | 1 + drivers/tty/serial/pic32_uart.c | 1 + drivers/tty/serial/pmac_zilog.c | 1 + drivers/tty/serial/pxa.c| 1 + drivers/tty/serial/qcom_geni_serial.c | 1 + drivers/tty/serial/rda-uart.c | 1 + drivers/tty/serial/samsung_tty.c| 1 + drivers/tty/serial/sc16is7xx.c | 1 + drivers/tty/serial/serial-tegra.c | 1 + drivers/tty/serial/sh-sci.c | 1 + drivers/tty/serial/sifive.c | 1 + drivers/tty/serial/sprd_serial.c| 1 + drivers/tty/serial/st-asc.c | 1 + drivers/tty/serial/stm32-usart.c| 1 + drivers/tty/serial/sunhv.c | 1 + drivers/tty/serial/sunplus-uart.c | 1 + drivers/tty/serial/sunsab.c | 1 + drivers/tty/serial/sunsu.c | 1 + drivers/tty/serial/sunzilog.c | 1 + drivers/tty/serial/tegra-tcu.c | 1 + drivers/tty/serial/uartlite.c | 1 + drivers/tty/serial/ucc_uart.c | 1 + drivers/tty/serial/vt8500_serial.c | 1 + drivers/tty/serial/xilinx_uartps.c | 1 + include/linux/device.h | 7 +++ include/linux/device/driver.h | 11 +++ 71 files changed, 95 insertions(+) -- 2.37.0.rc0.161.g10f37bed90-goog ___ iommu mailing list iommu@lists.linux-foundation.org
[PATCH v2 1/2] driver core: Add probe_no_timeout flag for drivers
This flag only needs to be set for drivers of devices that meet all the
following conditions:
- Need to probe successfully before userspace init in started
- Have optional suppliers
- Can't wait for deferred_probe_timeout to expire
fw_devlink=on uses this info, as needed, to ignore dependencies on supplier
devices that have not been added or supplier devices that don't have any
drivers. It's still up to the driver to decide which of the missing
suppliers are optional or not.
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Reported-by: Fabio Estevam
Reported-by: Ahmad Fatoum
Tested-by: Fabio Estevam
Signed-off-by: Saravana Kannan
---
drivers/base/base.h | 1 +
drivers/base/core.c | 7 +++
drivers/base/dd.c | 3 +++
include/linux/device.h| 7 +++
include/linux/device/driver.h | 11 +++
5 files changed, 29 insertions(+)
diff --git a/drivers/base/base.h b/drivers/base/base.h
index b3a43a164dcd..149822d2086f 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -193,6 +193,7 @@ extern void device_links_no_driver(struct device *dev);
extern bool device_links_busy(struct device *dev);
extern void device_links_unbind_consumers(struct device *dev);
extern void fw_devlink_drivers_done(void);
+extern void fw_devlink_probe_no_timeout(void);
/* device pm support */
void device_pm_move_to_tail(struct device *dev);
diff --git a/drivers/base/core.c b/drivers/base/core.c
index ccdd5b4295de..8e18904a1584 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -54,6 +54,7 @@ static unsigned int defer_sync_state_count = 1;
static DEFINE_MUTEX(fwnode_link_lock);
static bool fw_devlink_is_permissive(void);
static bool fw_devlink_drv_reg_done;
+static bool fw_devlink_no_timeout;
static bool fw_devlink_best_effort;
/**
@@ -969,6 +970,7 @@ static void device_links_missing_supplier(struct device
*dev)
static bool dev_is_best_effort(struct device *dev)
{
return (fw_devlink_best_effort && dev->can_match) ||
+ (fw_devlink_no_timeout && dev->probe_no_timeout) ||
(dev->fwnode && (dev->fwnode->flags & FWNODE_FLAG_BEST_EFFORT));
}
@@ -1688,6 +1690,11 @@ void fw_devlink_drivers_done(void)
device_links_write_unlock();
}
+void fw_devlink_probe_no_timeout(void)
+{
+ fw_devlink_no_timeout = true;
+}
+
/**
* wait_for_init_devices_probe - Try to probe any device needed for init
*
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 70f79fc71539..943b0363aaab 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -324,6 +324,8 @@ static int deferred_probe_initcall(void)
if (!IS_ENABLED(CONFIG_MODULES))
fw_devlink_drivers_done();
+ else
+ fw_devlink_probe_no_timeout();
/*
* Trigger deferred probe again, this time we won't defer anything
@@ -734,6 +736,7 @@ static int __driver_probe_device(struct device_driver *drv,
struct device *dev)
return -EBUSY;
dev->can_match = true;
+ dev->probe_no_timeout = drv->probe_no_timeout;
pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
drv->bus->name, __func__, dev_name(dev), drv->name);
diff --git a/include/linux/device.h b/include/linux/device.h
index 424b55df0272..e6246b6cf6cf 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -536,6 +536,12 @@ struct device_physical_location {
* @can_match: The device has matched with a driver at least once or it is in
* a bus (like AMBA) which can't check for matching drivers until
* other devices probe successfully.
+ * @probe_no_timeout: Set by driver core to indicate that this device's probe
+ * can't wait till driver_probe_timeout expires. This information
+ * is used by fw_devlink=on to avoid deferring the probe of this
+ * device to wait on supplier devices that haven't been added or
+ * probed successfully.
+ * See also: probe_no_timeout in struct driver.
* @dma_coherent: this particular device is dma coherent, even if the
* architecture supports non-coherent devices.
* @dma_ops_bypass: If set to %true then the dma_ops are bypassed for the
@@ -642,6 +648,7 @@ struct device {
boolof_node_reused:1;
boolstate_synced:1;
boolcan_match:1;
+ boolprobe_no_timeout:1;
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
index 7acaabde5396..2ce60e511504 100644
--- a/include/linux/device/driver.h
+++ b/include/linux/device/driver.h
@@ -55,6 +55,15 @@ enum probe_type {
* @owner: The module
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Thu, Jun 23, 2022 at 5:08 AM Alexander Stein
wrote:
>
> Hi,
>
> Am Dienstag, 21. Juni 2022, 09:28:43 CEST schrieb Tony Lindgren:
> > Hi,
> >
> > * Saravana Kannan [700101 02:00]:
> > > Now that fw_devlink=on by default and fw_devlink supports
> > > "power-domains" property, the execution will never get to the point
> > > where driver_deferred_probe_check_state() is called before the supplier
> > > has probed successfully or before deferred probe timeout has expired.
> > >
> > > So, delete the call and replace it with -ENODEV.
> >
> > Looks like this causes omaps to not boot in Linux next. With this
> > simple-pm-bus fails to probe initially as the power-domain is not
> > yet available. On platform_probe() genpd_get_from_provider() returns
> > -ENOENT.
> >
> > Seems like other stuff is potentially broken too, any ideas on
> > how to fix this?
>
> I think I'm hit by this as well, although I do not get a lockup.
> In my case I'm using arch/arm64/boot/dts/freescale/imx8mq-tqma8mq-mba8mx.dts
> and probing of 3832.blk-ctrl fails as the power-domain is not (yet)
> registed.
Ok, took a look.
The problem is that there are two drivers for the same device and they
both initialize this device.
gpc: gpc@303a {
compatible = "fsl,imx8mq-gpc";
}
$ git grep -l "fsl,imx7d-gpc" -- drivers/
drivers/irqchip/irq-imx-gpcv2.c
drivers/soc/imx/gpcv2.c
IMHO, this is a bad/broken design.
So what's happening is that fw_devlink will block the probe of
3832.blk-ctrl until 303a.gpc is initialized. And it stops
blocking the probe of 3832.blk-ctrl as soon as the first driver
initializes the device. In this case, it's the irqchip driver.
I'd recommend combining these drivers into one. Something like the
patch I'm attaching (sorry for the attachment, copy-paste is mangling
the tabs). Can you give it a shot please?
-Saravana
From 08b8795b6300de89502a26ba3347b2e54d43381d Mon Sep 17 00:00:00 2001
From: Saravana Kannan
Date: Thu, 30 Jun 2022 17:04:26 -0700
Subject: [PATCH] combine drivers
---
drivers/irqchip/irq-imx-gpcv2.c | 19 ---
drivers/soc/imx/gpcv2.c | 10 ++
2 files changed, 14 insertions(+), 15 deletions(-)
diff --git a/drivers/irqchip/irq-imx-gpcv2.c b/drivers/irqchip/irq-imx-gpcv2.c
index b9c22f764b4d..621211e5800a 100644
--- a/drivers/irqchip/irq-imx-gpcv2.c
+++ b/drivers/irqchip/irq-imx-gpcv2.c
@@ -199,13 +199,13 @@ static const struct of_device_id gpcv2_of_match[] = {
{ /* END */ }
};
-static int __init imx_gpcv2_irqchip_init(struct device_node *node,
- struct device_node *parent)
+int imx_gpcv2_irqchip_init(struct platform_device *pdev,
+ unsigned long core_num)
{
+ struct device_node *node = pdev->dev.of_node;
+ struct device_node *parent = of_irq_find_parent(node);
struct irq_domain *parent_domain, *domain;
struct gpcv2_irqchip_data *cd;
- const struct of_device_id *id;
- unsigned long core_num;
int i;
if (!parent) {
@@ -213,14 +213,6 @@ static int __init imx_gpcv2_irqchip_init(struct device_node *node,
return -ENODEV;
}
- id = of_match_node(gpcv2_of_match, node);
- if (!id) {
- pr_err("%pOF: unknown compatibility string\n", node);
- return -ENODEV;
- }
-
- core_num = (unsigned long)id->data;
-
parent_domain = irq_find_host(parent);
if (!parent_domain) {
pr_err("%pOF: unable to get parent domain\n", node);
@@ -285,6 +277,3 @@ static int __init imx_gpcv2_irqchip_init(struct device_node *node,
of_node_clear_flag(node, OF_POPULATED);
return 0;
}
-
-IRQCHIP_DECLARE(imx_gpcv2_imx7d, "fsl,imx7d-gpc", imx_gpcv2_irqchip_init);
-IRQCHIP_DECLARE(imx_gpcv2_imx8mq, "fsl,imx8mq-gpc", imx_gpcv2_irqchip_init);
diff --git a/drivers/soc/imx/gpcv2.c b/drivers/soc/imx/gpcv2.c
index 85aa86e1338a..3ca5382ff5d3 100644
--- a/drivers/soc/imx/gpcv2.c
+++ b/drivers/soc/imx/gpcv2.c
@@ -305,6 +305,7 @@ struct imx_pgc_domain_data {
size_t domains_num;
const struct regmap_access_table *reg_access_table;
const struct imx_pgc_regs *pgc_regs;
+ unsigned long irq_core_num;
};
static inline struct imx_pgc_domain *
@@ -549,6 +550,7 @@ static const struct imx_pgc_domain_data imx7_pgc_domain_data = {
.domains_num = ARRAY_SIZE(imx7_pgc_domains),
.reg_access_table = _access_table,
.pgc_regs = _pgc_regs,
+ .irq_core_num = 2,
};
static const struct imx_pgc_domain imx8m_pgc_domains[] = {
@@ -718,6 +720,7 @@ static const struct imx_pgc_domain_data imx8m_pgc_domain_data = {
.domains_num = ARRAY_SIZE(imx8m_pgc_domains),
.reg_access_table = _access_table,
.pgc_regs = _pgc_regs,
+ .irq_core_num = 4,
};
static const struct imx_pgc_domain imx8mm_pgc_domains[] = {
@@ -1425,6 +1428,9 @@ static struct platform_driver imx_pgc_domain_driver = {
};
builtin_platform_driver(imx_pgc_domain_driver)
+extern int imx_gpcv2_irqchip_init(struct platform_device *pdev,
+ unsigned long core_num);
+
static int imx_gpcv2_probe(struct platform_device *pdev)
{
const struct imx_pgc_domain_data *domain_data =
@@ -1444,6
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Thu, Jun 30, 2022 at 4:26 PM Rob Herring wrote:
>
> On Thu, Jun 30, 2022 at 5:11 PM Saravana Kannan wrote:
> >
> > On Mon, Jun 27, 2022 at 2:10 AM Tony Lindgren wrote:
> > >
> > > * Saravana Kannan [220623 08:17]:
> > > > On Thu, Jun 23, 2022 at 12:01 AM Tony Lindgren wrote:
> > > > >
> > > > > * Saravana Kannan [220622 19:05]:
> > > > > > On Tue, Jun 21, 2022 at 9:59 PM Tony Lindgren
> > > > > > wrote:
> > > > > > > This issue is no directly related fw_devlink. It is a side effect
> > > > > > > of
> > > > > > > removing driver_deferred_probe_check_state(). We no longer return
> > > > > > > -EPROBE_DEFER at the end of driver_deferred_probe_check_state().
> > > > > >
> > > > > > Yes, I understand the issue. But driver_deferred_probe_check_state()
> > > > > > was deleted because fw_devlink=on should have short circuited the
> > > > > > probe attempt with an -EPROBE_DEFER before reaching the bus/driver
> > > > > > probe function and hitting this -ENOENT failure. That's why I was
> > > > > > asking the other questions.
> > > > >
> > > > > OK. So where is the -EPROBE_DEFER supposed to happen without
> > > > > driver_deferred_probe_check_state() then?
> > > >
> > > > device_links_check_suppliers() call inside really_probe() would short
> > > > circuit and return an -EPROBE_DEFER if the device links are created as
> > > > expected.
> > >
> > > OK
> > >
> > > > > Hmm so I'm not seeing any supplier for the top level ocp device in
> > > > > the booting case without your patches. I see the suppliers for the
> > > > > ocp child device instances only.
> > > >
> > > > Hmmm... this is strange (that the device link isn't there), but this
> > > > is what I suspected.
> > >
> > > Yup, maybe it's because of the supplier being a device in the child
> > > interconnect for the ocp.
> >
> > Ugh... yeah, this is why the normal (not SYNC_STATE_ONLY) device link
> > isn't being created.
> >
> > So the aggregated view is something like (I had to set tabs = 4 space
> > to fit it within 80 cols):
> >
> > ocp: ocp { <= Consumer
> > compatible = "simple-pm-bus";
> > power-domains = <_per>; <=== Supplier ref
> >
> > l4_wkup: interconnect@44c0 {
> > compatible = "ti,am33xx-l4-wkup", "simple-pm-bus";
> >
> > segment@20 { /* 0x44e0 */
> > compatible = "simple-pm-bus";
> >
> > target-module@0 { /* 0x44e0, ap 8 58.0 */
> > compatible = "ti,sysc-omap4", "ti,sysc";
> >
> > prcm: prcm@0 {
> > compatible = "ti,am3-prcm", "simple-bus";
> >
> > prm_per: prm@c00 { <= Actual Supplier
> > compatible = "ti,am3-prm-inst",
> > "ti,omap-prm-inst";
> > };
> > };
> > };
> > };
> > };
> > };
> >
> > The power-domain supplier is the great-great-great-grand-child of the
> > consumer. It's not clear to me how this is valid. What does it even
> > mean?
> >
> > Rob, is this considered a valid DT?
>
> Valid DT for broken h/w.
I'm not sure even in that case it's valid. When the parent device is
in reset (when the SoC is coming out of reset), there's no way the
descendant is functional. And if the descendant is not functional, how
is the parent device powered up? This just feels like an incorrect
representation of the real h/w.
> So the domain must be default on and then simple-pm-bus is going to
> hold a reference to the domain preventing it from ever getting powered
> off and things seem to work. Except what happens during suspend?
But how can simple-pm-bus even get a reference? The PM domain can't
get added until we are well into the probe of the simple-pm-bus and
AFAICT the genpd attach is done before the driver probe is even
called.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Mon, Jun 27, 2022 at 2:10 AM Tony Lindgren wrote:
>
> * Saravana Kannan [220623 08:17]:
> > On Thu, Jun 23, 2022 at 12:01 AM Tony Lindgren wrote:
> > >
> > > * Saravana Kannan [220622 19:05]:
> > > > On Tue, Jun 21, 2022 at 9:59 PM Tony Lindgren wrote:
> > > > > This issue is no directly related fw_devlink. It is a side effect of
> > > > > removing driver_deferred_probe_check_state(). We no longer return
> > > > > -EPROBE_DEFER at the end of driver_deferred_probe_check_state().
> > > >
> > > > Yes, I understand the issue. But driver_deferred_probe_check_state()
> > > > was deleted because fw_devlink=on should have short circuited the
> > > > probe attempt with an -EPROBE_DEFER before reaching the bus/driver
> > > > probe function and hitting this -ENOENT failure. That's why I was
> > > > asking the other questions.
> > >
> > > OK. So where is the -EPROBE_DEFER supposed to happen without
> > > driver_deferred_probe_check_state() then?
> >
> > device_links_check_suppliers() call inside really_probe() would short
> > circuit and return an -EPROBE_DEFER if the device links are created as
> > expected.
>
> OK
>
> > > Hmm so I'm not seeing any supplier for the top level ocp device in
> > > the booting case without your patches. I see the suppliers for the
> > > ocp child device instances only.
> >
> > Hmmm... this is strange (that the device link isn't there), but this
> > is what I suspected.
>
> Yup, maybe it's because of the supplier being a device in the child
> interconnect for the ocp.
Ugh... yeah, this is why the normal (not SYNC_STATE_ONLY) device link
isn't being created.
So the aggregated view is something like (I had to set tabs = 4 space
to fit it within 80 cols):
ocp: ocp { <= Consumer
compatible = "simple-pm-bus";
power-domains = <_per>; <=== Supplier ref
l4_wkup: interconnect@44c0 {
compatible = "ti,am33xx-l4-wkup", "simple-pm-bus";
segment@20 { /* 0x44e0 */
compatible = "simple-pm-bus";
target-module@0 { /* 0x44e0, ap 8 58.0 */
compatible = "ti,sysc-omap4", "ti,sysc";
prcm: prcm@0 {
compatible = "ti,am3-prcm", "simple-bus";
prm_per: prm@c00 { <= Actual Supplier
compatible = "ti,am3-prm-inst", "ti,omap-prm-inst";
};
};
};
};
};
};
The power-domain supplier is the great-great-great-grand-child of the
consumer. It's not clear to me how this is valid. What does it even
mean?
Rob, is this considered a valid DT?
Geert, thoughts on whether this is a correct use of simple-pm-bus device?
Also, how is the power domain attach/get working in this case? As far
as I can tell, at least for "simple-pm-bus" devices, the pm domain
attachment is happening under:
really_probe() -> call_driver_probe -> platform_probe() ->
dev_pm_domain_attach()
So, how is the pm domain attach succeeding in the first place without
my changes?
> > Now we need to figure out why it's missing. There are only a few
> > things that could cause this and I don't see any of those. I already
> > checked to make sure the power domain in this instance had a proper
> > driver with a probe() function -- if it didn't, then that's one thing
> > that'd could have caused the missing device link. The device does seem
> > to have a proper driver, so looks like I can rule that out.
> >
> > Can you point me to the dts file that corresponds to the specific
> > board you are testing this one? I probably won't find anything, but I
> > want to rule out some of the possibilities.
>
> You can use the beaglebone black dts for example, that's
> arch/arm/boot/dts/am335x-boneblack.dts and uses am33xx.dtsi for
> ocp interconnect with simple-pm-bus.
>
> > All the device link creation logic is inside drivers/base/core.c. So
> > if you can look at the existing messages or add other stuff to figure
> > out why the device link isn't getting created, that'd be handy. In
> > either case, I'll continue staring at the DT and code to see what
> > might be happening here.
>
> In device_links_check_suppliers() I see these ocp suppliers:
>
> platform ocp: device_links_check_suppliers: 1024: supplier 44e00d00.prm:
> link->status: 0 link->flags: 01c0
> platform ocp: device_links_check_suppliers: 1024: supplier 44e01000.prm:
> link->status: 0 link->flags: 01c0
> platform ocp: device_links_check_suppliers: 1024: supplier 44e00c00.prm:
> link->status: 0 link->flags: 01c0
> platform ocp: device_links_check_suppliers: 1024: supplier 44e00e00.prm:
> link->status: 0 link->flags: 01c0
> platform ocp: device_links_check_suppliers: 1024: supplier 44e01100.prm:
> link->status: 0 link->flags: 01c0
> platform ocp: device_links_check_suppliers: 1024: supplier fixedregulator0:
> link->status: 1
Re: [PATCH v1 2/2] serial: Set probe_no_timeout for all DT based drivers
On Tue, Jun 28, 2022 at 7:00 AM Tobias Klauser wrote:
>
> On 2022-06-28 at 04:01:03 +0200, Saravana Kannan wrote:
> > diff --git a/drivers/tty/serial/8250/8250_acorn.c
> > b/drivers/tty/serial/8250/8250_acorn.c
> > index 758c4aa203ab..5a6f2f67de4f 100644
> > --- a/drivers/tty/serial/8250/8250_acorn.c
> > +++ b/drivers/tty/serial/8250/8250_acorn.c
> > @@ -114,7 +114,6 @@ static const struct ecard_id serial_cids[] = {
> > static struct ecard_driver serial_card_driver = {
> > .probe = serial_card_probe,
> > .remove = serial_card_remove,
> > - .id_table = serial_cids,
>
> Is this change intentional? All other drivers are only changed to set
> .probe_no_time and I don't see anything mentioned in the commit message
> re. this driver's change.
No, that's a mistake. Thanks for catching it! I'll check this patch again.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v1 2/2] serial: Set probe_no_timeout for all DT based drivers
With commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by
default") the probing of TTY consoles could get delayed if they have
optional suppliers that are listed in DT, but those suppliers don't
probe by the time kernel boot finishes. The console devices will probe
eventually after driver_probe_timeout expires.
However, since consoles are often used for debugging kernel issues, it
does not make sense to delay their probe. So, set the newly added
probe_no_timeout flag for all serial drivers that at DT based. This way,
fw_devlink will know not to delay the probing of the consoles past
kernel boot.
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Reported-by: Fabio Estevam
Reported-by: Ahmad Fatoum
Signed-off-by: Saravana Kannan
---
drivers/tty/ehv_bytechan.c | 1 +
drivers/tty/goldfish.c | 1 +
drivers/tty/hvc/hvc_opal.c | 1 +
drivers/tty/serial/8250/8250_acorn.c| 1 -
drivers/tty/serial/8250/8250_aspeed_vuart.c | 1 +
drivers/tty/serial/8250/8250_bcm2835aux.c | 1 +
drivers/tty/serial/8250/8250_bcm7271.c | 1 +
drivers/tty/serial/8250/8250_dw.c | 1 +
drivers/tty/serial/8250/8250_em.c | 1 +
drivers/tty/serial/8250/8250_ingenic.c | 1 +
drivers/tty/serial/8250/8250_lpc18xx.c | 1 +
drivers/tty/serial/8250/8250_mtk.c | 1 +
drivers/tty/serial/8250/8250_of.c | 1 +
drivers/tty/serial/8250/8250_omap.c | 1 +
drivers/tty/serial/8250/8250_pxa.c | 1 +
drivers/tty/serial/8250/8250_tegra.c| 1 +
drivers/tty/serial/8250/8250_uniphier.c | 1 +
drivers/tty/serial/altera_jtaguart.c| 1 +
drivers/tty/serial/altera_uart.c| 1 +
drivers/tty/serial/amba-pl011.c | 1 +
drivers/tty/serial/apbuart.c| 1 +
drivers/tty/serial/ar933x_uart.c| 1 +
drivers/tty/serial/arc_uart.c | 1 +
drivers/tty/serial/atmel_serial.c | 1 +
drivers/tty/serial/bcm63xx_uart.c | 1 +
drivers/tty/serial/clps711x.c | 1 +
drivers/tty/serial/cpm_uart/cpm_uart_core.c | 1 +
drivers/tty/serial/digicolor-usart.c| 1 +
drivers/tty/serial/fsl_linflexuart.c| 1 +
drivers/tty/serial/fsl_lpuart.c | 1 +
drivers/tty/serial/imx.c| 1 +
drivers/tty/serial/lantiq.c | 1 +
drivers/tty/serial/liteuart.c | 1 +
drivers/tty/serial/lpc32xx_hs.c | 1 +
drivers/tty/serial/max310x.c| 1 +
drivers/tty/serial/meson_uart.c | 1 +
drivers/tty/serial/milbeaut_usio.c | 1 +
drivers/tty/serial/mpc52xx_uart.c | 1 +
drivers/tty/serial/mps2-uart.c | 1 +
drivers/tty/serial/msm_serial.c | 1 +
drivers/tty/serial/mvebu-uart.c | 1 +
drivers/tty/serial/mxs-auart.c | 1 +
drivers/tty/serial/omap-serial.c| 1 +
drivers/tty/serial/owl-uart.c | 1 +
drivers/tty/serial/pic32_uart.c | 1 +
drivers/tty/serial/pmac_zilog.c | 1 +
drivers/tty/serial/pxa.c| 1 +
drivers/tty/serial/qcom_geni_serial.c | 1 +
drivers/tty/serial/rda-uart.c | 1 +
drivers/tty/serial/samsung_tty.c| 1 +
drivers/tty/serial/sc16is7xx.c | 1 +
drivers/tty/serial/serial-tegra.c | 1 +
drivers/tty/serial/sh-sci.c | 1 +
drivers/tty/serial/sifive.c | 1 +
drivers/tty/serial/sprd_serial.c| 1 +
drivers/tty/serial/st-asc.c | 1 +
drivers/tty/serial/stm32-usart.c| 1 +
drivers/tty/serial/sunhv.c | 1 +
drivers/tty/serial/sunplus-uart.c | 1 +
drivers/tty/serial/sunsab.c | 1 +
drivers/tty/serial/sunsu.c | 1 +
drivers/tty/serial/sunzilog.c | 1 +
drivers/tty/serial/tegra-tcu.c | 1 +
drivers/tty/serial/uartlite.c | 1 +
drivers/tty/serial/ucc_uart.c | 1 +
drivers/tty/serial/vt8500_serial.c | 1 +
drivers/tty/serial/xilinx_uartps.c | 1 +
67 files changed, 66 insertions(+), 1 deletion(-)
diff --git a/drivers/tty/ehv_bytechan.c b/drivers/tty/ehv_bytechan.c
index 19d32cb6af84..6de710da99be 100644
--- a/drivers/tty/ehv_bytechan.c
+++ b/drivers/tty/ehv_bytechan.c
@@ -739,6 +739,7 @@ static struct platform_driver ehv_bc_tty_driver = {
.driver = {
.name = "ehv-bc",
.of_match_table = ehv_bc_tty_of_ids,
+ .probe_no_timeout = true,
.suppress_bind_attrs = true,
},
.probe = ehv_bc_tty_probe,
diff --git a/drivers/tty/goldfish.c b/drivers/tty/goldfish.c
index c7968aecd870..f9760598c836 100644
--- a/drivers/tty/goldfish.c
+++ b/drivers/tty/goldfish.c
@@ -474,6 +474,7 @@ static struct
[PATCH v1 1/2] driver core: Add probe_no_timeout flag for drivers
This flag only needs to be set for drivers of devices that meet all the
following conditions:
- Need to probe successfully before userspace init in started
- Have optional suppliers
- Can't wait for deferred_probe_timeout to expire
fw_devlink=on uses this info, as needed, to ignore dependencies on supplier
devices that have not been added or supplier devices that don't have any
drivers. It's still up to the driver to decide which of the missing
suppliers are optional or not.
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Reported-by: Fabio Estevam
Reported-by: Ahmad Fatoum
Signed-off-by: Saravana Kannan
---
drivers/base/base.h | 1 +
drivers/base/core.c | 7 +++
drivers/base/dd.c | 3 +++
include/linux/device.h| 7 +++
include/linux/device/driver.h | 11 +++
5 files changed, 29 insertions(+)
diff --git a/drivers/base/base.h b/drivers/base/base.h
index b3a43a164dcd..149822d2086f 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -193,6 +193,7 @@ extern void device_links_no_driver(struct device *dev);
extern bool device_links_busy(struct device *dev);
extern void device_links_unbind_consumers(struct device *dev);
extern void fw_devlink_drivers_done(void);
+extern void fw_devlink_probe_no_timeout(void);
/* device pm support */
void device_pm_move_to_tail(struct device *dev);
diff --git a/drivers/base/core.c b/drivers/base/core.c
index ccdd5b4295de..8e18904a1584 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -54,6 +54,7 @@ static unsigned int defer_sync_state_count = 1;
static DEFINE_MUTEX(fwnode_link_lock);
static bool fw_devlink_is_permissive(void);
static bool fw_devlink_drv_reg_done;
+static bool fw_devlink_no_timeout;
static bool fw_devlink_best_effort;
/**
@@ -969,6 +970,7 @@ static void device_links_missing_supplier(struct device
*dev)
static bool dev_is_best_effort(struct device *dev)
{
return (fw_devlink_best_effort && dev->can_match) ||
+ (fw_devlink_no_timeout && dev->probe_no_timeout) ||
(dev->fwnode && (dev->fwnode->flags & FWNODE_FLAG_BEST_EFFORT));
}
@@ -1688,6 +1690,11 @@ void fw_devlink_drivers_done(void)
device_links_write_unlock();
}
+void fw_devlink_probe_no_timeout(void)
+{
+ fw_devlink_no_timeout = true;
+}
+
/**
* wait_for_init_devices_probe - Try to probe any device needed for init
*
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index e600dd2afc35..9b0ef2b6a058 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -324,6 +324,8 @@ static int deferred_probe_initcall(void)
if (!IS_ENABLED(CONFIG_MODULES))
fw_devlink_drivers_done();
+ else
+ fw_devlink_probe_no_timeout();
/*
* Trigger deferred probe again, this time we won't defer anything
@@ -734,6 +736,7 @@ static int __driver_probe_device(struct device_driver *drv,
struct device *dev)
return -EBUSY;
dev->can_match = true;
+ dev->probe_no_timeout = drv->probe_no_timeout;
pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
drv->bus->name, __func__, dev_name(dev), drv->name);
diff --git a/include/linux/device.h b/include/linux/device.h
index 424b55df0272..e6246b6cf6cf 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -536,6 +536,12 @@ struct device_physical_location {
* @can_match: The device has matched with a driver at least once or it is in
* a bus (like AMBA) which can't check for matching drivers until
* other devices probe successfully.
+ * @probe_no_timeout: Set by driver core to indicate that this device's probe
+ * can't wait till driver_probe_timeout expires. This information
+ * is used by fw_devlink=on to avoid deferring the probe of this
+ * device to wait on supplier devices that haven't been added or
+ * probed successfully.
+ * See also: probe_no_timeout in struct driver.
* @dma_coherent: this particular device is dma coherent, even if the
* architecture supports non-coherent devices.
* @dma_ops_bypass: If set to %true then the dma_ops are bypassed for the
@@ -642,6 +648,7 @@ struct device {
boolof_node_reused:1;
boolstate_synced:1;
boolcan_match:1;
+ boolprobe_no_timeout:1;
#if defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_DEVICE) || \
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU) || \
defined(CONFIG_ARCH_HAS_SYNC_DMA_FOR_CPU_ALL)
diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
index 7acaabde5396..2ce60e511504 100644
--- a/include/linux/device/driver.h
+++ b/include/linux/device/driver.h
@@ -55,6 +55,15 @@ enum probe_type {
* @owner: The module owner.
* @mod_name: Used
[PATCH v1 0/2] Fix console probe delay when stdout-path isn't set
Since the series that fixes console probe delay based on stdout-path[1] got pulled into driver-core-next, I made these patches on top of them. Even if stdout-path isn't set in DT, this patch should take console probe times back to how they were before the deferred_probe_timeout clean up series[2]. Fabio/Ahmad/Sascha, Can you give this a shot please? [1] - https://lore.kernel.org/lkml/20220623080344.783549-1-sarava...@google.com/ [2] - https://lore.kernel.org/lkml/20220601070707.3946847-1-sarava...@google.com/ Thanks, Saravana cc: Rob Herring cc: sascha hauer cc: peng fan cc: kevin hilman cc: ulf hansson cc: len brown cc: pavel machek cc: joerg roedel cc: will deacon cc: andrew lunn cc: heiner kallweit cc: russell king cc: "david s. miller" cc: eric dumazet cc: jakub kicinski cc: paolo abeni cc: linus walleij cc: hideaki yoshifuji cc: david ahern cc: kernel-t...@android.com cc: linux-ker...@vger.kernel.org cc: linux...@vger.kernel.org cc: iommu@lists.linux-foundation.org cc: net...@vger.kernel.org cc: linux-g...@vger.kernel.org Cc: ker...@pengutronix.de Saravana Kannan (2): driver core: Add probe_no_timeout flag for drivers serial: Set probe_no_timeout for all DT based drivers drivers/base/base.h | 1 + drivers/base/core.c | 7 +++ drivers/base/dd.c | 3 +++ drivers/tty/ehv_bytechan.c | 1 + drivers/tty/goldfish.c | 1 + drivers/tty/hvc/hvc_opal.c | 1 + drivers/tty/serial/8250/8250_acorn.c| 1 - drivers/tty/serial/8250/8250_aspeed_vuart.c | 1 + drivers/tty/serial/8250/8250_bcm2835aux.c | 1 + drivers/tty/serial/8250/8250_bcm7271.c | 1 + drivers/tty/serial/8250/8250_dw.c | 1 + drivers/tty/serial/8250/8250_em.c | 1 + drivers/tty/serial/8250/8250_ingenic.c | 1 + drivers/tty/serial/8250/8250_lpc18xx.c | 1 + drivers/tty/serial/8250/8250_mtk.c | 1 + drivers/tty/serial/8250/8250_of.c | 1 + drivers/tty/serial/8250/8250_omap.c | 1 + drivers/tty/serial/8250/8250_pxa.c | 1 + drivers/tty/serial/8250/8250_tegra.c| 1 + drivers/tty/serial/8250/8250_uniphier.c | 1 + drivers/tty/serial/altera_jtaguart.c| 1 + drivers/tty/serial/altera_uart.c| 1 + drivers/tty/serial/amba-pl011.c | 1 + drivers/tty/serial/apbuart.c| 1 + drivers/tty/serial/ar933x_uart.c| 1 + drivers/tty/serial/arc_uart.c | 1 + drivers/tty/serial/atmel_serial.c | 1 + drivers/tty/serial/bcm63xx_uart.c | 1 + drivers/tty/serial/clps711x.c | 1 + drivers/tty/serial/cpm_uart/cpm_uart_core.c | 1 + drivers/tty/serial/digicolor-usart.c| 1 + drivers/tty/serial/fsl_linflexuart.c| 1 + drivers/tty/serial/fsl_lpuart.c | 1 + drivers/tty/serial/imx.c| 1 + drivers/tty/serial/lantiq.c | 1 + drivers/tty/serial/liteuart.c | 1 + drivers/tty/serial/lpc32xx_hs.c | 1 + drivers/tty/serial/max310x.c| 1 + drivers/tty/serial/meson_uart.c | 1 + drivers/tty/serial/milbeaut_usio.c | 1 + drivers/tty/serial/mpc52xx_uart.c | 1 + drivers/tty/serial/mps2-uart.c | 1 + drivers/tty/serial/msm_serial.c | 1 + drivers/tty/serial/mvebu-uart.c | 1 + drivers/tty/serial/mxs-auart.c | 1 + drivers/tty/serial/omap-serial.c| 1 + drivers/tty/serial/owl-uart.c | 1 + drivers/tty/serial/pic32_uart.c | 1 + drivers/tty/serial/pmac_zilog.c | 1 + drivers/tty/serial/pxa.c| 1 + drivers/tty/serial/qcom_geni_serial.c | 1 + drivers/tty/serial/rda-uart.c | 1 + drivers/tty/serial/samsung_tty.c| 1 + drivers/tty/serial/sc16is7xx.c | 1 + drivers/tty/serial/serial-tegra.c | 1 + drivers/tty/serial/sh-sci.c | 1 + drivers/tty/serial/sifive.c | 1 + drivers/tty/serial/sprd_serial.c| 1 + drivers/tty/serial/st-asc.c | 1 + drivers/tty/serial/stm32-usart.c| 1 + drivers/tty/serial/sunhv.c | 1 + drivers/tty/serial/sunplus-uart.c | 1 + drivers/tty/serial/sunsab.c | 1 + drivers/tty/serial/sunsu.c | 1 + drivers/tty/serial/sunzilog.c | 1 + drivers/tty/serial/tegra-tcu.c | 1 + drivers/tty/serial/uartlite.c | 1 + drivers/tty/serial/ucc_uart.c | 1 + drivers/tty/serial/vt8500_serial.c | 1 + drivers/tty/serial/xilinx_uartps.c | 1 + include/linux/device.h | 7 +++ include/linux/device/driver.h | 11 +++ 72
Re: [PATCH v2 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
On Mon, Jun 27, 2022 at 10:50 AM Rob Herring wrote:
>
> On Thu, Jun 23, 2022 at 12:04:21PM +0200, sascha hauer wrote:
> > On Thu, Jun 23, 2022 at 01:03:43AM -0700, Saravana Kannan wrote:
> > > Commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > > enabled iommus and dmas dependency enforcement by default. On some
> > > systems, this caused the console device's probe to get delayed until the
> > > deferred_probe_timeout expires.
> > >
> > > We need consoles to work as soon as possible, so mark the console device
> > > node with FWNODE_FLAG_BEST_EFFORT so that fw_delink knows not to delay
> > > the probe of the console device for suppliers without drivers. The
> > > driver can then make the decision on where it can probe without those
> > > suppliers or defer its probe.
> > >
> > > Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > > Reported-by: Sascha Hauer
> > > Reported-by: Peng Fan
> > > Signed-off-by: Saravana Kannan
> > > Tested-by: Peng Fan
> > > ---
> > > drivers/of/base.c | 2 ++
> > > 1 file changed, 2 insertions(+)
> > >
> > > diff --git a/drivers/of/base.c b/drivers/of/base.c
> > > index d4f98c8469ed..a19cd0c73644 100644
> > > --- a/drivers/of/base.c
> > > +++ b/drivers/of/base.c
> > > @@ -1919,6 +1919,8 @@ void of_alias_scan(void * (*dt_alloc)(u64 size, u64
> > > align))
> > > of_property_read_string(of_aliases, "stdout", );
> > > if (name)
> > > of_stdout = of_find_node_opts_by_path(name,
> > > _stdout_options);
> > > + if (of_stdout)
> > > + of_stdout->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;
> >
> > The device given in the stdout-path property doesn't necessarily have to
> > be consistent with the console= parameter. The former is usually
> > statically set in the device trees contained in the kernel while the
> > latter is dynamically set by the bootloader. So if you change the
> > console uart in the bootloader then you'll still run into this trap.
> >
> > It's problematic to consult only the device tree for dependencies. I
> > found several examples of drivers in the tree for which dma support
> > is optional. They use it if they can, but continue without it when
> > not available. "hwlock" is another property which consider several
> > drivers as optional. Also consider SoCs in early upstreaming phases
> > when the device tree is merged with "dmas" or "hwlock" properties,
> > but the corresponding drivers are not yet upstreamed. It's not nice
> > to defer probing of all these devices for a long time.
> >
> > I wonder if it wouldn't be a better approach to just probe all devices
> > and record the device(node) they are waiting on. Then you know that you
> > don't need to probe them again until the device they are waiting for
> > is available.
>
> Can't we have a driver flag 'I have optional dependencies' that will
> trigger probe without all dependencies and then the driver can defer
> probe if required dependencies are not yet met.
Haha... that's kinda what I'm working on right now. But named
intentionally in a more limited sense of "I can't wait for the
timeout" where fw_devlink will relax and allow the driver to probe
(and have it make the call) once we hit late_initcall(). I'm
explicitly limiting it to "timeout" because we don't want everyone
adding this flag everytime they hit an issue. That'll beat the point
of fw_devlink=on.
Also, setting the flag for a driver to fix one system might break
another system because in the other system the user might want to wait
for the timeout because the supplier drivers would be loaded before
the timeout.
Another option is to restart the timer (if it hasn't expired) when
filesystems get mounted (in addition to the current "when new driver
gets registered). That way, we might be able to drop the timeout from
10s to 5s.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
On Thu, Jun 23, 2022 at 1:37 PM sascha hauer wrote:
>
> On Thu, Jun 23, 2022 at 10:26:46AM -0700, Saravana Kannan wrote:
> > On Thu, Jun 23, 2022 at 3:05 AM sascha hauer wrote:
> > >
> > > On Thu, Jun 23, 2022 at 01:03:43AM -0700, Saravana Kannan wrote:
> > > > Commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > > > enabled iommus and dmas dependency enforcement by default. On some
> > > > systems, this caused the console device's probe to get delayed until the
> > > > deferred_probe_timeout expires.
> > > >
> > > > We need consoles to work as soon as possible, so mark the console device
> > > > node with FWNODE_FLAG_BEST_EFFORT so that fw_delink knows not to delay
> > > > the probe of the console device for suppliers without drivers. The
> > > > driver can then make the decision on where it can probe without those
> > > > suppliers or defer its probe.
> > > >
> > > > Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > > > Reported-by: Sascha Hauer
> > > > Reported-by: Peng Fan
> > > > Signed-off-by: Saravana Kannan
> > > > Tested-by: Peng Fan
> > > > ---
> > > > drivers/of/base.c | 2 ++
> > > > 1 file changed, 2 insertions(+)
> > > >
> > > > diff --git a/drivers/of/base.c b/drivers/of/base.c
> > > > index d4f98c8469ed..a19cd0c73644 100644
> > > > --- a/drivers/of/base.c
> > > > +++ b/drivers/of/base.c
> > > > @@ -1919,6 +1919,8 @@ void of_alias_scan(void * (*dt_alloc)(u64 size,
> > > > u64 align))
> > > > of_property_read_string(of_aliases, "stdout",
> > > > );
> > > > if (name)
> > > > of_stdout = of_find_node_opts_by_path(name,
> > > > _stdout_options);
> > > > + if (of_stdout)
> > > > + of_stdout->fwnode.flags |=
> > > > FWNODE_FLAG_BEST_EFFORT;
> > >
> > > The device given in the stdout-path property doesn't necessarily have to
> > > be consistent with the console= parameter. The former is usually
> > > statically set in the device trees contained in the kernel while the
> > > latter is dynamically set by the bootloader. So if you change the
> > > console uart in the bootloader then you'll still run into this trap.
> > >
> > > It's problematic to consult only the device tree for dependencies. I
> > > found several examples of drivers in the tree for which dma support
> > > is optional. They use it if they can, but continue without it when
> > > not available. "hwlock" is another property which consider several
> > > drivers as optional. Also consider SoCs in early upstreaming phases
> > > when the device tree is merged with "dmas" or "hwlock" properties,
> > > but the corresponding drivers are not yet upstreamed. It's not nice
> > > to defer probing of all these devices for a long time.
> > >
> > > I wonder if it wouldn't be a better approach to just probe all devices
> > > and record the device(node) they are waiting on. Then you know that you
> > > don't need to probe them again until the device they are waiting for
> > > is available.
> >
> > That actually breaks things in a worse sense. There are cases where
> > the consumer driver is built in and the optional supplier driver is
> > loaded at boot. Without fw_devlink and the deferred probe timeout, we
> > end up probing the consumer with limited functionality. With the
> > current setup, sure we delay some probes a bit but at least everything
> > works with the right functionality. And you can reduce or remove the
> > delay if you want to optimize it.
>
> We have optional and mandatory resources. In this situation a driver has
> to decide what to do. Either it continues with limited resources or it
> defers probing. Some drivers try to allocate the optional resources at
> open time so that they are able to use them once they are available. We
> could even think of an asynchronous callback into a driver when a
> resource becomes available. Whether we put this decision what is
> optional or not into the driver or in the framework doesn't make a
> difference to the problem, it is still the same: When a resource is not
> yet available we have no idea if and when it becomes available, if it's
> worth waiting for it or not.
>
> The difference is that with my proposal (which isn't actually mine but
> from my collegue Lucas) a driver can decide very fine grained how it
> wants to deal with the situation. With fw_devlink we try to put this
> intelligence into the framework and it seems there are quite some quirks
> necessary to get that running for everyone.
That's one possible solution, but for that to work, all drivers with
optional suppliers would need to be changed to take advantage of this
callback to work correctly when the optional suppliers become
available. We could add this callback, but it would be a long time
before the callback handles all/most cases of optional suppliers.
One of the goals of fw_devlink is so that people can stop having to
play initcall chicken where they try to tune their
Re: [PATCH v2 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
On Thu, Jun 23, 2022 at 10:36 AM Ahmad Fatoum wrote:
>
> Hello Saravana,
>
> On 23.06.22 19:26, Saravana Kannan wrote:
> > On Thu, Jun 23, 2022 at 3:05 AM sascha hauer wrote:
> >>
> >> On Thu, Jun 23, 2022 at 01:03:43AM -0700, Saravana Kannan wrote:
> >>> Commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> >>> enabled iommus and dmas dependency enforcement by default. On some
> >>> systems, this caused the console device's probe to get delayed until the
> >>> deferred_probe_timeout expires.
> >>>
> >>> We need consoles to work as soon as possible, so mark the console device
> >>> node with FWNODE_FLAG_BEST_EFFORT so that fw_delink knows not to delay
> >>> the probe of the console device for suppliers without drivers. The
> >>> driver can then make the decision on where it can probe without those
> >>> suppliers or defer its probe.
> >>>
> >>> Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> >>> Reported-by: Sascha Hauer
> >>> Reported-by: Peng Fan
> >>> Signed-off-by: Saravana Kannan
> >>> Tested-by: Peng Fan
> >>> ---
> >>> drivers/of/base.c | 2 ++
> >>> 1 file changed, 2 insertions(+)
> >>>
> >>> diff --git a/drivers/of/base.c b/drivers/of/base.c
> >>> index d4f98c8469ed..a19cd0c73644 100644
> >>> --- a/drivers/of/base.c
> >>> +++ b/drivers/of/base.c
> >>> @@ -1919,6 +1919,8 @@ void of_alias_scan(void * (*dt_alloc)(u64 size, u64
> >>> align))
> >>> of_property_read_string(of_aliases, "stdout",
> >>> );
> >>> if (name)
> >>> of_stdout = of_find_node_opts_by_path(name,
> >>> _stdout_options);
> >>> + if (of_stdout)
> >>> + of_stdout->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;
> >>
> >> The device given in the stdout-path property doesn't necessarily have to
> >> be consistent with the console= parameter. The former is usually
> >> statically set in the device trees contained in the kernel while the
> >> latter is dynamically set by the bootloader. So if you change the
> >> console uart in the bootloader then you'll still run into this trap.
> >>
> >> It's problematic to consult only the device tree for dependencies. I
> >> found several examples of drivers in the tree for which dma support
> >> is optional. They use it if they can, but continue without it when
> >> not available. "hwlock" is another property which consider several
> >> drivers as optional. Also consider SoCs in early upstreaming phases
> >> when the device tree is merged with "dmas" or "hwlock" properties,
> >> but the corresponding drivers are not yet upstreamed. It's not nice
> >> to defer probing of all these devices for a long time.
> >>
> >> I wonder if it wouldn't be a better approach to just probe all devices
> >> and record the device(node) they are waiting on. Then you know that you
> >> don't need to probe them again until the device they are waiting for
> >> is available.
> >
> > That actually breaks things in a worse sense. There are cases where
> > the consumer driver is built in and the optional supplier driver is
> > loaded at boot. Without fw_devlink and the deferred probe timeout, we
> > end up probing the consumer with limited functionality. With the
> > current setup, sure we delay some probes a bit but at least everything
> > works with the right functionality. And you can reduce or remove the
> > delay if you want to optimize it.
>
> I have a system that doesn't use stdout-path and has the bootloader
> set console= either to ttynull when secure booting or to an UART
> when booting normally. How would I optimize the kernel to avoid
> my UART being loaded after DMA controller probe without touching
> the bootloader?
>
Thanks for the report Ahmad. I think someone else reported a similar
thing in another thread. I plan to take a look at it. It should be
possible to find the device and set the flag for those devices too.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
On Thu, Jun 23, 2022 at 9:39 AM Andy Shevchenko wrote: > > On Thu, Jun 23, 2022 at 12:04:21PM +0200, sascha hauer wrote: > > On Thu, Jun 23, 2022 at 01:03:43AM -0700, Saravana Kannan wrote: > > ... > > > I wonder if it wouldn't be a better approach to just probe all devices > > and record the device(node) they are waiting on. Then you know that you > > don't need to probe them again until the device they are waiting for > > is available. > > There may be no device, but resource. And we become again to the something > like > deferred probe ugly hack. > > The real solution is to rework device driver model in the kernel that it will > create a graph of dependencies and then simply follow it. But actually it > should > be more than 1 graph, because there are resources and there are power, clock > and > resets that may be orthogonal to the higher dependencies (like driver X > provides > a resource to driver Y). We already do this with fw_devlink for DT based systems and we do effectively just probe the devices in graph order (by deferring any attempts that happen too early and before it even gets to the driver). The problem is the knowledge of what's considered an optional vs mandatory dependency and that's affected by the global state of driver support in the kernel. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
On Thu, Jun 23, 2022 at 3:05 AM sascha hauer wrote:
>
> On Thu, Jun 23, 2022 at 01:03:43AM -0700, Saravana Kannan wrote:
> > Commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > enabled iommus and dmas dependency enforcement by default. On some
> > systems, this caused the console device's probe to get delayed until the
> > deferred_probe_timeout expires.
> >
> > We need consoles to work as soon as possible, so mark the console device
> > node with FWNODE_FLAG_BEST_EFFORT so that fw_delink knows not to delay
> > the probe of the console device for suppliers without drivers. The
> > driver can then make the decision on where it can probe without those
> > suppliers or defer its probe.
> >
> > Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > Reported-by: Sascha Hauer
> > Reported-by: Peng Fan
> > Signed-off-by: Saravana Kannan
> > Tested-by: Peng Fan
> > ---
> > drivers/of/base.c | 2 ++
> > 1 file changed, 2 insertions(+)
> >
> > diff --git a/drivers/of/base.c b/drivers/of/base.c
> > index d4f98c8469ed..a19cd0c73644 100644
> > --- a/drivers/of/base.c
> > +++ b/drivers/of/base.c
> > @@ -1919,6 +1919,8 @@ void of_alias_scan(void * (*dt_alloc)(u64 size, u64
> > align))
> > of_property_read_string(of_aliases, "stdout", );
> > if (name)
> > of_stdout = of_find_node_opts_by_path(name,
> > _stdout_options);
> > + if (of_stdout)
> > + of_stdout->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;
>
> The device given in the stdout-path property doesn't necessarily have to
> be consistent with the console= parameter. The former is usually
> statically set in the device trees contained in the kernel while the
> latter is dynamically set by the bootloader. So if you change the
> console uart in the bootloader then you'll still run into this trap.
>
> It's problematic to consult only the device tree for dependencies. I
> found several examples of drivers in the tree for which dma support
> is optional. They use it if they can, but continue without it when
> not available. "hwlock" is another property which consider several
> drivers as optional. Also consider SoCs in early upstreaming phases
> when the device tree is merged with "dmas" or "hwlock" properties,
> but the corresponding drivers are not yet upstreamed. It's not nice
> to defer probing of all these devices for a long time.
>
> I wonder if it wouldn't be a better approach to just probe all devices
> and record the device(node) they are waiting on. Then you know that you
> don't need to probe them again until the device they are waiting for
> is available.
That actually breaks things in a worse sense. There are cases where
the consumer driver is built in and the optional supplier driver is
loaded at boot. Without fw_devlink and the deferred probe timeout, we
end up probing the consumer with limited functionality. With the
current setup, sure we delay some probes a bit but at least everything
works with the right functionality. And you can reduce or remove the
delay if you want to optimize it.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Thu, Jun 23, 2022 at 12:01 AM Tony Lindgren wrote:
>
> * Saravana Kannan [220622 19:05]:
> > On Tue, Jun 21, 2022 at 9:59 PM Tony Lindgren wrote:
> > >
> > > Hi,
> > >
> > > * Saravana Kannan [220621 19:29]:
> > > > On Tue, Jun 21, 2022 at 12:28 AM Tony Lindgren wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > * Saravana Kannan [700101 02:00]:
> > > > > > Now that fw_devlink=on by default and fw_devlink supports
> > > > > > "power-domains" property, the execution will never get to the point
> > > > > > where driver_deferred_probe_check_state() is called before the
> > > > > > supplier
> > > > > > has probed successfully or before deferred probe timeout has
> > > > > > expired.
> > > > > >
> > > > > > So, delete the call and replace it with -ENODEV.
> > > > >
> > > > > Looks like this causes omaps to not boot in Linux next.
> > > >
> > > > Can you please point me to an example DTS I could use for debugging
> > > > this? I'm assuming you are leaving fw_devlink=on and not turning it
> > > > off or putting it in permissive mode.
> > >
> > > Sure, this seems to happen at least with simple-pm-bus as the top
> > > level interconnect with a configured power-domains property:
> > >
> > > $ git grep -A10 "ocp {" arch/arm/boot/dts/*.dtsi | grep -B3 -A4
> > > simple-pm-bus
> >
> > Thanks for the example. I generally start looking from dts (not dtsi)
> > files in case there are some DT property override/additions after the
> > dtsi files are included in the dts file. But I'll assume for now
> > that's not the case. If there's a specific dts file for a board I can
> > look from that'd be helpful to rule out those kinds of issues.
> >
> > For now, I looked at arch/arm/boot/dts/omap4.dtsi.
>
> OK it should be very similar for all the affected SoCs.
>
> > > This issue is no directly related fw_devlink. It is a side effect of
> > > removing driver_deferred_probe_check_state(). We no longer return
> > > -EPROBE_DEFER at the end of driver_deferred_probe_check_state().
> >
> > Yes, I understand the issue. But driver_deferred_probe_check_state()
> > was deleted because fw_devlink=on should have short circuited the
> > probe attempt with an -EPROBE_DEFER before reaching the bus/driver
> > probe function and hitting this -ENOENT failure. That's why I was
> > asking the other questions.
>
> OK. So where is the -EPROBE_DEFER supposed to happen without
> driver_deferred_probe_check_state() then?
device_links_check_suppliers() call inside really_probe() would short
circuit and return an -EPROBE_DEFER if the device links are created as
expected.
>
> > > > > On platform_probe() genpd_get_from_provider() returns
> > > > > -ENOENT.
> > > >
> > > > This error is with the series I assume?
> > >
> > > On the first probe genpd_get_from_provider() will return -ENOENT in
> > > both cases. The list is empty on the first probe and there are no
> > > genpd providers at this point.
> > >
> > > Earlier with driver_deferred_probe_check_state(), the initial -ENOENT
> > > ends up getting changed to -EPROBE_DEFER at the end of
> > > driver_deferred_probe_check_state(), we are now missing that.
> >
> > Right, I was aware -ENOENT would be returned if we got this far. But
> > the point of this series is that you shouldn't have gotten that far
> > before your pm domain device is ready. Hence my questions from the
> > earlier reply.
>
> OK
>
> > Can I get answers to rest of my questions in the first reply please?
> > That should help us figure out why fw_devlink let us get this far.
> > Summarize them here to make it easy:
> > * Are you running with fw_devlink=on?
>
> Yes with the default with no specific kernel params so looks like
> FW_DEVLINK_FLAGS_ON.
>
> > * Is the"ti,omap4-prm-inst"/"ti,omap-prm-inst" built-in in this case?
>
> Yes
>
> > * If it's not built-in, can you please try deferred_probe_timeout=0
> > and deferred_probe_timeout=30 and see if either one of them help?
>
> It's built in so I did not try these.
>
> > * Can I get the output of "ls -d supplier:*" and "cat
> > supplier:*/status" output from the sysfs dir for the ocp device
> > without this series where it boots properly.
>
> Hmm so I'm not seeing any supplier for the top level ocp device in
> the booting case without your patches. I see the suppliers for the
> ocp child device instances only.
Hmmm... this is strange (that the device link isn't there), but this
is what I suspected.
Now we need to figure out why it's missing. There are only a few
things that could cause this and I don't see any of those. I already
checked to make sure the power domain in this instance had a proper
driver with a probe() function -- if it didn't, then that's one thing
that'd could have caused the missing device link. The device does seem
to have a proper driver, so looks like I can rule that out.
Can you point me to the dts file that corresponds to the specific
board you are testing this one? I probably won't find anything, but I
want to rule out some of the possibilities.
All the
[PATCH v2 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
Commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
enabled iommus and dmas dependency enforcement by default. On some
systems, this caused the console device's probe to get delayed until the
deferred_probe_timeout expires.
We need consoles to work as soon as possible, so mark the console device
node with FWNODE_FLAG_BEST_EFFORT so that fw_delink knows not to delay
the probe of the console device for suppliers without drivers. The
driver can then make the decision on where it can probe without those
suppliers or defer its probe.
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Signed-off-by: Saravana Kannan
Tested-by: Peng Fan
---
drivers/of/base.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/of/base.c b/drivers/of/base.c
index d4f98c8469ed..a19cd0c73644 100644
--- a/drivers/of/base.c
+++ b/drivers/of/base.c
@@ -1919,6 +1919,8 @@ void of_alias_scan(void * (*dt_alloc)(u64 size, u64
align))
of_property_read_string(of_aliases, "stdout", );
if (name)
of_stdout = of_find_node_opts_by_path(name,
_stdout_options);
+ if (of_stdout)
+ of_stdout->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;
}
if (!of_aliases)
--
2.37.0.rc0.161.g10f37bed90-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 1/2] driver core: fw_devlink: Allow firmware to mark devices as best effort
When firmware sets the FWNODE_FLAG_BEST_EFFORT flag for a fwnode,
fw_devlink will do a best effort ordering for that device where it'll
only enforce the probe/suspend/resume ordering of that device with
suppliers that have drivers. The driver of that device can then decide
if it wants to defer probe or probe without the suppliers.
This will be useful for avoid probe delays of the console device that
were caused by commit 71066545b48e ("driver core: Set
fw_devlink.strict=1 by default").
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Signed-off-by: Saravana Kannan
Tested-by: Peng Fan
---
drivers/base/core.c| 3 ++-
include/linux/fwnode.h | 4
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 839f64485a55..ccdd5b4295de 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -968,7 +968,8 @@ static void device_links_missing_supplier(struct device
*dev)
static bool dev_is_best_effort(struct device *dev)
{
- return fw_devlink_best_effort && dev->can_match;
+ return (fw_devlink_best_effort && dev->can_match) ||
+ (dev->fwnode && (dev->fwnode->flags & FWNODE_FLAG_BEST_EFFORT));
}
/**
diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
index 9a81c4410b9f..89b9bdfca925 100644
--- a/include/linux/fwnode.h
+++ b/include/linux/fwnode.h
@@ -27,11 +27,15 @@ struct device;
* driver needs its child devices to be bound with
* their respective drivers as soon as they are
* added.
+ * BEST_EFFORT: The fwnode/device needs to probe early and might be missing
some
+ * suppliers. Only enforce ordering with suppliers that have
+ * drivers.
*/
#define FWNODE_FLAG_LINKS_ADDEDBIT(0)
#define FWNODE_FLAG_NOT_DEVICE BIT(1)
#define FWNODE_FLAG_INITIALIZEDBIT(2)
#define FWNODE_FLAG_NEEDS_CHILD_BOUND_ON_ADD BIT(3)
+#define FWNODE_FLAG_BEST_EFFORTBIT(4)
struct fwnode_handle {
struct fwnode_handle *secondary;
--
2.37.0.rc0.161.g10f37bed90-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 0/2] Fix console probe delay due to fw_devlink
fw_devlink.strict=1 has been enabled by default. This was delaying the probe of console devices. This series fixes that. v1->v2: - Added missing NULL check - Added Tested-by tags -Saravana cc: sascha hauer cc: peng fan cc: kevin hilman cc: ulf hansson cc: len brown cc: pavel machek cc: joerg roedel cc: will deacon cc: andrew lunn cc: heiner kallweit cc: russell king cc: "david s. miller" cc: eric dumazet cc: jakub kicinski cc: paolo abeni cc: linus walleij cc: hideaki yoshifuji cc: david ahern cc: kernel-t...@android.com cc: linux-ker...@vger.kernel.org cc: linux...@vger.kernel.org cc: iommu@lists.linux-foundation.org cc: net...@vger.kernel.org cc: linux-g...@vger.kernel.org Cc: ker...@pengutronix.de Saravana Kannan (2): driver core: fw_devlink: Allow firmware to mark devices as best effort of: base: Avoid console probe delay when fw_devlink.strict=1 drivers/base/core.c| 3 ++- drivers/of/base.c | 2 ++ include/linux/fwnode.h | 4 3 files changed, 8 insertions(+), 1 deletion(-) -- 2.37.0.rc0.161.g10f37bed90-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1 1/2] driver core: fw_devlink: Allow firmware to mark devices as best effort
On Wed, Jun 22, 2022 at 11:50 PM Sascha Hauer wrote:
>
> On Wed, Jun 22, 2022 at 02:59:10PM -0700, Saravana Kannan wrote:
> > When firmware sets the FWNODE_FLAG_BEST_EFFORT flag for a fwnode,
> > fw_devlink will do a best effort ordering for that device where it'll
> > only enforce the probe/suspend/resume ordering of that device with
> > suppliers that have drivers. The driver of that device can then decide
> > if it wants to defer probe or probe without the suppliers.
> >
> > This will be useful for avoid probe delays of the console device that
> > were caused by commit 71066545b48e ("driver core: Set
> > fw_devlink.strict=1 by default").
> >
> > Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
> > Reported-by: Sascha Hauer
> > Reported-by: Peng Fan
> > Signed-off-by: Saravana Kannan
> > ---
> > drivers/base/core.c| 3 ++-
> > include/linux/fwnode.h | 4
> > 2 files changed, 6 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/base/core.c b/drivers/base/core.c
> > index 839f64485a55..61edd18b7bf3 100644
> > --- a/drivers/base/core.c
> > +++ b/drivers/base/core.c
> > @@ -968,7 +968,8 @@ static void device_links_missing_supplier(struct device
> > *dev)
> >
> > static bool dev_is_best_effort(struct device *dev)
> > {
> > - return fw_devlink_best_effort && dev->can_match;
> > + return (fw_devlink_best_effort && dev->can_match) ||
> > + dev->fwnode->flags & FWNODE_FLAG_BEST_EFFORT;
>
> Check for dev->fwnode first. I am running in a NULL pointer exception
> here for a device that doesn't have a fwnode.
Oops. Fixed and sent out a v2.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1 0/2] Fix console probe delay due to fw_devlink
On Wed, Jun 22, 2022 at 3:32 PM Peng Fan wrote: > > > Subject: [PATCH v1 0/2] Fix console probe delay due to fw_devlink > > > > fw_devlink.strict=1 has been enabled by default. This was delaying the probe > > of console devices. This series fixes that. > > > > Sasha/Peng, > > > > Can you test this please? > > Thanks, just give a test on i.MX8MP-EVK, works well now. > > Tested-by: Peng Fan #i.MX8MP-EVK Lol, that was quick! Thanks! -Saravana > > Thanks, > Peng. > > > > > -Saravana > > > > Cc: Sascha Hauer > > Cc: Peng Fan > > Cc: Kevin Hilman > > Cc: Ulf Hansson > > Cc: Len Brown > > Cc: Pavel Machek > > Cc: Joerg Roedel > > Cc: Will Deacon > > Cc: Andrew Lunn > > Cc: Heiner Kallweit > > Cc: Russell King > > Cc: "David S. Miller" > > Cc: Eric Dumazet > > Cc: Jakub Kicinski > > Cc: Paolo Abeni > > Cc: Linus Walleij > > Cc: Hideaki YOSHIFUJI > > Cc: David Ahern > > Cc: kernel-t...@android.com > > Cc: linux-ker...@vger.kernel.org > > Cc: linux...@vger.kernel.org > > Cc: iommu@lists.linux-foundation.org > > Cc: net...@vger.kernel.org > > Cc: linux-g...@vger.kernel.org > > Cc: ker...@pengutronix.de > > > > Saravana Kannan (2): > > driver core: fw_devlink: Allow firmware to mark devices as best effort > > of: base: Avoid console probe delay when fw_devlink.strict=1 > > > > drivers/base/core.c| 3 ++- > > drivers/of/base.c | 2 ++ > > include/linux/fwnode.h | 4 > > 3 files changed, 8 insertions(+), 1 deletion(-) > > > > -- > > 2.37.0.rc0.161.g10f37bed90-goog > ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 7/9] driver core: Set fw_devlink.strict=1 by default
On Wed, Jun 22, 2022 at 1:35 PM Saravana Kannan wrote: > > On Wed, Jun 22, 2022 at 12:40 PM Saravana Kannan wrote: > > > > On Wed, Jun 22, 2022 at 1:44 AM Linus Walleij > > wrote: > > > > > > On Wed, Jun 22, 2022 at 9:48 AM Sascha Hauer wrote: > > > > > > > This patch has the effect that console UART devices which have "dmas" > > > > properties specified in the device tree get deferred for 10 to 20 > > > > seconds. This happens on i.MX and likely on other SoCs as well. On i.MX > > > > the dma channel is only requested at UART startup time and not at probe > > > > time. dma is not used for the console. Nevertheless with this driver > > > > probe > > > > defers until the dma engine driver is available. > > > > FYI, if most of the drivers are built in, you could set > > deferred_probe_timeout=1 to reduce the impact of this (should drop > > down to 1 to 2 seconds). Is that an option until we figure out > > something better? > > > > Actually, why isn't earlyconsole being used? That doesn't get blocked > > on anything and the main point of that is to have console working from > > really early on. > > > > > > > > > > It shouldn't go in as-is. > > > > > > This affects all machines with the PL011 UART and DMAs specified as > > > well. > > > > > > It would be best if the console subsystem could be treated special and > > > not require DMA devlink to be satisfied before probing. > > > > If we can mark the console devices somehow before their drivers probe > > them, I can make fw_devlink give them special treatment. Is there any > > way I could identify them before their drivers probe? > > > > > It seems devlink is not quite aware of the concept of resources that are > > > necessary to probe vs resources that are nice to have and might be > > > added after probe. > > > > Correct, it can't tell them apart. Which is why it tries its best to > > enforce them, get most of them ordered properly and then gives up > > enforcing the rest after deferred_probe_timeout= expires. There's a > > bit more nuance than what I explained here (explained in earlier > > commit texts, LPC talks), but that's the gist of it. That's what's > > going on in this case Sascha is pointing out.z > > > > > We need a strong devlink for the first category > > > and maybe a weak devlink for the latter category. > > > > > > I don't know if this is a generic hardware property for all operating > > > systems so it could be a DT property such as dma-weak-dependency? > > > > > > Or maybe compromize and add a linux,dma-weak-dependency; > > > property? > > > > The linux,dma-weak-dependency might be an option, but then if the > > kernel version changes and we want to enforce it because we now have a > > dma driver (not related to Shasha's example) support, then the > > fw_devlink still can't enforce it because of that property. But maybe > > that's okay? The consumer can try to use dma and defer probe if it > > fails? > > > > Another option is to mark console devices in DT with some property and > > we can give special treatment for those without waiting for > > deferred_probe_timeout= to expire. > > Heh, looks like there's already a property for that: stdout-path. > > Let me send a series that'll use that to give special treatment to > console devices. Here's the fix. https://lore.kernel.org/lkml/20220622215912.550419-1-sarava...@google.com/ Sascha, can you give it a shot? -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v1 2/2] of: base: Avoid console probe delay when fw_devlink.strict=1
Commit 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
enabled iommus and dmas dependency enforcement by default. On some
systems, this caused the console device's probe to get delayed until the
deferred_probe_timeout expires.
We need consoles to work as soon as possible, so mark the console device
node with FWNODE_FLAG_BEST_EFFORT so that fw_delink knows not to delay
the probe of the console device for suppliers without drivers. The
driver can then make the decision on where it can probe without those
suppliers or defer its probe.
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Signed-off-by: Saravana Kannan
---
drivers/of/base.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/of/base.c b/drivers/of/base.c
index d4f98c8469ed..a19cd0c73644 100644
--- a/drivers/of/base.c
+++ b/drivers/of/base.c
@@ -1919,6 +1919,8 @@ void of_alias_scan(void * (*dt_alloc)(u64 size, u64
align))
of_property_read_string(of_aliases, "stdout", );
if (name)
of_stdout = of_find_node_opts_by_path(name,
_stdout_options);
+ if (of_stdout)
+ of_stdout->fwnode.flags |= FWNODE_FLAG_BEST_EFFORT;
}
if (!of_aliases)
--
2.37.0.rc0.161.g10f37bed90-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v1 1/2] driver core: fw_devlink: Allow firmware to mark devices as best effort
When firmware sets the FWNODE_FLAG_BEST_EFFORT flag for a fwnode,
fw_devlink will do a best effort ordering for that device where it'll
only enforce the probe/suspend/resume ordering of that device with
suppliers that have drivers. The driver of that device can then decide
if it wants to defer probe or probe without the suppliers.
This will be useful for avoid probe delays of the console device that
were caused by commit 71066545b48e ("driver core: Set
fw_devlink.strict=1 by default").
Fixes: 71066545b48e ("driver core: Set fw_devlink.strict=1 by default")
Reported-by: Sascha Hauer
Reported-by: Peng Fan
Signed-off-by: Saravana Kannan
---
drivers/base/core.c| 3 ++-
include/linux/fwnode.h | 4
2 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 839f64485a55..61edd18b7bf3 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -968,7 +968,8 @@ static void device_links_missing_supplier(struct device
*dev)
static bool dev_is_best_effort(struct device *dev)
{
- return fw_devlink_best_effort && dev->can_match;
+ return (fw_devlink_best_effort && dev->can_match) ||
+ dev->fwnode->flags & FWNODE_FLAG_BEST_EFFORT;
}
/**
diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
index 9a81c4410b9f..89b9bdfca925 100644
--- a/include/linux/fwnode.h
+++ b/include/linux/fwnode.h
@@ -27,11 +27,15 @@ struct device;
* driver needs its child devices to be bound with
* their respective drivers as soon as they are
* added.
+ * BEST_EFFORT: The fwnode/device needs to probe early and might be missing
some
+ * suppliers. Only enforce ordering with suppliers that have
+ * drivers.
*/
#define FWNODE_FLAG_LINKS_ADDEDBIT(0)
#define FWNODE_FLAG_NOT_DEVICE BIT(1)
#define FWNODE_FLAG_INITIALIZEDBIT(2)
#define FWNODE_FLAG_NEEDS_CHILD_BOUND_ON_ADD BIT(3)
+#define FWNODE_FLAG_BEST_EFFORTBIT(4)
struct fwnode_handle {
struct fwnode_handle *secondary;
--
2.37.0.rc0.161.g10f37bed90-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v1 0/2] Fix console probe delay due to fw_devlink
fw_devlink.strict=1 has been enabled by default. This was delaying the probe of console devices. This series fixes that. Sasha/Peng, Can you test this please? -Saravana Cc: Sascha Hauer Cc: Peng Fan Cc: Kevin Hilman Cc: Ulf Hansson Cc: Len Brown Cc: Pavel Machek Cc: Joerg Roedel Cc: Will Deacon Cc: Andrew Lunn Cc: Heiner Kallweit Cc: Russell King Cc: "David S. Miller" Cc: Eric Dumazet Cc: Jakub Kicinski Cc: Paolo Abeni Cc: Linus Walleij Cc: Hideaki YOSHIFUJI Cc: David Ahern Cc: kernel-t...@android.com Cc: linux-ker...@vger.kernel.org Cc: linux...@vger.kernel.org Cc: iommu@lists.linux-foundation.org Cc: net...@vger.kernel.org Cc: linux-g...@vger.kernel.org Cc: ker...@pengutronix.de Saravana Kannan (2): driver core: fw_devlink: Allow firmware to mark devices as best effort of: base: Avoid console probe delay when fw_devlink.strict=1 drivers/base/core.c| 3 ++- drivers/of/base.c | 2 ++ include/linux/fwnode.h | 4 3 files changed, 8 insertions(+), 1 deletion(-) -- 2.37.0.rc0.161.g10f37bed90-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 7/9] driver core: Set fw_devlink.strict=1 by default
On Wed, Jun 22, 2022 at 12:40 PM Saravana Kannan wrote: > > On Wed, Jun 22, 2022 at 1:44 AM Linus Walleij > wrote: > > > > On Wed, Jun 22, 2022 at 9:48 AM Sascha Hauer wrote: > > > > > This patch has the effect that console UART devices which have "dmas" > > > properties specified in the device tree get deferred for 10 to 20 > > > seconds. This happens on i.MX and likely on other SoCs as well. On i.MX > > > the dma channel is only requested at UART startup time and not at probe > > > time. dma is not used for the console. Nevertheless with this driver probe > > > defers until the dma engine driver is available. > > FYI, if most of the drivers are built in, you could set > deferred_probe_timeout=1 to reduce the impact of this (should drop > down to 1 to 2 seconds). Is that an option until we figure out > something better? > > Actually, why isn't earlyconsole being used? That doesn't get blocked > on anything and the main point of that is to have console working from > really early on. > > > > > > > It shouldn't go in as-is. > > > > This affects all machines with the PL011 UART and DMAs specified as > > well. > > > > It would be best if the console subsystem could be treated special and > > not require DMA devlink to be satisfied before probing. > > If we can mark the console devices somehow before their drivers probe > them, I can make fw_devlink give them special treatment. Is there any > way I could identify them before their drivers probe? > > > It seems devlink is not quite aware of the concept of resources that are > > necessary to probe vs resources that are nice to have and might be > > added after probe. > > Correct, it can't tell them apart. Which is why it tries its best to > enforce them, get most of them ordered properly and then gives up > enforcing the rest after deferred_probe_timeout= expires. There's a > bit more nuance than what I explained here (explained in earlier > commit texts, LPC talks), but that's the gist of it. That's what's > going on in this case Sascha is pointing out.z > > > We need a strong devlink for the first category > > and maybe a weak devlink for the latter category. > > > > I don't know if this is a generic hardware property for all operating > > systems so it could be a DT property such as dma-weak-dependency? > > > > Or maybe compromize and add a linux,dma-weak-dependency; > > property? > > The linux,dma-weak-dependency might be an option, but then if the > kernel version changes and we want to enforce it because we now have a > dma driver (not related to Shasha's example) support, then the > fw_devlink still can't enforce it because of that property. But maybe > that's okay? The consumer can try to use dma and defer probe if it > fails? > > Another option is to mark console devices in DT with some property and > we can give special treatment for those without waiting for > deferred_probe_timeout= to expire. Heh, looks like there's already a property for that: stdout-path. Let me send a series that'll use that to give special treatment to console devices. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 7/9] driver core: Set fw_devlink.strict=1 by default
On Wed, Jun 22, 2022 at 1:44 AM Linus Walleij wrote: > > On Wed, Jun 22, 2022 at 9:48 AM Sascha Hauer wrote: > > > This patch has the effect that console UART devices which have "dmas" > > properties specified in the device tree get deferred for 10 to 20 > > seconds. This happens on i.MX and likely on other SoCs as well. On i.MX > > the dma channel is only requested at UART startup time and not at probe > > time. dma is not used for the console. Nevertheless with this driver probe > > defers until the dma engine driver is available. FYI, if most of the drivers are built in, you could set deferred_probe_timeout=1 to reduce the impact of this (should drop down to 1 to 2 seconds). Is that an option until we figure out something better? Actually, why isn't earlyconsole being used? That doesn't get blocked on anything and the main point of that is to have console working from really early on. > > > > It shouldn't go in as-is. > > This affects all machines with the PL011 UART and DMAs specified as > well. > > It would be best if the console subsystem could be treated special and > not require DMA devlink to be satisfied before probing. If we can mark the console devices somehow before their drivers probe them, I can make fw_devlink give them special treatment. Is there any way I could identify them before their drivers probe? > It seems devlink is not quite aware of the concept of resources that are > necessary to probe vs resources that are nice to have and might be > added after probe. Correct, it can't tell them apart. Which is why it tries its best to enforce them, get most of them ordered properly and then gives up enforcing the rest after deferred_probe_timeout= expires. There's a bit more nuance than what I explained here (explained in earlier commit texts, LPC talks), but that's the gist of it. That's what's going on in this case Sascha is pointing out.z > We need a strong devlink for the first category > and maybe a weak devlink for the latter category. > > I don't know if this is a generic hardware property for all operating > systems so it could be a DT property such as dma-weak-dependency? > > Or maybe compromize and add a linux,dma-weak-dependency; > property? The linux,dma-weak-dependency might be an option, but then if the kernel version changes and we want to enforce it because we now have a dma driver (not related to Shasha's example) support, then the fw_devlink still can't enforce it because of that property. But maybe that's okay? The consumer can try to use dma and defer probe if it fails? Another option is to mark console devices in DT with some property and we can give special treatment for those without waiting for deferred_probe_timeout= to expire. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Tue, Jun 21, 2022 at 9:59 PM Tony Lindgren wrote:
>
> Hi,
>
> * Saravana Kannan [220621 19:29]:
> > On Tue, Jun 21, 2022 at 12:28 AM Tony Lindgren wrote:
> > >
> > > Hi,
> > >
> > > * Saravana Kannan [700101 02:00]:
> > > > Now that fw_devlink=on by default and fw_devlink supports
> > > > "power-domains" property, the execution will never get to the point
> > > > where driver_deferred_probe_check_state() is called before the supplier
> > > > has probed successfully or before deferred probe timeout has expired.
> > > >
> > > > So, delete the call and replace it with -ENODEV.
> > >
> > > Looks like this causes omaps to not boot in Linux next.
> >
> > Can you please point me to an example DTS I could use for debugging
> > this? I'm assuming you are leaving fw_devlink=on and not turning it
> > off or putting it in permissive mode.
>
> Sure, this seems to happen at least with simple-pm-bus as the top
> level interconnect with a configured power-domains property:
>
> $ git grep -A10 "ocp {" arch/arm/boot/dts/*.dtsi | grep -B3 -A4 simple-pm-bus
Thanks for the example. I generally start looking from dts (not dtsi)
files in case there are some DT property override/additions after the
dtsi files are included in the dts file. But I'll assume for now
that's not the case. If there's a specific dts file for a board I can
look from that'd be helpful to rule out those kinds of issues.
For now, I looked at arch/arm/boot/dts/omap4.dtsi.
>
> This issue is no directly related fw_devlink. It is a side effect of
> removing driver_deferred_probe_check_state(). We no longer return
> -EPROBE_DEFER at the end of driver_deferred_probe_check_state().
Yes, I understand the issue. But driver_deferred_probe_check_state()
was deleted because fw_devlink=on should have short circuited the
probe attempt with an -EPROBE_DEFER before reaching the bus/driver
probe function and hitting this -ENOENT failure. That's why I was
asking the other questions.
> > > On platform_probe() genpd_get_from_provider() returns
> > > -ENOENT.
> >
> > This error is with the series I assume?
>
> On the first probe genpd_get_from_provider() will return -ENOENT in
> both cases. The list is empty on the first probe and there are no
> genpd providers at this point.
>
> Earlier with driver_deferred_probe_check_state(), the initial -ENOENT
> ends up getting changed to -EPROBE_DEFER at the end of
> driver_deferred_probe_check_state(), we are now missing that.
Right, I was aware -ENOENT would be returned if we got this far. But
the point of this series is that you shouldn't have gotten that far
before your pm domain device is ready. Hence my questions from the
earlier reply.
Can I get answers to rest of my questions in the first reply please?
That should help us figure out why fw_devlink let us get this far.
Summarize them here to make it easy:
* Are you running with fw_devlink=on?
* Is the"ti,omap4-prm-inst"/"ti,omap-prm-inst" built-in in this case?
* If it's not built-in, can you please try deferred_probe_timeout=0
and deferred_probe_timeout=30 and see if either one of them help?
* Can I get the output of "ls -d supplier:*" and "cat
supplier:*/status" output from the sysfs dir for the ocp device
without this series where it boots properly.
Thanks,
Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Tue, Jun 21, 2022 at 12:28 AM Tony Lindgren wrote: > > Hi, > > * Saravana Kannan [700101 02:00]: > > Now that fw_devlink=on by default and fw_devlink supports > > "power-domains" property, the execution will never get to the point > > where driver_deferred_probe_check_state() is called before the supplier > > has probed successfully or before deferred probe timeout has expired. > > > > So, delete the call and replace it with -ENODEV. > > Looks like this causes omaps to not boot in Linux next. Can you please point me to an example DTS I could use for debugging this? I'm assuming you are leaving fw_devlink=on and not turning it off or putting it in permissive mode. > With this > simple-pm-bus fails to probe initially as the power-domain is not > yet available. Before we get to late_initcall(), I'd expect this series to not have any impact because both fw_devlink and driver_deferred_probe_check_state() should be causing the device's probe to get deferred until the PM domain device comes up. To double check this, without this series, can you give me the list of "supplier:*" symlinks under a simple-pm-bus device's sysfs folder that's having problems with this series? And for all those symlinks, cat the "status" file under that directory? In the system where this is failing, is the PM domain driver loaded as a module at a later point? Couple of other things to try (with the patches) to narrow this down: * Can you set driver_probe_timeout=0 in the command line and see if that helps? * Can you set it to something high like 30 or even larger and see if it helps? > On platform_probe() genpd_get_from_provider() returns > -ENOENT. This error is with the series I assume? > Seems like other stuff is potentially broken too, any ideas on > how to fix this? I'll want to understand the issue first. It's not yet clear to me why fw_devlink isn't blocking the probe of the simple-pm-bus device until the PM domain device shows up. And if it is not blocking, then why and at what point in boot it's giving up and letting the probe get to this point where there's an error. -Saravana > > > > > > Signed-off-by: Saravana Kannan > > --- > > drivers/base/power/domain.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > > index 739e52cd4aba..3e86772d5fac 100644 > > --- a/drivers/base/power/domain.c > > +++ b/drivers/base/power/domain.c > > @@ -2730,7 +2730,7 @@ static int __genpd_dev_pm_attach(struct device *dev, > > struct device *base_dev, > > mutex_unlock(_list_lock); > > dev_dbg(dev, "%s() failed to find PM domain: %ld\n", > > __func__, PTR_ERR(pd)); > > - return driver_deferred_probe_check_state(base_dev); > > + return -ENODEV; > > } > > > > dev_dbg(dev, "adding to PM domain %s\n", pd->name); > > -- > > 2.36.1.255.ge46751e96f-goog > > > > -- > To unsubscribe from this group and stop receiving emails from it, send an > email to kernel-team+unsubscr...@android.com. > ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
On Thu, Jun 9, 2022 at 4:45 AM Ulf Hansson wrote: > > On Wed, 1 Jun 2022 at 09:07, Saravana Kannan wrote: > > > > Now that fw_devlink=on by default and fw_devlink supports > > "power-domains" property, the execution will never get to the point > > where driver_deferred_probe_check_state() is called before the supplier > > has probed successfully or before deferred probe timeout has expired. > > > > So, delete the call and replace it with -ENODEV. > > With fw_devlink=on by default - does that mean that the parameter > can't be changed? > > Or perhaps the point is that we don't want to go back, but rather drop > the fw_devlink parameter altogether when moving forward? Good question. For now, keeping fw_devlink=off and fw_devlink=permissive as debugging options that I can ask people to try if some probe is getting blocked. Or maybe if some ultra low memory use case wants to avoid create device links, fwnode links, etc and can build everything in and have init/probe happen in the right order. But in the long run, I see a strong possibility for fw_devlink=off/permissive being removed. I'd still want to keep it for implementing =rpm where it'd also automatically enable PM runtime tracking, but I don't understand that well enough yet to do it by default. > > > > Signed-off-by: Saravana Kannan > > Just a minor nitpick below. Nevertheless, feel free to add: > > Reviewed-by: Ulf Hansson Thanks! > > > --- > > drivers/base/power/domain.c | 2 +- > > 1 file changed, 1 insertion(+), 1 deletion(-) > > > > diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c > > index 739e52cd4aba..3e86772d5fac 100644 > > --- a/drivers/base/power/domain.c > > +++ b/drivers/base/power/domain.c > > @@ -2730,7 +2730,7 @@ static int __genpd_dev_pm_attach(struct device *dev, > > struct device *base_dev, > > mutex_unlock(_list_lock); > > dev_dbg(dev, "%s() failed to find PM domain: %ld\n", > > __func__, PTR_ERR(pd)); > > - return driver_deferred_probe_check_state(base_dev); > > Adding a brief comment about why -EPROBE_DEFER doesn't make sense > here, would be nice. Will do once all the reviews comeout/when I send v3. I'm thinking something like: /* fw_devlink will take care of retrying for missing suppliers */ -Saravana > > > + return -ENODEV; > > } > > > > dev_dbg(dev, "adding to PM domain %s\n", pd->name); > > -- > > 2.36.1.255.ge46751e96f-goog > > > > Kind regards > Uffe ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/9] deferred_probe_timeout logic clean up
On Wed, Jun 8, 2022 at 3:49 PM Jakub Kicinski wrote:
>
> On Wed, 8 Jun 2022 14:07:44 -0700 Saravana Kannan wrote:
> > David/Jakub,
> >
> > Do the IP4 autoconfig changes look reasonable to you?
>
> I'm no expert in this area, I'd trust the opinion of the embedded folks
> (adding Florian as well) more than myself.
Thanks.
> It's unclear to me why we'd
> wait_for_init_devices_probe() after the first failed iteration,
wait_for_init_devices_probe() relaxes ordering rules for all devices
and it's not something we want to do unless we really need it. That's
why we are doing that only if we can't find any network device in the
first iteration.
> sleep,
> and then allow 11 more iterations with wait_for_device_probe().
> Let me also add Thomas since he wrote e2ffe3ff6f5e ("net: ipconfig:
> Wait for deferred device probes").
Even without this change, I'm not sure the wait_for_device_probe()
needs to be within the loop. It's probably sufficient to just do it
once in the beginning, but it's already there and I'm not sure if I'm
missing some scenarios, so I left that part as is.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/9] deferred_probe_timeout logic clean up
On Wed, Jun 8, 2022 at 11:54 AM Geert Uytterhoeven wrote:
>
> Hi Saravana,
>
> On Wed, Jun 8, 2022 at 8:13 PM Saravana Kannan wrote:
> > On Wed, Jun 8, 2022 at 3:26 AM Geert Uytterhoeven
> > wrote:
> > > On Wed, Jun 8, 2022 at 6:17 AM Saravana Kannan
> > > wrote:
> > > > On Tue, Jun 7, 2022 at 5:55 PM Saravana Kannan
> > > > wrote:
> > > > > On Tue, Jun 7, 2022 at 11:13 AM Geert Uytterhoeven
> > > > > wrote:
> > > > > > On Wed, Jun 1, 2022 at 12:46 PM Saravana Kannan
> > > > > > wrote:
> > > > > > > This series is based on linux-next + these 2 small patches
> > > > > > > applies on top:
> > > > > > > https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/
> > > > > > >
> > > > > > > A lot of the deferred_probe_timeout logic is redundant with
> > > > > > > fw_devlink=on. Also, enabling deferred_probe_timeout by default
> > > > > > > breaks
> > > > > > > a few cases.
> > > > > > >
> > > > > > > This series tries to delete the redundant logic, simplify the
> > > > > > > frameworks
> > > > > > > that use driver_deferred_probe_check_state(), enable
> > > > > > > deferred_probe_timeout=10 by default, and fixes the nfsroot
> > > > > > > failure
> > > > > > > case.
> > > > > > >
> > > > > > > The overall idea of this series is to replace the global behavior
> > > > > > > of
> > > > > > > driver_deferred_probe_check_state() where all devices give up
> > > > > > > waiting on
> > > > > > > supplier at the same time with a more granular behavior:
> > > > > > >
> > > > > > > 1. Devices with all their suppliers successfully probed by
> > > > > > > late_initcall
> > > > > > >probe as usual and avoid unnecessary deferred probe attempts.
> > > > > > >
> > > > > > > 2. At or after late_initcall, in cases where boot would break
> > > > > > > because of
> > > > > > >fw_devlink=on being strict about the ordering, we
> > > > > > >
> > > > > > >a. Temporarily relax the enforcement to probe any unprobed
> > > > > > > devices
> > > > > > > that can probe successfully in the current state of the
> > > > > > > system.
> > > > > > > For example, when we boot with a NFS rootfs and no network
> > > > > > > device
> > > > > > > has probed.
> > > > > > >b. Go back to enforcing the ordering for any devices that
> > > > > > > haven't
> > > > > > > probed.
> > > > > > >
> > > > > > > 3. After deferred probe timeout expires, we permanently give up
> > > > > > > waiting
> > > > > > >on supplier devices without drivers. At this point, whatever
> > > > > > > devices
> > > > > > >can probe without some of their optional suppliers end up
> > > > > > > probing.
> > > > > > >
> > > > > > > In the case where module support is disabled, it's fairly
> > > > > > > straightforward and all device probes are completed before the
> > > > > > > initcalls
> > > > > > > are done.
> > > > > > >
> > > > > > > Patches 1 to 3 are fairly straightforward and can probably be
> > > > > > > applied
> > > > > > > right away.
> > > > > > >
> > > > > > > Patches 4 to 6 are for fixing the NFS rootfs issue and setting the
> > > > > > > default deferred_probe_timeout back to 10 seconds when modules are
> > > > > > > enabled.
> > > > > > >
> > > > > > > Patches 7 to 9 are further clean up of the deferred_probe_timeout
> > > > > > > logic
> > > > > > > so that no framework has to know/care about
> > > > > > > deferred_probe_timeout.
> > > > > > >
> > > > > > > Yoshihiro/Geert,
> > > > > > >
> > > > > > > If you can test this patch series and confirm that the NFS root
> > > > > > > case
> > > > > > > works, I'd really appreciate that.
> > > > > >
> > > > > > Thanks, I gave this a try on various boards I have access to.
> > > > > > The results were quite positive. E.g. the compile error I saw on v1
> > > > > > (implicit declation of fw_devlink_unblock_may_probe(), which is no
> > > > > > longer
> > > > > > used in v2) is gone.
> > > > >
> > > > > Thanks a lot for testing these.
> > > > >
> > > > > > However, I'm seeing a weird error when userspace (Debian9 nfsroot)
> > > > > > is
> > > > > > starting:
>
> > > Setting fw_devlink_strict to true vs. false seems to influence which of
> > > two different failures will happen:
> > > - rcar-csi2: probe of feaa.csi2 failed with error -22
> > > - rcar-vin: probe of e6ef5000.video failed with error -22
> > > The former causes the NULL pointer dereference later.
> > > The latter existed before, but I hadn't noticed it, and bisection
> > > led to the real culprit (commit 3e52419ec04f9769 ("media: rcar-{csi2,vin}:
> > > Move to full Virtual Channel routing per CSI-2 IP").
> >
> > If you revert that patch, does this series work fine? If yes, are you
> > happy with giving this a Tested-by?
>
> Sure, sorry for forgetting that ;-)
>
> Tested-by: Geert Uytterhoeven
+few folks who I forgot to add.
Geert,
Thanks for the extensive testing!
Linus W, Ulf, Kevin, Will, Rob, Vladimir,
Can I get your reviews for the deletion of
Re: [PATCH v2 0/9] deferred_probe_timeout logic clean up
On Wed, Jun 8, 2022 at 3:26 AM Geert Uytterhoeven wrote: > > Hi Saravana, > > On Wed, Jun 8, 2022 at 6:17 AM Saravana Kannan wrote: > > On Tue, Jun 7, 2022 at 5:55 PM Saravana Kannan wrote: > > > On Tue, Jun 7, 2022 at 11:13 AM Geert Uytterhoeven > > > wrote: > > > > On Wed, Jun 1, 2022 at 12:46 PM Saravana Kannan > > > > wrote: > > > > > This series is based on linux-next + these 2 small patches applies on > > > > > top: > > > > > https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/ > > > > > > > > > > A lot of the deferred_probe_timeout logic is redundant with > > > > > fw_devlink=on. Also, enabling deferred_probe_timeout by default > > > > > breaks > > > > > a few cases. > > > > > > > > > > This series tries to delete the redundant logic, simplify the > > > > > frameworks > > > > > that use driver_deferred_probe_check_state(), enable > > > > > deferred_probe_timeout=10 by default, and fixes the nfsroot failure > > > > > case. > > > > > > > > > > The overall idea of this series is to replace the global behavior of > > > > > driver_deferred_probe_check_state() where all devices give up waiting > > > > > on > > > > > supplier at the same time with a more granular behavior: > > > > > > > > > > 1. Devices with all their suppliers successfully probed by > > > > > late_initcall > > > > >probe as usual and avoid unnecessary deferred probe attempts. > > > > > > > > > > 2. At or after late_initcall, in cases where boot would break because > > > > > of > > > > >fw_devlink=on being strict about the ordering, we > > > > > > > > > >a. Temporarily relax the enforcement to probe any unprobed devices > > > > > that can probe successfully in the current state of the system. > > > > > For example, when we boot with a NFS rootfs and no network > > > > > device > > > > > has probed. > > > > >b. Go back to enforcing the ordering for any devices that haven't > > > > > probed. > > > > > > > > > > 3. After deferred probe timeout expires, we permanently give up > > > > > waiting > > > > >on supplier devices without drivers. At this point, whatever > > > > > devices > > > > >can probe without some of their optional suppliers end up probing. > > > > > > > > > > In the case where module support is disabled, it's fairly > > > > > straightforward and all device probes are completed before the > > > > > initcalls > > > > > are done. > > > > > > > > > > Patches 1 to 3 are fairly straightforward and can probably be applied > > > > > right away. > > > > > > > > > > Patches 4 to 6 are for fixing the NFS rootfs issue and setting the > > > > > default deferred_probe_timeout back to 10 seconds when modules are > > > > > enabled. > > > > > > > > > > Patches 7 to 9 are further clean up of the deferred_probe_timeout > > > > > logic > > > > > so that no framework has to know/care about deferred_probe_timeout. > > > > > > > > > > Yoshihiro/Geert, > > > > > > > > > > If you can test this patch series and confirm that the NFS root case > > > > > works, I'd really appreciate that. > > > > > > > > Thanks, I gave this a try on various boards I have access to. > > > > The results were quite positive. E.g. the compile error I saw on v1 > > > > (implicit declation of fw_devlink_unblock_may_probe(), which is no > > > > longer > > > > used in v2) is gone. > > > > > > Thanks a lot for testing these. > > > > > > > However, I'm seeing a weird error when userspace (Debian9 nfsroot) is > > > > starting: > > > > > > > > [ OK ] Started D-Bus System Message Bus. > > > > Unable to handle kernel NULL pointer dereference at virtual > > > > address > > > > Unable to handle kernel NULL pointer dereference at virtual > > > > address > > > > Mem abort info: > > > > ESR = 0x9604 > > > > Mem abort info: > > > > ESR = 0x9604 > > > > EC = 0x25: DABT (current EL), IL = 32 bits > > > > SET = 0, FnV = 0 > > > > EC = 0x25: DABT (current EL), IL = 32 bits > > > > EA = 0, S1PTW = 0 > > > > FSC = 0x04: level 0 translation fault > > > > SET = 0, FnV = 0 > > > > Data abort info: > > > > ISV = 0, ISS = 0x0004 > > > > EA = 0, S1PTW = 0 > > > > FSC = 0x04: level 0 translation fault > > > > CM = 0, WnR = 0 > > > > user pgtable: 4k pages, 48-bit VAs, pgdp=4ec45000 > > > > [] pgd=, p4d= > > > > Data abort info: > > > > Internal error: Oops: 9604 [#1] PREEMPT SMP > > > > CPU: 0 PID: 374 Comm: v4l_id Tainted: GW > > > > 5.19.0-rc1-arm64-renesas-00799-gc13c3e49e8bd #1660 > > > > ISV = 0, ISS = 0x0004 > > > > Hardware name: Renesas Ebisu-4D board based on r8a77990 (DT) > > > > pstate: 6005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) > > > > CM = 0, WnR = 0 > > > > pc : subdev_open+0x8c/0x128 > > > > lr :
Re: [PATCH v2 0/9] deferred_probe_timeout logic clean up
On Tue, Jun 7, 2022 at 5:55 PM Saravana Kannan wrote: > > -Hideaki -- their email keeps bouncing. > > On Tue, Jun 7, 2022 at 11:13 AM Geert Uytterhoeven > wrote: > > > > Hi Saravana, > > > > On Wed, Jun 1, 2022 at 12:46 PM Saravana Kannan > > wrote: > > > This series is based on linux-next + these 2 small patches applies on top: > > > https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/ > > > > > > A lot of the deferred_probe_timeout logic is redundant with > > > fw_devlink=on. Also, enabling deferred_probe_timeout by default breaks > > > a few cases. > > > > > > This series tries to delete the redundant logic, simplify the frameworks > > > that use driver_deferred_probe_check_state(), enable > > > deferred_probe_timeout=10 by default, and fixes the nfsroot failure > > > case. > > > > > > The overall idea of this series is to replace the global behavior of > > > driver_deferred_probe_check_state() where all devices give up waiting on > > > supplier at the same time with a more granular behavior: > > > > > > 1. Devices with all their suppliers successfully probed by late_initcall > > >probe as usual and avoid unnecessary deferred probe attempts. > > > > > > 2. At or after late_initcall, in cases where boot would break because of > > >fw_devlink=on being strict about the ordering, we > > > > > >a. Temporarily relax the enforcement to probe any unprobed devices > > > that can probe successfully in the current state of the system. > > > For example, when we boot with a NFS rootfs and no network device > > > has probed. > > >b. Go back to enforcing the ordering for any devices that haven't > > > probed. > > > > > > 3. After deferred probe timeout expires, we permanently give up waiting > > >on supplier devices without drivers. At this point, whatever devices > > >can probe without some of their optional suppliers end up probing. > > > > > > In the case where module support is disabled, it's fairly > > > straightforward and all device probes are completed before the initcalls > > > are done. > > > > > > Patches 1 to 3 are fairly straightforward and can probably be applied > > > right away. > > > > > > Patches 4 to 6 are for fixing the NFS rootfs issue and setting the > > > default deferred_probe_timeout back to 10 seconds when modules are > > > enabled. > > > > > > Patches 7 to 9 are further clean up of the deferred_probe_timeout logic > > > so that no framework has to know/care about deferred_probe_timeout. > > > > > > Yoshihiro/Geert, > > > > > > If you can test this patch series and confirm that the NFS root case > > > works, I'd really appreciate that. > > > > Thanks, I gave this a try on various boards I have access to. > > The results were quite positive. E.g. the compile error I saw on v1 > > (implicit declation of fw_devlink_unblock_may_probe(), which is no longer > > used in v2) is gone. > > Thanks a lot for testing these. > > > However, I'm seeing a weird error when userspace (Debian9 nfsroot) is > > starting: > > > > [ OK ] Started D-Bus System Message Bus. > > Unable to handle kernel NULL pointer dereference at virtual > > address > > Unable to handle kernel NULL pointer dereference at virtual > > address > > Mem abort info: > > ESR = 0x9604 > > Mem abort info: > > ESR = 0x9604 > > EC = 0x25: DABT (current EL), IL = 32 bits > > SET = 0, FnV = 0 > > EC = 0x25: DABT (current EL), IL = 32 bits > > EA = 0, S1PTW = 0 > > FSC = 0x04: level 0 translation fault > > SET = 0, FnV = 0 > > Data abort info: > > ISV = 0, ISS = 0x0004 > > EA = 0, S1PTW = 0 > > FSC = 0x04: level 0 translation fault > > CM = 0, WnR = 0 > > user pgtable: 4k pages, 48-bit VAs, pgdp=4ec45000 > > [] pgd=, p4d= > > Data abort info: > > Internal error: Oops: 9604 [#1] PREEMPT SMP > > CPU: 0 PID: 374 Comm: v4l_id Tainted: GW > > 5.19.0-rc1-arm64-renesas-00799-gc13c3e49e8bd #1660 > > ISV = 0, ISS = 0x0004 > > Hardware name: Renesas Ebisu-4D board based on r8a77990 (DT) > > pstate: 6005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) > > CM = 0, WnR = 0 > > pc : subdev_open+0x8c/0x128 > > lr : subdev_open+0x78/0x128 > > sp : 8aadba60 > > x29: 8aadba60 x28: x27: 8aadbc58 > > x26: 0002 x25: 0b3aaf00 x24: > > x23: 0c331c00 x22: 09aa61b8 x21: 09aa6000 > > x20: 08bae3e8 x19: 0c3fe200 x18: > > x17: 800076945000 x16: 88004000 x15: 8cc6bf550c7c > > x14: 038f x13: 001a x12: 7fba8618 > > x11: 0001 x10: x9 : 89253954 > > x8 : 0b3aaf00 x7 :
Re: [PATCH v2 0/9] deferred_probe_timeout logic clean up
-Hideaki -- their email keeps bouncing. On Tue, Jun 7, 2022 at 11:13 AM Geert Uytterhoeven wrote: > > Hi Saravana, > > On Wed, Jun 1, 2022 at 12:46 PM Saravana Kannan wrote: > > This series is based on linux-next + these 2 small patches applies on top: > > https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/ > > > > A lot of the deferred_probe_timeout logic is redundant with > > fw_devlink=on. Also, enabling deferred_probe_timeout by default breaks > > a few cases. > > > > This series tries to delete the redundant logic, simplify the frameworks > > that use driver_deferred_probe_check_state(), enable > > deferred_probe_timeout=10 by default, and fixes the nfsroot failure > > case. > > > > The overall idea of this series is to replace the global behavior of > > driver_deferred_probe_check_state() where all devices give up waiting on > > supplier at the same time with a more granular behavior: > > > > 1. Devices with all their suppliers successfully probed by late_initcall > >probe as usual and avoid unnecessary deferred probe attempts. > > > > 2. At or after late_initcall, in cases where boot would break because of > >fw_devlink=on being strict about the ordering, we > > > >a. Temporarily relax the enforcement to probe any unprobed devices > > that can probe successfully in the current state of the system. > > For example, when we boot with a NFS rootfs and no network device > > has probed. > >b. Go back to enforcing the ordering for any devices that haven't > > probed. > > > > 3. After deferred probe timeout expires, we permanently give up waiting > >on supplier devices without drivers. At this point, whatever devices > >can probe without some of their optional suppliers end up probing. > > > > In the case where module support is disabled, it's fairly > > straightforward and all device probes are completed before the initcalls > > are done. > > > > Patches 1 to 3 are fairly straightforward and can probably be applied > > right away. > > > > Patches 4 to 6 are for fixing the NFS rootfs issue and setting the > > default deferred_probe_timeout back to 10 seconds when modules are > > enabled. > > > > Patches 7 to 9 are further clean up of the deferred_probe_timeout logic > > so that no framework has to know/care about deferred_probe_timeout. > > > > Yoshihiro/Geert, > > > > If you can test this patch series and confirm that the NFS root case > > works, I'd really appreciate that. > > Thanks, I gave this a try on various boards I have access to. > The results were quite positive. E.g. the compile error I saw on v1 > (implicit declation of fw_devlink_unblock_may_probe(), which is no longer > used in v2) is gone. Thanks a lot for testing these. > However, I'm seeing a weird error when userspace (Debian9 nfsroot) is > starting: > > [ OK ] Started D-Bus System Message Bus. > Unable to handle kernel NULL pointer dereference at virtual > address > Unable to handle kernel NULL pointer dereference at virtual > address > Mem abort info: > ESR = 0x9604 > Mem abort info: > ESR = 0x9604 > EC = 0x25: DABT (current EL), IL = 32 bits > SET = 0, FnV = 0 > EC = 0x25: DABT (current EL), IL = 32 bits > EA = 0, S1PTW = 0 > FSC = 0x04: level 0 translation fault > SET = 0, FnV = 0 > Data abort info: > ISV = 0, ISS = 0x0004 > EA = 0, S1PTW = 0 > FSC = 0x04: level 0 translation fault > CM = 0, WnR = 0 > user pgtable: 4k pages, 48-bit VAs, pgdp=4ec45000 > [] pgd=, p4d= > Data abort info: > Internal error: Oops: 9604 [#1] PREEMPT SMP > CPU: 0 PID: 374 Comm: v4l_id Tainted: GW > 5.19.0-rc1-arm64-renesas-00799-gc13c3e49e8bd #1660 > ISV = 0, ISS = 0x0004 > Hardware name: Renesas Ebisu-4D board based on r8a77990 (DT) > pstate: 6005 (nZCv daif -PAN -UAO -TCO -DIT -SSBS BTYPE=--) > CM = 0, WnR = 0 > pc : subdev_open+0x8c/0x128 > lr : subdev_open+0x78/0x128 > sp : 8aadba60 > x29: 8aadba60 x28: x27: 8aadbc58 > x26: 0002 x25: 0b3aaf00 x24: > x23: 0c331c00 x22: 09aa61b8 x21: 09aa6000 > x20: 08bae3e8 x19: 0c3fe200 x18: > x17: 800076945000 x16: 88004000 x15: 8cc6bf550c7c > x14: 038f x13: 001a x12: 7fba8618 > x11: 0001 x10: x9 : 89253954 > x8 : 0b3aaf00 x7 : 0004 x6 : 001a > x5 : x4 : x3 : 0001 > x2 : 00010001 x1 : x0 : > Call trace: > subdev_open+0x8c/0x128 > v4l2_open+0xa4/0x120 >
Re: [RFC PATCH v1 4/9] Revert "driver core: Set default deferred_probe_timeout back to 0."
On Sat, Jun 4, 2022 at 8:18 PM Saravana Kannan wrote: > > On Mon, May 30, 2022 at 2:13 AM Geert Uytterhoeven > wrote: > > > > Hi Saravana, > > > > On Thu, May 26, 2022 at 10:16 AM Saravana Kannan > > wrote: > > > This reverts commit 11f7e7ef553b6b93ac1aa74a3c2011b9cc8aeb61. > > > > scripts/chdeckpatch.pl says: > > > > WARNING: Unknown commit id > > '11f7e7ef553b6b93ac1aa74a3c2011b9cc8aeb61', maybe rebased or not > > pulled? > > > > I assume this is your local copy of > > https://lore.kernel.org/r/20220526034609.480766-3-sarava...@google.com? > > I somehow missed all your replies and noticed it just now. > > That commit should be based on driver-core-next. My bad, it was indeed a local commit, but it's one I had already sent to LKML. It's one of the 2 patches I asked to pull-in in the cover letter. I'll fix this once I get some tests/reviews on this series or if I need to send out a v3 series before that. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC PATCH v1 0/9] deferred_probe_timeout logic clean up
On Mon, May 30, 2022 at 2:38 AM Geert Uytterhoeven wrote: > > Hi Saravana, > > On Thu, May 26, 2022 at 10:15 AM Saravana Kannan wrote: > > This series is based on linux-next + these 2 small patches applies on top: > > https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/ > > > > A lot of the deferred_probe_timeout logic is redundant with > > fw_devlink=on. Also, enabling deferred_probe_timeout by default breaks > > a few cases. > > > > This series tries to delete the redundant logic, simplify the frameworks > > that use driver_deferred_probe_check_state(), enable > > deferred_probe_timeout=10 by default, and fixes the nfsroot failure > > case. > > > > Patches 1 to 3 are fairly straightforward and can probably be applied > > right away. > > > > Patches 4 to 9 are related and are the complicated bits of this series. > > > > Patch 8 is where someone with more knowledge of the IP auto config code > > can help rewrite the patch to limit the scope of the workaround by > > running the work around only if IP auto config fails the first time > > around. But it's also something that can be optimized in the future > > because it's already limited to the case where IP auto config is enabled > > using the kernel commandline. > > Thanks for your series! > > > Yoshihiro/Geert, > > > > If you can test this patch series and confirm that the NFS root case > > works, I'd really appreciate that. > > On Salvator-XS, Micrel KSZ9031 Gigabit PHY probe is no longer delayed > by 9s after applying the two earlier patches, and the same is true > after applying this series on top. > Tested-by: Geert Uytterhoeven > > I will do testing on more boards, but that may take a while, as we're > in the middle of the merge window. Thanks for testing. I missed your email until now. I sent out a v2 series a few days back and that's a much better solution than v1. If you can test that series instead, that'd be nice. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC PATCH v1 0/9] deferred_probe_timeout logic clean up
On Mon, May 30, 2022 at 1:49 AM Sebastian Andrzej Siewior
wrote:
>
> On 2022-05-26 01:15:39 [-0700], Saravana Kannan wrote:
> > Yoshihiro/Geert,
> Hi Saravana,
>
> > If you can test this patch series and confirm that the NFS root case
> > works, I'd really appreciate that.
>
> The two patches you sent earlier, plus this series, plus
>
> diff --git a/drivers/base/core.c b/drivers/base/core.c
> index 7ff7fbb006431..829d9b1f7403f 100644
> --- a/drivers/base/core.c
> +++ b/drivers/base/core.c
> @@ -1697,8 +1697,6 @@ static int fw_devlink_may_probe(struct device *dev,
> void *data)
> */
> void __init fw_devlink_unblock_may_probe(void)
> {
> - struct device_link *link, *ln;
> -
> if (!fw_devlink_flags || fw_devlink_is_permissive())
> return;
>
> and it compiles + boots without a delay.
Thanks for testing! I missed your reply until now and I ended up
sending out a v2 a few days back. That's a much better series IMO
because it only temporarily ignores dependencies and only when NFS
root mounting might not work. If you can test that too (and it
shouldn't have the useless variable), I'd appreciate it.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC PATCH v1 2/9] pinctrl: devicetree: Delete usage of driver_deferred_probe_check_state()
On Mon, May 30, 2022 at 2:22 AM Geert Uytterhoeven wrote:
>
> Hi Saravana,
>
> Thanks for your patch!
>
> On Thu, May 26, 2022 at 10:16 AM Saravana Kannan wrote:
> > Now that fw_devlink=on by default and fw_devlink supports
> > "pinctrl-[0-8]" property, the execution will never get to the point
>
> 0-9?
>
> oh, it's really 0-8:
>
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl0, "pinctrl-0", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl1, "pinctrl-1", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl2, "pinctrl-2", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl3, "pinctrl-3", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl4, "pinctrl-4", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl5, "pinctrl-5", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl6, "pinctrl-6", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl7, "pinctrl-7", NULL)
> drivers/of/property.c:DEFINE_SIMPLE_PROP(pinctrl8, "pinctrl-8", NULL)
>
> Looks fragile, especially since we now have:
>
> arch/arm64/boot/dts/microchip/sparx5_pcb134_board.dtsi:
> pinctrl-9 = <_9>;
> arch/arm64/boot/dts/microchip/sparx5_pcb134_board.dtsi: pinctrl-10
> = <_10>;
> arch/arm64/boot/dts/microchip/sparx5_pcb134_board.dtsi: pinctrl-11
> = <_11>;
> arch/arm64/boot/dts/microchip/sparx5_pcb134_board.dtsi: pinctrl-12
> = <_pins_i>;
Checking for pinctrl-* and then verifying if * matches %d would be
more complicated and probably more expensive compared to listing
pinctrl-[0-8]. Especially because more than 50% of pinctrl-*
properties in DT files are NOT pinctrl-%d. So back when we merged
this, Rob and I agreed [0-8] was good enough for now and we can add
more if we needed to. Also, when I checked back then, all the
pinctrl-5+ properties ended up pointing to the same suppliers as the
lower numbered ones. So it didn't make a difference.
Ok, I just checked linux-next all the pinctrl-9+ instances and it's
still true that they all point to the same supplier pointed to by
pinctrl-[0-8].
So yeah, it looks fragile, but is not broken and it's more efficient
than looking for pinctrl-%d or adding more pinctrl-xx entries. So,
let's fix it if it actually breaks? Not going to oppose a patch if
anyone wants to make it more complete.
-Saravana
>
> > where driver_deferred_probe_check_state() is called before the supplier
> > has probed successfully or before deferred probe timeout has expired.
> >
> > So, delete the call and replace it with -ENODEV.
> >
> > Signed-off-by: Saravana Kannan
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
> ge...@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like
> that.
> -- Linus Torvalds
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC PATCH v1 4/9] Revert "driver core: Set default deferred_probe_timeout back to 0."
On Mon, May 30, 2022 at 2:13 AM Geert Uytterhoeven wrote:
>
> Hi Saravana,
>
> On Thu, May 26, 2022 at 10:16 AM Saravana Kannan wrote:
> > This reverts commit 11f7e7ef553b6b93ac1aa74a3c2011b9cc8aeb61.
>
> scripts/chdeckpatch.pl says:
>
> WARNING: Unknown commit id
> '11f7e7ef553b6b93ac1aa74a3c2011b9cc8aeb61', maybe rebased or not
> pulled?
>
> I assume this is your local copy of
> https://lore.kernel.org/r/20220526034609.480766-3-sarava...@google.com?
I somehow missed all your replies and noticed it just now.
That commit should be based on driver-core-next.
-Saravana
>
> Gr{oetje,eeting}s,
>
> Geert
>
> --
> Geert Uytterhoeven -- There's lots of Linux beyond ia32 --
> ge...@linux-m68k.org
>
> In personal conversations with technical people, I call myself a hacker. But
> when I'm talking to journalists I just say "programmer" or something like
> that.
> -- Linus Torvalds
>
> --
> To unsubscribe from this group and stop receiving emails from it, send an
> email to kernel-team+unsubscr...@android.com.
>
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 9/9] driver core: Delete driver_deferred_probe_check_state()
The function is no longer used. So delete it.
Signed-off-by: Saravana Kannan
---
drivers/base/dd.c | 30 --
include/linux/device/driver.h | 1 -
2 files changed, 31 deletions(-)
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 335e71d3a618..e600dd2afc35 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -274,42 +274,12 @@ static int __init deferred_probe_timeout_setup(char *str)
}
__setup("deferred_probe_timeout=", deferred_probe_timeout_setup);
-/**
- * driver_deferred_probe_check_state() - Check deferred probe state
- * @dev: device to check
- *
- * Return:
- * * -ENODEV if initcalls have completed and modules are disabled.
- * * -ETIMEDOUT if the deferred probe timeout was set and has expired
- * and modules are enabled.
- * * -EPROBE_DEFER in other cases.
- *
- * Drivers or subsystems can opt-in to calling this function instead of
directly
- * returning -EPROBE_DEFER.
- */
-int driver_deferred_probe_check_state(struct device *dev)
-{
- if (!IS_ENABLED(CONFIG_MODULES) && initcalls_done) {
- dev_warn(dev, "ignoring dependency for device, assuming no
driver\n");
- return -ENODEV;
- }
-
- if (!driver_deferred_probe_timeout && initcalls_done) {
- dev_warn(dev, "deferred probe timeout, ignoring dependency\n");
- return -ETIMEDOUT;
- }
-
- return -EPROBE_DEFER;
-}
-EXPORT_SYMBOL_GPL(driver_deferred_probe_check_state);
-
static void deferred_probe_timeout_work_func(struct work_struct *work)
{
struct device_private *p;
fw_devlink_drivers_done();
- driver_deferred_probe_timeout = 0;
driver_deferred_probe_trigger();
flush_work(_probe_work);
diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
index 2114d65b862f..7acaabde5396 100644
--- a/include/linux/device/driver.h
+++ b/include/linux/device/driver.h
@@ -242,7 +242,6 @@ driver_find_device_by_acpi_dev(struct device_driver *drv,
const void *adev)
extern int driver_deferred_probe_timeout;
void driver_deferred_probe_add(struct device *dev);
-int driver_deferred_probe_check_state(struct device *dev);
void driver_init(void);
/**
--
2.36.1.255.ge46751e96f-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 8/9] iommu/of: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on and fw_devlink.strict=1 by default and fw_devlink supports iommu DT properties, the execution will never get to the point where driver_deferred_probe_check_state() is called before the supplier has probed successfully or before deferred probe timeout has expired. So, delete the call and replace it with -ENODEV. Signed-off-by: Saravana Kannan --- drivers/iommu/of_iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c index 5696314ae69e..41f4eb005219 100644 --- a/drivers/iommu/of_iommu.c +++ b/drivers/iommu/of_iommu.c @@ -40,7 +40,7 @@ static int of_iommu_xlate(struct device *dev, * a proper probe-ordering dependency mechanism in future. */ if (!ops) - return driver_deferred_probe_check_state(dev); + return -ENODEV; if (!try_module_get(ops->owner)) return -ENODEV; -- 2.36.1.255.ge46751e96f-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 7/9] driver core: Set fw_devlink.strict=1 by default
Now that deferred_probe_timeout is non-zero by default, fw_devlink will
never permanently block the probing of devices. It'll try its best to
probe the devices in the right order and then finally let devices probe
even if their suppliers don't have any drivers.
Signed-off-by: Saravana Kannan
---
drivers/base/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 61fdfe99b348..977b379a495b 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1613,7 +1613,7 @@ static int __init fw_devlink_setup(char *arg)
}
early_param("fw_devlink", fw_devlink_setup);
-static bool fw_devlink_strict;
+static bool fw_devlink_strict = true;
static int __init fw_devlink_strict_setup(char *arg)
{
return strtobool(arg, _devlink_strict);
--
2.36.1.255.ge46751e96f-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 6/9] Revert "driver core: Set default deferred_probe_timeout back to 0."
This reverts commit 11f7e7ef553b6b93ac1aa74a3c2011b9cc8aeb61. Let's take another shot at getting deferred_probe_timeout=10 to work. Signed-off-by: Saravana Kannan --- drivers/base/dd.c | 5 + 1 file changed, 5 insertions(+) diff --git a/drivers/base/dd.c b/drivers/base/dd.c index 4a55fbb7e0da..335e71d3a618 100644 --- a/drivers/base/dd.c +++ b/drivers/base/dd.c @@ -256,7 +256,12 @@ static int deferred_devs_show(struct seq_file *s, void *data) } DEFINE_SHOW_ATTRIBUTE(deferred_devs); +#ifdef CONFIG_MODULES +int driver_deferred_probe_timeout = 10; +#else int driver_deferred_probe_timeout; +#endif + EXPORT_SYMBOL_GPL(driver_deferred_probe_timeout); static int __init deferred_probe_timeout_setup(char *str) -- 2.36.1.255.ge46751e96f-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 5/9] net: ipconfig: Relax fw_devlink if we need to mount a network rootfs
If there are network devices that could probe without some of their
suppliers probing and those network devices are needed to mount a
network rootfs, then fw_devlink=on might break that usecase by blocking
the network devices from probing by the time IP auto config starts.
So, if no network devices are available when IP auto config is enabled
and we have a network rootfs, make sure fw_devlink doesn't block the
probing of any device that has a driver and then retry finding a network
device.
Signed-off-by: Saravana Kannan
---
net/ipv4/ipconfig.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/net/ipv4/ipconfig.c b/net/ipv4/ipconfig.c
index 9d41d5d5cd1e..2342debd7066 100644
--- a/net/ipv4/ipconfig.c
+++ b/net/ipv4/ipconfig.c
@@ -1434,6 +1434,7 @@ __be32 __init root_nfs_parse_addr(char *name)
static int __init wait_for_devices(void)
{
int i;
+ bool try_init_devs = true;
for (i = 0; i < DEVICE_WAIT_MAX; i++) {
struct net_device *dev;
@@ -1452,6 +1453,11 @@ static int __init wait_for_devices(void)
rtnl_unlock();
if (found)
return 0;
+ if (try_init_devs &&
+ (ROOT_DEV == Root_NFS || ROOT_DEV == Root_CIFS)) {
+ try_init_devs = false;
+ wait_for_init_devices_probe();
+ }
ssleep(1);
}
return -ENODEV;
--
2.36.1.255.ge46751e96f-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 4/9] driver core: Add wait_for_init_devices_probe helper function
Some devices might need to be probed and bound successfully before the
kernel boot sequence can finish and move on to init/userspace. For
example, a network interface might need to be bound to be able to mount
a NFS rootfs.
With fw_devlink=on by default, some of these devices might be blocked
from probing because they are waiting on a optional supplier that
doesn't have a driver. While fw_devlink will eventually identify such
devices and unblock the probing automatically, it might be too late by
the time it unblocks the probing of devices. For example, the IP4
autoconfig might timeout before fw_devlink unblocks probing of the
network interface.
This function is available to temporarily try and probe all devices that
have a driver even if some of their suppliers haven't been added or
don't have drivers.
The drivers can then decide which of the suppliers are optional vs
mandatory and probe the device if possible. By the time this function
returns, all such "best effort" probes are guaranteed to be completed.
If a device successfully probes in this mode, we delete all fw_devlink
discovered dependencies of that device where the supplier hasn't yet
probed successfully because they have to be optional dependencies.
This also means that some devices that aren't needed for init and could
have waited for their optional supplier to probe (when the supplier's
module is loaded later on) would end up probing prematurely with limited
functionality. So call this function only when boot would fail without
it.
Signed-off-by: Saravana Kannan
---
drivers/base/base.h | 1 +
drivers/base/core.c | 100 --
drivers/base/dd.c | 19 +--
include/linux/device/driver.h | 1 +
4 files changed, 110 insertions(+), 11 deletions(-)
diff --git a/drivers/base/base.h b/drivers/base/base.h
index ab71403d102f..b3a43a164dcd 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -160,6 +160,7 @@ extern int devres_release_all(struct device *dev);
extern void device_block_probing(void);
extern void device_unblock_probing(void);
extern void deferred_probe_extend_timeout(void);
+extern void driver_deferred_probe_trigger(void);
/* /sys/devices directory */
extern struct kset *devices_kset;
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 7cd789c4985d..61fdfe99b348 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -54,6 +54,7 @@ static unsigned int defer_sync_state_count = 1;
static DEFINE_MUTEX(fwnode_link_lock);
static bool fw_devlink_is_permissive(void);
static bool fw_devlink_drv_reg_done;
+static bool fw_devlink_best_effort;
/**
* fwnode_link_add - Create a link between two fwnode_handles.
@@ -965,6 +966,11 @@ static void device_links_missing_supplier(struct device
*dev)
}
}
+static bool dev_is_best_effort(struct device *dev)
+{
+ return fw_devlink_best_effort && dev->can_match;
+}
+
/**
* device_links_check_suppliers - Check presence of supplier drivers.
* @dev: Consumer device.
@@ -984,7 +990,7 @@ static void device_links_missing_supplier(struct device
*dev)
int device_links_check_suppliers(struct device *dev)
{
struct device_link *link;
- int ret = 0;
+ int ret = 0, fwnode_ret = 0;
struct fwnode_handle *sup_fw;
/*
@@ -997,12 +1003,17 @@ int device_links_check_suppliers(struct device *dev)
sup_fw = list_first_entry(>fwnode->suppliers,
struct fwnode_link,
c_hook)->supplier;
- dev_err_probe(dev, -EPROBE_DEFER, "wait for supplier %pfwP\n",
- sup_fw);
- mutex_unlock(_link_lock);
- return -EPROBE_DEFER;
+ if (!dev_is_best_effort(dev)) {
+ fwnode_ret = -EPROBE_DEFER;
+ dev_err_probe(dev, -EPROBE_DEFER,
+ "wait for supplier %pfwP\n", sup_fw);
+ } else {
+ fwnode_ret = -EAGAIN;
+ }
}
mutex_unlock(_link_lock);
+ if (fwnode_ret == -EPROBE_DEFER)
+ return fwnode_ret;
device_links_write_lock();
@@ -1012,6 +1023,14 @@ int device_links_check_suppliers(struct device *dev)
if (link->status != DL_STATE_AVAILABLE &&
!(link->flags & DL_FLAG_SYNC_STATE_ONLY)) {
+
+ if (dev_is_best_effort(dev) &&
+ link->flags & DL_FLAG_INFERRED &&
+ !link->supplier->can_match) {
+ ret = -EAGAIN;
+ continue;
+ }
+
device_links_missing_supplier(dev);
dev_err_probe(dev, -EPROBE_DEFER,
"supplier %s not ready\n",
@@ -1024,7 +1043,8 @@ int
[PATCH v2 3/9] net: mdio: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on by default and fw_devlink supports interrupt
properties, the execution will never get to the point where
driver_deferred_probe_check_state() is called before the supplier has
probed successfully or before deferred probe timeout has expired.
So, delete the call and replace it with -ENODEV.
Signed-off-by: Saravana Kannan
---
drivers/net/mdio/fwnode_mdio.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/net/mdio/fwnode_mdio.c b/drivers/net/mdio/fwnode_mdio.c
index 1c1584fca632..3e79c2c51929 100644
--- a/drivers/net/mdio/fwnode_mdio.c
+++ b/drivers/net/mdio/fwnode_mdio.c
@@ -47,9 +47,7 @@ int fwnode_mdiobus_phy_device_register(struct mii_bus *mdio,
* just fall back to poll mode
*/
if (rc == -EPROBE_DEFER)
- rc = driver_deferred_probe_check_state(>mdio.dev);
- if (rc == -EPROBE_DEFER)
- return rc;
+ rc = -ENODEV;
if (rc > 0) {
phy->irq = rc;
--
2.36.1.255.ge46751e96f-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 2/9] pinctrl: devicetree: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on by default and fw_devlink supports
"pinctrl-[0-8]" property, the execution will never get to the point
where driver_deferred_probe_check_state() is called before the supplier
has probed successfully or before deferred probe timeout has expired.
So, delete the call and replace it with -ENODEV.
Signed-off-by: Saravana Kannan
---
drivers/pinctrl/devicetree.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pinctrl/devicetree.c b/drivers/pinctrl/devicetree.c
index 3fb238714718..ef898ee8ca6b 100644
--- a/drivers/pinctrl/devicetree.c
+++ b/drivers/pinctrl/devicetree.c
@@ -129,7 +129,7 @@ static int dt_to_map_one_config(struct pinctrl *p,
np_pctldev = of_get_next_parent(np_pctldev);
if (!np_pctldev || of_node_is_root(np_pctldev)) {
of_node_put(np_pctldev);
- ret = driver_deferred_probe_check_state(p->dev);
+ ret = -ENODEV;
/* keep deferring if modules are enabled */
if (IS_ENABLED(CONFIG_MODULES) && !allow_default && ret
< 0)
ret = -EPROBE_DEFER;
--
2.36.1.255.ge46751e96f-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 0/9] deferred_probe_timeout logic clean up
This series is based on linux-next + these 2 small patches applies on top: https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/ A lot of the deferred_probe_timeout logic is redundant with fw_devlink=on. Also, enabling deferred_probe_timeout by default breaks a few cases. This series tries to delete the redundant logic, simplify the frameworks that use driver_deferred_probe_check_state(), enable deferred_probe_timeout=10 by default, and fixes the nfsroot failure case. The overall idea of this series is to replace the global behavior of driver_deferred_probe_check_state() where all devices give up waiting on supplier at the same time with a more granular behavior: 1. Devices with all their suppliers successfully probed by late_initcall probe as usual and avoid unnecessary deferred probe attempts. 2. At or after late_initcall, in cases where boot would break because of fw_devlink=on being strict about the ordering, we a. Temporarily relax the enforcement to probe any unprobed devices that can probe successfully in the current state of the system. For example, when we boot with a NFS rootfs and no network device has probed. b. Go back to enforcing the ordering for any devices that haven't probed. 3. After deferred probe timeout expires, we permanently give up waiting on supplier devices without drivers. At this point, whatever devices can probe without some of their optional suppliers end up probing. In the case where module support is disabled, it's fairly straightforward and all device probes are completed before the initcalls are done. Patches 1 to 3 are fairly straightforward and can probably be applied right away. Patches 4 to 6 are for fixing the NFS rootfs issue and setting the default deferred_probe_timeout back to 10 seconds when modules are enabled. Patches 7 to 9 are further clean up of the deferred_probe_timeout logic so that no framework has to know/care about deferred_probe_timeout. Yoshihiro/Geert, If you can test this patch series and confirm that the NFS root case works, I'd really appreciate that. Thanks, Saravana v1 -> v2: Rewrote the NFS rootfs fix to be a lot less destructive on the fw_devlink ordering for devices that don't end up probing during the "best effort" attempt at probing all devices needed for a network rootfs Saravana Kannan (9): PM: domains: Delete usage of driver_deferred_probe_check_state() pinctrl: devicetree: Delete usage of driver_deferred_probe_check_state() net: mdio: Delete usage of driver_deferred_probe_check_state() driver core: Add wait_for_init_devices_probe helper function net: ipconfig: Relax fw_devlink if we need to mount a network rootfs Revert "driver core: Set default deferred_probe_timeout back to 0." driver core: Set fw_devlink.strict=1 by default iommu/of: Delete usage of driver_deferred_probe_check_state() driver core: Delete driver_deferred_probe_check_state() drivers/base/base.h| 1 + drivers/base/core.c| 102 ++--- drivers/base/dd.c | 54 ++--- drivers/base/power/domain.c| 2 +- drivers/iommu/of_iommu.c | 2 +- drivers/net/mdio/fwnode_mdio.c | 4 +- drivers/pinctrl/devicetree.c | 2 +- include/linux/device/driver.h | 2 +- net/ipv4/ipconfig.c| 6 ++ 9 files changed, 126 insertions(+), 49 deletions(-) -- 2.36.1.255.ge46751e96f-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v2 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on by default and fw_devlink supports "power-domains" property, the execution will never get to the point where driver_deferred_probe_check_state() is called before the supplier has probed successfully or before deferred probe timeout has expired. So, delete the call and replace it with -ENODEV. Signed-off-by: Saravana Kannan --- drivers/base/power/domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 739e52cd4aba..3e86772d5fac 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -2730,7 +2730,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, mutex_unlock(_list_lock); dev_dbg(dev, "%s() failed to find PM domain: %ld\n", __func__, PTR_ERR(pd)); - return driver_deferred_probe_check_state(base_dev); + return -ENODEV; } dev_dbg(dev, "adding to PM domain %s\n", pd->name); -- 2.36.1.255.ge46751e96f-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Sun, May 29, 2022 at 1:34 AM 'Niklas Cassel' via kernel-team
wrote:
>
> On Wed, May 25, 2022 at 12:49:00PM -0700, Saravana Kannan wrote:
> > On Wed, May 25, 2022 at 12:12 AM Sebastian Andrzej Siewior
> > wrote:
> > >
> > > On 2022-05-24 10:46:49 [-0700], Saravana Kannan wrote:
> > > > > Removing probe_timeout_waitqueue (as suggested) or setting the timeout
> > > > > to 0 avoids the delay.
> > > >
> > > > In your case, I think it might be working as intended? Curious, what
> > > > was the call stack in your case where it was blocked?
> > >
> > > Why is then there 10sec delay during boot? The backtrace is
> > > |[ cut here ]
> > > |WARNING: CPU: 4 PID: 1 at drivers/base/dd.c:742
> > > wait_for_device_probe+0x30/0x110
> > > |Modules linked in:
> > > |CPU: 4 PID: 1 Comm: swapper/0 Not tainted 5.18.0-rc5+ #154
> > > |RIP: 0010:wait_for_device_probe+0x30/0x110
> > > |Call Trace:
> > > |
> > > | prepare_namespace+0x2b/0x160
> > > | kernel_init_freeable+0x2b3/0x2dd
> > > | kernel_init+0x11/0x110
> > > | ret_from_fork+0x22/0x30
> > > |
> > >
> > > Looking closer, it can't access init. This in particular box boots
> > > directly the kernel without an initramfs so the kernel later mounts
> > > /dev/sda1 and everything is good. So that seems to be the reason…
> >
>
> Hello there,
>
> My (QEMU) boot times were recently extended by 10 seconds.
> Looking at the timestamps, it looks like nothing is being done for 10 whole
> seconds.
>
> A git bisect landed me at the patch in $subject:
> 2b28a1a84a0e ("driver core: Extend deferred probe timeout on driver
> registration")
>
> Adding a WARN_ON(1) in wait_for_device_probe(), as requested by the patch
> author from the others seeing a regression with this patch, gives two
> different
> stacktraces during boot:
Thanks for the report. My patch was exposing an existing issue. I
already sent a fix for that a few days ago. Waiting for it to get
picked up.
-Saravana
>
> [0.459633] printk: console [netcon0] enabled
> [0.459636] printk: console [netcon0] printing thread started
> [0.459637] netconsole: network logging started
> [0.459896] cfg80211: Loading compiled-in X.509 certificates for
> regulatory database
> [0.460230] kworker/u8:6 (105) used greatest stack depth: 14744 bytes left
> [0.461031] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7'
> [0.461077] platform regulatory.0: Direct firmware load for regulatory.db
> failed with error -2
> [0.461085] cfg80211: failed to load regulatory.db
> [0.461113] ALSA device list:
> [0.461116] No soundcards found.
> [0.461614] [ cut here ]
> [0.461615] WARNING: CPU: 2 PID: 1 at drivers/base/dd.c:741
> wait_for_device_probe+0x1a/0x160
> [0.485809] Modules linked in:
> [0.486089] CPU: 2 PID: 1 Comm: swapper/0 Not tainted
> 5.18.0-next-20220526-4-g74f936013b08-dirty #20
> [0.486842] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
> rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014
> [0.487707] RIP: 0010:wait_for_device_probe+0x1a/0x160
> [0.488103] Code: 00 e8 fa e4 b5 ff 8b 44 24 04 48 83 c4 08 5b c3 0f 1f 44
> 00 00 53 48 83 ec 30 65 48 8b 04 25 28 00 00 00 48 89 44 24 28 31 c0 <0f> 0b
> e8 1f ac 57 00 8b 15 f1 b3 24 01 85 d2 75 3d 48 c7 c7 60 2f
> [0.489539] RSP: :9c7900013ed8 EFLAGS: 00010246
> [0.489965] RAX: RBX: 0008 RCX:
> 0d02
> [0.490597] RDX: 0cc2 RSI: RDI:
> 0002e990
> [0.491181] RBP: 0214 R08: 000f R09:
> 0064
> [0.491788] R10: 9c7900013c6c R11: R12:
> 8964c0343640
> [0.492384] R13: 9e51791c R14: R15:
>
> [0.492960] FS: () GS:896637d0()
> knlGS:
> [0.493658] CS: 0010 DS: ES: CR0: 80050033
> [0.494501] CR2: CR3: 0001ed20c001 CR4:
> 00370ee0
> [0.495621] Call Trace:
> [0.496059]
> [0.496266] ? init_eaccess+0x3b/0x76
> [0.496657] prepare_namespace+0x30/0x16a
> [0.497016] kernel_init_freeable+0x207/0x212
> [0.497407] ? rest_init+0xc0/0xc0
> [0.497714] kernel_init+0x16/0x120
> [0.498250] ret_from_fork+0x1f/0x30
> [0.498898]
> [0.499307] ---[ end trace ]---
> [0.748413] ata2: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> [0.749053] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> [0.749461] ata2.00: ATA-7: QEMU HARDDISK, version, max UDMA/100
> [0.749470] ata2.00: 732 sectors, multi 16: LBA48 NCQ (depth 32)
> [0.749479] ata2.00: applying bridge limits
> [0.750915] ata4: SATA link down (SStatus 0 SControl 300)
> [0.752110] ata5: SATA link down (SStatus 0 SControl 300)
> [0.753424] ata6: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
> [
[RFC PATCH v1 9/9] driver core: Delete driver_deferred_probe_check_state()
The function is no longer used. So delete it.
Signed-off-by: Saravana Kannan
---
drivers/base/dd.c | 30 --
include/linux/device/driver.h | 1 -
2 files changed, 31 deletions(-)
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index af8138d44e6c..789b0871dc45 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -274,42 +274,12 @@ static int __init deferred_probe_timeout_setup(char *str)
}
__setup("deferred_probe_timeout=", deferred_probe_timeout_setup);
-/**
- * driver_deferred_probe_check_state() - Check deferred probe state
- * @dev: device to check
- *
- * Return:
- * * -ENODEV if initcalls have completed and modules are disabled.
- * * -ETIMEDOUT if the deferred probe timeout was set and has expired
- * and modules are enabled.
- * * -EPROBE_DEFER in other cases.
- *
- * Drivers or subsystems can opt-in to calling this function instead of
directly
- * returning -EPROBE_DEFER.
- */
-int driver_deferred_probe_check_state(struct device *dev)
-{
- if (!IS_ENABLED(CONFIG_MODULES) && initcalls_done) {
- dev_warn(dev, "ignoring dependency for device, assuming no
driver\n");
- return -ENODEV;
- }
-
- if (!driver_deferred_probe_timeout && initcalls_done) {
- dev_warn(dev, "deferred probe timeout, ignoring dependency\n");
- return -ETIMEDOUT;
- }
-
- return -EPROBE_DEFER;
-}
-EXPORT_SYMBOL_GPL(driver_deferred_probe_check_state);
-
static void deferred_probe_timeout_work_func(struct work_struct *work)
{
struct device_private *p;
fw_devlink_drivers_done();
- driver_deferred_probe_timeout = 0;
driver_deferred_probe_trigger();
flush_work(_probe_work);
diff --git a/include/linux/device/driver.h b/include/linux/device/driver.h
index 700453017e1c..7c245d269feb 100644
--- a/include/linux/device/driver.h
+++ b/include/linux/device/driver.h
@@ -241,7 +241,6 @@ driver_find_device_by_acpi_dev(struct device_driver *drv,
const void *adev)
extern int driver_deferred_probe_timeout;
void driver_deferred_probe_add(struct device *dev);
-int driver_deferred_probe_check_state(struct device *dev);
void driver_init(void);
/**
--
2.36.1.124.g0e6072fb45-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 8/9] net: ipconfig: Force fw_devlink to unblock any devices that might probe
If there are network devices that could probe without some of their
suppliers probing and those network devices are needed for IP auto
config to work, then fw_devlink=on might break that usecase by blocking
the network devices from probing by the time IP auto config starts.
So, when IP auto config is enabled, make sure fw_devlink doesn't block
the probing of any device that has a driver by the time we get to IP
auto config.
Signed-off-by: Saravana Kannan
---
net/ipv4/ipconfig.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/net/ipv4/ipconfig.c b/net/ipv4/ipconfig.c
index 9d41d5d5cd1e..aa7b8ba68ca6 100644
--- a/net/ipv4/ipconfig.c
+++ b/net/ipv4/ipconfig.c
@@ -1435,6 +1435,8 @@ static int __init wait_for_devices(void)
{
int i;
+ fw_devlink_unblock_may_probe();
+
for (i = 0; i < DEVICE_WAIT_MAX; i++) {
struct net_device *dev;
int found = 0;
--
2.36.1.124.g0e6072fb45-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 7/9] driver core: Add fw_devlink_unblock_may_probe() helper function
This function can be used during the kernel boot sequence to forcefully
override fw_devlink=on and unblock the probing of all devices that have
a driver.
It's mainly meant to be called from late_initcall() or
late_initcall_sync() where a device needs to probe before the kernel can
mount rootfs.
Signed-off-by: Saravana Kannan
---
drivers/base/base.h| 1 +
drivers/base/core.c| 58 ++
drivers/base/dd.c | 2 +-
include/linux/fwnode.h | 2 ++
4 files changed, 62 insertions(+), 1 deletion(-)
diff --git a/drivers/base/base.h b/drivers/base/base.h
index ab71403d102f..b3a43a164dcd 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -160,6 +160,7 @@ extern int devres_release_all(struct device *dev);
extern void device_block_probing(void);
extern void device_unblock_probing(void);
extern void deferred_probe_extend_timeout(void);
+extern void driver_deferred_probe_trigger(void);
/* /sys/devices directory */
extern struct kset *devices_kset;
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 7672f23231c1..7ff7fbb00643 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1655,6 +1655,64 @@ void fw_devlink_drivers_done(void)
device_links_write_unlock();
}
+static int fw_devlink_may_probe(struct device *dev, void *data)
+{
+ struct device_link *link = to_devlink(dev);
+
+ if (!link->supplier->can_match && link->consumer->can_match)
+ fw_devlink_relax_link(link);
+
+ return 0;
+}
+
+/**
+ * fw_devlink_unblock_may_probe - Force unblock any device that has a driver
+ *
+ * This function is more of a sledge hammer than a scalpel. Use this very
+ * sparingly.
+ *
+ * Some devices might need to be probed and bound successfully before the
kernel
+ * boot sequence can finish and move on to init/userspace. For example, a
+ * network interface might need to be bound to be able to mount a NFS rootfs.
+ *
+ * With fw_devlink=on by default, some of these devices might be blocked from
+ * probing because they are waiting on a optional supplier that doesn't have a
+ * driver. While fw_devlink will eventually identify such devices and unblock
+ * the probing automatically, it might be too late by the time it unblocks the
+ * probing of devices. For example, the IP4 autoconfig might timeout before
+ * fw_devlink unblocks probing of the network interface. This function is
+ * available to unblock the probing of such devices.
+ *
+ * Since there's no easy way to know which unprobed device needs to probe for
+ * boot to succeed, this function makes sure fw_devlink doesn't block any
device
+ * that has a driver at the point in time this function is called.
+ *
+ * It does this by relaxing (fw_devlink=permissive behavior) all the device
+ * links created by fw_devlink where the consumer has a driver and the supplier
+ * doesn't have a driver.
+ *
+ * It's extremely unlikely that a proper use of this function will be outside
of
+ * an initcall. So, until a case is made for that, this function is
+ * intentionally marked with __init.
+ */
+void __init fw_devlink_unblock_may_probe(void)
+{
+ struct device_link *link, *ln;
+
+ if (!fw_devlink_flags || fw_devlink_is_permissive())
+ return;
+
+ /* Wait for current probes to finish to limit impact. */
+ wait_for_device_probe();
+
+ device_links_write_lock();
+ class_for_each_device(_class, NULL, NULL,
+ fw_devlink_may_probe);
+ device_links_write_unlock();
+
+ driver_deferred_probe_trigger();
+}
+
static void fw_devlink_unblock_consumers(struct device *dev)
{
struct device_link *link;
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index f963d9010d7f..af8138d44e6c 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -172,7 +172,7 @@ static bool driver_deferred_probe_enable;
* changes in the midst of a probe, then deferred processing should be
triggered
* again.
*/
-static void driver_deferred_probe_trigger(void)
+void driver_deferred_probe_trigger(void)
{
if (!driver_deferred_probe_enable)
return;
diff --git a/include/linux/fwnode.h b/include/linux/fwnode.h
index 9a81c4410b9f..0770edda7068 100644
--- a/include/linux/fwnode.h
+++ b/include/linux/fwnode.h
@@ -13,6 +13,7 @@
#include
#include
#include
+#include
struct fwnode_operations;
struct device;
@@ -199,5 +200,6 @@ extern bool fw_devlink_is_strict(void);
int fwnode_link_add(struct fwnode_handle *con, struct fwnode_handle *sup);
void fwnode_links_purge(struct fwnode_handle *fwnode);
void fw_devlink_purge_absent_suppliers(struct fwnode_handle *fwnode);
+void __init fw_devlink_unblock_may_probe(void);
#endif
--
2.36.1.124.g0e6072fb45-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 6/9] iommu/of: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on and fw_devlink.strict=1 by default and fw_devlink supports iommu DT properties, the execution will never get to the point where driver_deferred_probe_check_state() is called before the supplier has probed successfully or before deferred probe timeout has expired. So, delete the call and replace it with -ENODEV. Signed-off-by: Saravana Kannan --- drivers/iommu/of_iommu.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c index 5696314ae69e..41f4eb005219 100644 --- a/drivers/iommu/of_iommu.c +++ b/drivers/iommu/of_iommu.c @@ -40,7 +40,7 @@ static int of_iommu_xlate(struct device *dev, * a proper probe-ordering dependency mechanism in future. */ if (!ops) - return driver_deferred_probe_check_state(dev); + return -ENODEV; if (!try_module_get(ops->owner)) return -ENODEV; -- 2.36.1.124.g0e6072fb45-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 5/9] driver core: Set fw_devlink.strict=1 by default
Now that deferred_probe_timeout is non-zero by default, fw_devlink will
never permanently block the probing of devices. It'll try its best to
probe the devices in the right order and then finally let devices probe
even if their suppliers don't have any drivers.
Signed-off-by: Saravana Kannan
---
drivers/base/core.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/base/core.c b/drivers/base/core.c
index 7cd789c4985d..7672f23231c1 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -1581,7 +1581,7 @@ static int __init fw_devlink_setup(char *arg)
}
early_param("fw_devlink", fw_devlink_setup);
-static bool fw_devlink_strict;
+static bool fw_devlink_strict = true;
static int __init fw_devlink_strict_setup(char *arg)
{
return strtobool(arg, _devlink_strict);
--
2.36.1.124.g0e6072fb45-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 4/9] Revert "driver core: Set default deferred_probe_timeout back to 0."
This reverts commit 11f7e7ef553b6b93ac1aa74a3c2011b9cc8aeb61. Let's take another shot at getting deferred_probe_timeout=10 to work. Signed-off-by: Saravana Kannan --- drivers/base/dd.c | 5 + 1 file changed, 5 insertions(+) diff --git a/drivers/base/dd.c b/drivers/base/dd.c index 11b0fb6414d3..f963d9010d7f 100644 --- a/drivers/base/dd.c +++ b/drivers/base/dd.c @@ -256,7 +256,12 @@ static int deferred_devs_show(struct seq_file *s, void *data) } DEFINE_SHOW_ATTRIBUTE(deferred_devs); +#ifdef CONFIG_MODULES +int driver_deferred_probe_timeout = 10; +#else int driver_deferred_probe_timeout; +#endif + EXPORT_SYMBOL_GPL(driver_deferred_probe_timeout); static int __init deferred_probe_timeout_setup(char *str) -- 2.36.1.124.g0e6072fb45-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 3/9] net: mdio: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on by default and fw_devlink supports interrupt
properties, the execution will never get to the point where
driver_deferred_probe_check_state() is called before the supplier has
probed successfully or before deferred probe timeout has expired.
So, delete the call and replace it with -ENODEV.
Signed-off-by: Saravana Kannan
---
drivers/net/mdio/fwnode_mdio.c | 4 +---
1 file changed, 1 insertion(+), 3 deletions(-)
diff --git a/drivers/net/mdio/fwnode_mdio.c b/drivers/net/mdio/fwnode_mdio.c
index 1c1584fca632..3e79c2c51929 100644
--- a/drivers/net/mdio/fwnode_mdio.c
+++ b/drivers/net/mdio/fwnode_mdio.c
@@ -47,9 +47,7 @@ int fwnode_mdiobus_phy_device_register(struct mii_bus *mdio,
* just fall back to poll mode
*/
if (rc == -EPROBE_DEFER)
- rc = driver_deferred_probe_check_state(>mdio.dev);
- if (rc == -EPROBE_DEFER)
- return rc;
+ rc = -ENODEV;
if (rc > 0) {
phy->irq = rc;
--
2.36.1.124.g0e6072fb45-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 2/9] pinctrl: devicetree: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on by default and fw_devlink supports
"pinctrl-[0-8]" property, the execution will never get to the point
where driver_deferred_probe_check_state() is called before the supplier
has probed successfully or before deferred probe timeout has expired.
So, delete the call and replace it with -ENODEV.
Signed-off-by: Saravana Kannan
---
drivers/pinctrl/devicetree.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/pinctrl/devicetree.c b/drivers/pinctrl/devicetree.c
index 3fb238714718..ef898ee8ca6b 100644
--- a/drivers/pinctrl/devicetree.c
+++ b/drivers/pinctrl/devicetree.c
@@ -129,7 +129,7 @@ static int dt_to_map_one_config(struct pinctrl *p,
np_pctldev = of_get_next_parent(np_pctldev);
if (!np_pctldev || of_node_is_root(np_pctldev)) {
of_node_put(np_pctldev);
- ret = driver_deferred_probe_check_state(p->dev);
+ ret = -ENODEV;
/* keep deferring if modules are enabled */
if (IS_ENABLED(CONFIG_MODULES) && !allow_default && ret
< 0)
ret = -EPROBE_DEFER;
--
2.36.1.124.g0e6072fb45-goog
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 1/9] PM: domains: Delete usage of driver_deferred_probe_check_state()
Now that fw_devlink=on by default and fw_devlink supports "power-domains" property, the execution will never get to the point where driver_deferred_probe_check_state() is called before the supplier has probed successfully or before deferred probe timeout has expired. So, delete the call and replace it with -ENODEV. Signed-off-by: Saravana Kannan --- drivers/base/power/domain.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c index 739e52cd4aba..3e86772d5fac 100644 --- a/drivers/base/power/domain.c +++ b/drivers/base/power/domain.c @@ -2730,7 +2730,7 @@ static int __genpd_dev_pm_attach(struct device *dev, struct device *base_dev, mutex_unlock(_list_lock); dev_dbg(dev, "%s() failed to find PM domain: %ld\n", __func__, PTR_ERR(pd)); - return driver_deferred_probe_check_state(base_dev); + return -ENODEV; } dev_dbg(dev, "adding to PM domain %s\n", pd->name); -- 2.36.1.124.g0e6072fb45-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RFC PATCH v1 0/9] deferred_probe_timeout logic clean up
This series is based on linux-next + these 2 small patches applies on top: https://lore.kernel.org/lkml/20220526034609.480766-1-sarava...@google.com/ A lot of the deferred_probe_timeout logic is redundant with fw_devlink=on. Also, enabling deferred_probe_timeout by default breaks a few cases. This series tries to delete the redundant logic, simplify the frameworks that use driver_deferred_probe_check_state(), enable deferred_probe_timeout=10 by default, and fixes the nfsroot failure case. Patches 1 to 3 are fairly straightforward and can probably be applied right away. Patches 4 to 9 are related and are the complicated bits of this series. Patch 8 is where someone with more knowledge of the IP auto config code can help rewrite the patch to limit the scope of the workaround by running the work around only if IP auto config fails the first time around. But it's also something that can be optimized in the future because it's already limited to the case where IP auto config is enabled using the kernel commandline. Yoshihiro/Geert, If you can test this patch series and confirm that the NFS root case works, I'd really appreciate that. Cc: Mark Brown Cc: Rob Herring Cc: Geert Uytterhoeven Cc: Yoshihiro Shimoda Cc: John Stultz Cc: Nathan Chancellor Cc: Sebastian Andrzej Siewior Saravana Kannan (9): PM: domains: Delete usage of driver_deferred_probe_check_state() pinctrl: devicetree: Delete usage of driver_deferred_probe_check_state() net: mdio: Delete usage of driver_deferred_probe_check_state() Revert "driver core: Set default deferred_probe_timeout back to 0." driver core: Set fw_devlink.strict=1 by default iommu/of: Delete usage of driver_deferred_probe_check_state() driver core: Add fw_devlink_unblock_may_probe() helper function net: ipconfig: Force fw_devlink to unblock any devices that might probe driver core: Delete driver_deferred_probe_check_state() drivers/base/base.h| 1 + drivers/base/core.c| 60 +- drivers/base/dd.c | 37 - drivers/base/power/domain.c| 2 +- drivers/iommu/of_iommu.c | 2 +- drivers/net/mdio/fwnode_mdio.c | 4 +-- drivers/pinctrl/devicetree.c | 2 +- include/linux/device/driver.h | 1 - include/linux/fwnode.h | 2 ++ net/ipv4/ipconfig.c| 2 ++ 10 files changed, 74 insertions(+), 39 deletions(-) -- 2.36.1.124.g0e6072fb45-goog ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Wed, May 25, 2022 at 12:12 AM Sebastian Andrzej Siewior wrote: > > On 2022-05-24 10:46:49 [-0700], Saravana Kannan wrote: > > > Removing probe_timeout_waitqueue (as suggested) or setting the timeout > > > to 0 avoids the delay. > > > > In your case, I think it might be working as intended? Curious, what > > was the call stack in your case where it was blocked? > > Why is then there 10sec delay during boot? The backtrace is > |[ cut here ] > |WARNING: CPU: 4 PID: 1 at drivers/base/dd.c:742 > wait_for_device_probe+0x30/0x110 > |Modules linked in: > |CPU: 4 PID: 1 Comm: swapper/0 Not tainted 5.18.0-rc5+ #154 > |RIP: 0010:wait_for_device_probe+0x30/0x110 > |Call Trace: > | > | prepare_namespace+0x2b/0x160 > | kernel_init_freeable+0x2b3/0x2dd > | kernel_init+0x11/0x110 > | ret_from_fork+0x22/0x30 > | > > Looking closer, it can't access init. This in particular box boots > directly the kernel without an initramfs so the kernel later mounts > /dev/sda1 and everything is good. So that seems to be the reason… Hmmm... that part shouldn't matter. As long as you are hitting the same code path. My guess is one of them has CONFIG_MODULES enabled and the other doesn't. In either case, I think the patch needs to be reverted (I'll send out one soon), but that'll also mean I need to revert part of my patch (sets the timeout back to 0) or I need to fix this case: https://lore.kernel.org/lkml/tyapr01mb45443df63b9ef29054f7c41fd8...@tyapr01mb4544.jpnprd01.prod.outlook.com/ I'll try to do the latter if I can get something reasonable soon. Otherwise, I'll just do the revert + partial revert. -Saravana > My other machine with an initramfs does not show this problem. > > > -Saravana > > Sebastian ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Tue, May 24, 2022 at 9:41 AM Sebastian Andrzej Siewior wrote: > > On 2022-05-23 20:43:06 [-0700], Saravana Kannan wrote: > … > > Thanks for all the help. I think I know what's going on. > > I, too got here because my boot recently was extended by 10 seconds and > bisected to that commit in question. > > > If you revert the following commit, then you'll see that your device > > no longer hangs with my changes. > > 35a672363ab3 driver core: Ensure wait_for_device_probe() waits until > > the deferred_probe_timeout fires > > Removing probe_timeout_waitqueue (as suggested) or setting the timeout > to 0 avoids the delay. In your case, I think it might be working as intended? Curious, what was the call stack in your case where it was blocked? -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Mon, May 23, 2022 at 3:14 PM Nathan Chancellor wrote:
>
> On Mon, May 23, 2022 at 01:04:03PM -0700, Saravana Kannan wrote:
> > On Mon, May 23, 2022 at 8:17 AM Nathan Chancellor wrote:
> > >
> > > On Fri, May 20, 2022 at 05:15:55PM -0700, Saravana Kannan wrote:
> > > > On Fri, May 20, 2022 at 5:04 PM Nathan Chancellor
> > > > wrote:
> > > > >
> > > > > On Fri, May 20, 2022 at 04:49:48PM -0700, Saravana Kannan wrote:
> > > > > > On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor
> > > > > > wrote:
> > > > > > >
> > > > > > > Hi Saravana,
> > > > > > >
> > > > > > > On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > > > > > > > The deferred probe timer that's used for this currently starts
> > > > > > > > at
> > > > > > > > late_initcall and runs for driver_deferred_probe_timeout
> > > > > > > > seconds. The
> > > > > > > > assumption being that all available drivers would be loaded and
> > > > > > > > registered before the timer expires. This means, the
> > > > > > > > driver_deferred_probe_timeout has to be pretty large for it to
> > > > > > > > cover the
> > > > > > > > worst case. But if we set the default value for it to cover the
> > > > > > > > worst
> > > > > > > > case, it would significantly slow down the average case. For
> > > > > > > > this
> > > > > > > > reason, the default value is set to 0.
> > > > > > > >
> > > > > > > > Also, with CONFIG_MODULES=y and the current default values of
> > > > > > > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with
> > > > > > > > missing
> > > > > > > > drivers will cause their consumer devices to always defer their
> > > > > > > > probes.
> > > > > > > > This is because device links created by fw_devlink defer the
> > > > > > > > probe even
> > > > > > > > before the consumer driver's probe() is called.
> > > > > > > >
> > > > > > > > Instead of a fixed timeout, if we extend an unexpired deferred
> > > > > > > > probe
> > > > > > > > timer on every successful driver registration, with the
> > > > > > > > expectation more
> > > > > > > > modules would be loaded in the near future, then the default
> > > > > > > > value of
> > > > > > > > driver_deferred_probe_timeout only needs to be as long as the
> > > > > > > > worst case
> > > > > > > > time difference between two consecutive module loads.
> > > > > > > >
> > > > > > > > So let's implement that and set the default value to 10 seconds
> > > > > > > > when
> > > > > > > > CONFIG_MODULES=y.
> > > > > > > >
> > > > > > > > Cc: Greg Kroah-Hartman
> > > > > > > > Cc: "Rafael J. Wysocki"
> > > > > > > > Cc: Rob Herring
> > > > > > > > Cc: Linus Walleij
> > > > > > > > Cc: Will Deacon
> > > > > > > > Cc: Ulf Hansson
> > > > > > > > Cc: Kevin Hilman
> > > > > > > > Cc: Thierry Reding
> > > > > > > > Cc: Mark Brown
> > > > > > > > Cc: Pavel Machek
> > > > > > > > Cc: Geert Uytterhoeven
> > > > > > > > Cc: Yoshihiro Shimoda
> > > > > > > > Cc: Paul Kocialkowski
> > > > > > > > Cc: linux-g...@vger.kernel.org
> > > > > > > > Cc: linux...@vger.kernel.org
> > > > > > > > Cc: iommu@lists.linux-foundation.org
> > > > > > > > Signed-off-by: Saravana Kannan
> > > > > > >
> > > > > > > I bisected a boot hang with ARCH=s390 defconfig in QEMU down to
> > > > > > > this
> > > > > > > change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> > > > > > > timeout on driver registration") in next-20220520 (bisect log
> > > > > > > below).
> > > > > > >
> > > > > > > $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu-
> > > > > > > defconfig bzImage
> > > > > > >
> > > > > > > $ timeout --foreground 15m stdbuf -oL -eL \
> > > > > > > qemu-system-s390x \
> > > > > > > -initrd ... \
> > > > > > > -M s390-ccw-virtio \
> > > > > > > -display none \
> > > > > > > -kernel arch/s390/boot/bzImage \
> > > > > > > -m 512m \
> > > > > > > -nodefaults \
> > > > > > > -serial mon:stdio
> > > > > > > ...
> > > > > > > [2.077303] In-situ OAM (IOAM) with IPv6
> > > > > > > [2.077639] NET: Registered PF_PACKET protocol family
> > > > > > > [2.078063] bridge: filtering via arp/ip/ip6tables is no
> > > > > > > longer available by default. Update your scripts to load
> > > > > > > br_netfilter if you need this.
> > > > > > > [2.078795] Key type dns_resolver registered
> > > > > > > [2.079317] cio: Channel measurement facility initialized
> > > > > > > using format extended (mode autodetected)
> > > > > > > [2.081494] Discipline DIAG cannot be used without z/VM
> > > > > > > [ 260.626363] random: crng init done
> > > > > > > qemu-system-s390x: terminating on signal 15 from pid 3815762
> > > > > > > (timeout)
> > > > > > >
> > > > > > > We have a simple rootfs available if necessary:
> > > > > > >
> > > > > > > https://github.com/ClangBuiltLinux/boot-utils/raw/bc0d17785eb67f1edd0ee0a134970a807895f741/images/s390/rootfs.cpio.zst
> > > > > > >
> > > > > > > If there is any other information I can provide, please let me
> > > > > > >
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Mon, May 23, 2022 at 8:17 AM Nathan Chancellor wrote:
>
> On Fri, May 20, 2022 at 05:15:55PM -0700, Saravana Kannan wrote:
> > On Fri, May 20, 2022 at 5:04 PM Nathan Chancellor wrote:
> > >
> > > On Fri, May 20, 2022 at 04:49:48PM -0700, Saravana Kannan wrote:
> > > > On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor
> > > > wrote:
> > > > >
> > > > > Hi Saravana,
> > > > >
> > > > > On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > > > > > The deferred probe timer that's used for this currently starts at
> > > > > > late_initcall and runs for driver_deferred_probe_timeout seconds.
> > > > > > The
> > > > > > assumption being that all available drivers would be loaded and
> > > > > > registered before the timer expires. This means, the
> > > > > > driver_deferred_probe_timeout has to be pretty large for it to
> > > > > > cover the
> > > > > > worst case. But if we set the default value for it to cover the
> > > > > > worst
> > > > > > case, it would significantly slow down the average case. For this
> > > > > > reason, the default value is set to 0.
> > > > > >
> > > > > > Also, with CONFIG_MODULES=y and the current default values of
> > > > > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with
> > > > > > missing
> > > > > > drivers will cause their consumer devices to always defer their
> > > > > > probes.
> > > > > > This is because device links created by fw_devlink defer the probe
> > > > > > even
> > > > > > before the consumer driver's probe() is called.
> > > > > >
> > > > > > Instead of a fixed timeout, if we extend an unexpired deferred probe
> > > > > > timer on every successful driver registration, with the expectation
> > > > > > more
> > > > > > modules would be loaded in the near future, then the default value
> > > > > > of
> > > > > > driver_deferred_probe_timeout only needs to be as long as the worst
> > > > > > case
> > > > > > time difference between two consecutive module loads.
> > > > > >
> > > > > > So let's implement that and set the default value to 10 seconds when
> > > > > > CONFIG_MODULES=y.
> > > > > >
> > > > > > Cc: Greg Kroah-Hartman
> > > > > > Cc: "Rafael J. Wysocki"
> > > > > > Cc: Rob Herring
> > > > > > Cc: Linus Walleij
> > > > > > Cc: Will Deacon
> > > > > > Cc: Ulf Hansson
> > > > > > Cc: Kevin Hilman
> > > > > > Cc: Thierry Reding
> > > > > > Cc: Mark Brown
> > > > > > Cc: Pavel Machek
> > > > > > Cc: Geert Uytterhoeven
> > > > > > Cc: Yoshihiro Shimoda
> > > > > > Cc: Paul Kocialkowski
> > > > > > Cc: linux-g...@vger.kernel.org
> > > > > > Cc: linux...@vger.kernel.org
> > > > > > Cc: iommu@lists.linux-foundation.org
> > > > > > Signed-off-by: Saravana Kannan
> > > > >
> > > > > I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
> > > > > change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> > > > > timeout on driver registration") in next-20220520 (bisect log below).
> > > > >
> > > > > $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu-
> > > > > defconfig bzImage
> > > > >
> > > > > $ timeout --foreground 15m stdbuf -oL -eL \
> > > > > qemu-system-s390x \
> > > > > -initrd ... \
> > > > > -M s390-ccw-virtio \
> > > > > -display none \
> > > > > -kernel arch/s390/boot/bzImage \
> > > > > -m 512m \
> > > > > -nodefaults \
> > > > > -serial mon:stdio
> > > > > ...
> > > > > [2.077303] In-situ OAM (IOAM) with IPv6
> > > > > [2.077639] NET: Registered PF_PACKET protocol family
> > > > > [2.078063] bridge: filtering via arp/ip/ip6tables is no longer
> > > > > available by default. Update your scripts to load br_netfilter if you
> > > > > need this.
> > > > > [2.078795] Key type dns_resolver registered
> > > > > [2.079317] cio: Channel measurement facility initialized using
> > > > > format extended (mode autodetected)
> > > > > [2.081494] Discipline DIAG cannot be used without z/VM
> > > > > [ 260.626363] random: crng init done
> > > > > qemu-system-s390x: terminating on signal 15 from pid 3815762 (timeout)
> > > > >
> > > > > We have a simple rootfs available if necessary:
> > > > >
> > > > > https://github.com/ClangBuiltLinux/boot-utils/raw/bc0d17785eb67f1edd0ee0a134970a807895f741/images/s390/rootfs.cpio.zst
> > > > >
> > > > > If there is any other information I can provide, please let me know!
> > > >
> > > > Hmm... strange. Can you please try the following command line options
> > > > and tell me which of these has the issue and which don't?
> > >
> > > Sure thing!
> > >
> > > > 1) deferred_probe_timeout=0
> > >
> > > No issue.
> > >
> > > > 2) deferred_probe_timeout=1
> > > > 3) deferred_probe_timeout=300
> > >
> > > Both of these appear to hang in the same way, I let each sit for five
> > > minutes.
> >
> > Strange that a sufficiently large timeout isn't helping. Is it trying
> > to boot off a network mount? I'll continue looking into this next
> > week.
>
> I don't think so, it seems like doing that requires some extra flags
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Fri, May 20, 2022 at 5:04 PM Nathan Chancellor wrote:
>
> On Fri, May 20, 2022 at 04:49:48PM -0700, Saravana Kannan wrote:
> > On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor wrote:
> > >
> > > Hi Saravana,
> > >
> > > On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > > > The deferred probe timer that's used for this currently starts at
> > > > late_initcall and runs for driver_deferred_probe_timeout seconds. The
> > > > assumption being that all available drivers would be loaded and
> > > > registered before the timer expires. This means, the
> > > > driver_deferred_probe_timeout has to be pretty large for it to cover the
> > > > worst case. But if we set the default value for it to cover the worst
> > > > case, it would significantly slow down the average case. For this
> > > > reason, the default value is set to 0.
> > > >
> > > > Also, with CONFIG_MODULES=y and the current default values of
> > > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with missing
> > > > drivers will cause their consumer devices to always defer their probes.
> > > > This is because device links created by fw_devlink defer the probe even
> > > > before the consumer driver's probe() is called.
> > > >
> > > > Instead of a fixed timeout, if we extend an unexpired deferred probe
> > > > timer on every successful driver registration, with the expectation more
> > > > modules would be loaded in the near future, then the default value of
> > > > driver_deferred_probe_timeout only needs to be as long as the worst case
> > > > time difference between two consecutive module loads.
> > > >
> > > > So let's implement that and set the default value to 10 seconds when
> > > > CONFIG_MODULES=y.
> > > >
> > > > Cc: Greg Kroah-Hartman
> > > > Cc: "Rafael J. Wysocki"
> > > > Cc: Rob Herring
> > > > Cc: Linus Walleij
> > > > Cc: Will Deacon
> > > > Cc: Ulf Hansson
> > > > Cc: Kevin Hilman
> > > > Cc: Thierry Reding
> > > > Cc: Mark Brown
> > > > Cc: Pavel Machek
> > > > Cc: Geert Uytterhoeven
> > > > Cc: Yoshihiro Shimoda
> > > > Cc: Paul Kocialkowski
> > > > Cc: linux-g...@vger.kernel.org
> > > > Cc: linux...@vger.kernel.org
> > > > Cc: iommu@lists.linux-foundation.org
> > > > Signed-off-by: Saravana Kannan
> > >
> > > I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
> > > change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> > > timeout on driver registration") in next-20220520 (bisect log below).
> > >
> > > $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- defconfig
> > > bzImage
> > >
> > > $ timeout --foreground 15m stdbuf -oL -eL \
> > > qemu-system-s390x \
> > > -initrd ... \
> > > -M s390-ccw-virtio \
> > > -display none \
> > > -kernel arch/s390/boot/bzImage \
> > > -m 512m \
> > > -nodefaults \
> > > -serial mon:stdio
> > > ...
> > > [2.077303] In-situ OAM (IOAM) with IPv6
> > > [2.077639] NET: Registered PF_PACKET protocol family
> > > [2.078063] bridge: filtering via arp/ip/ip6tables is no longer
> > > available by default. Update your scripts to load br_netfilter if you
> > > need this.
> > > [2.078795] Key type dns_resolver registered
> > > [2.079317] cio: Channel measurement facility initialized using format
> > > extended (mode autodetected)
> > > [2.081494] Discipline DIAG cannot be used without z/VM
> > > [ 260.626363] random: crng init done
> > > qemu-system-s390x: terminating on signal 15 from pid 3815762 (timeout)
> > >
> > > We have a simple rootfs available if necessary:
> > >
> > > https://github.com/ClangBuiltLinux/boot-utils/raw/bc0d17785eb67f1edd0ee0a134970a807895f741/images/s390/rootfs.cpio.zst
> > >
> > > If there is any other information I can provide, please let me know!
> >
> > Hmm... strange. Can you please try the following command line options
> > and tell me which of these has the issue and which don't?
>
> Sure thing!
>
> > 1) deferred_probe_timeout=0
>
> No issue.
>
> > 2) deferred_probe_timeout=1
> > 3) deferred_probe_timeout=300
>
> Both of these appear to hang in the same way, I let each sit for five
> minutes.
Strange that a sufficiently large timeout isn't helping. Is it trying
to boot off a network mount? I'll continue looking into this next
week.
-Saravana
>
> Cheers,
> Nathan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Fri, May 20, 2022 at 4:30 PM Nathan Chancellor wrote:
>
> Hi Saravana,
>
> On Fri, Apr 29, 2022 at 03:09:32PM -0700, Saravana Kannan wrote:
> > The deferred probe timer that's used for this currently starts at
> > late_initcall and runs for driver_deferred_probe_timeout seconds. The
> > assumption being that all available drivers would be loaded and
> > registered before the timer expires. This means, the
> > driver_deferred_probe_timeout has to be pretty large for it to cover the
> > worst case. But if we set the default value for it to cover the worst
> > case, it would significantly slow down the average case. For this
> > reason, the default value is set to 0.
> >
> > Also, with CONFIG_MODULES=y and the current default values of
> > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with missing
> > drivers will cause their consumer devices to always defer their probes.
> > This is because device links created by fw_devlink defer the probe even
> > before the consumer driver's probe() is called.
> >
> > Instead of a fixed timeout, if we extend an unexpired deferred probe
> > timer on every successful driver registration, with the expectation more
> > modules would be loaded in the near future, then the default value of
> > driver_deferred_probe_timeout only needs to be as long as the worst case
> > time difference between two consecutive module loads.
> >
> > So let's implement that and set the default value to 10 seconds when
> > CONFIG_MODULES=y.
> >
> > Cc: Greg Kroah-Hartman
> > Cc: "Rafael J. Wysocki"
> > Cc: Rob Herring
> > Cc: Linus Walleij
> > Cc: Will Deacon
> > Cc: Ulf Hansson
> > Cc: Kevin Hilman
> > Cc: Thierry Reding
> > Cc: Mark Brown
> > Cc: Pavel Machek
> > Cc: Geert Uytterhoeven
> > Cc: Yoshihiro Shimoda
> > Cc: Paul Kocialkowski
> > Cc: linux-g...@vger.kernel.org
> > Cc: linux...@vger.kernel.org
> > Cc: iommu@lists.linux-foundation.org
> > Signed-off-by: Saravana Kannan
>
> I bisected a boot hang with ARCH=s390 defconfig in QEMU down to this
> change as commit 2b28a1a84a0e ("driver core: Extend deferred probe
> timeout on driver registration") in next-20220520 (bisect log below).
>
> $ make -skj"$(nproc)" ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- defconfig
> bzImage
>
> $ timeout --foreground 15m stdbuf -oL -eL \
> qemu-system-s390x \
> -initrd ... \
> -M s390-ccw-virtio \
> -display none \
> -kernel arch/s390/boot/bzImage \
> -m 512m \
> -nodefaults \
> -serial mon:stdio
> ...
> [2.077303] In-situ OAM (IOAM) with IPv6
> [2.077639] NET: Registered PF_PACKET protocol family
> [2.078063] bridge: filtering via arp/ip/ip6tables is no longer available
> by default. Update your scripts to load br_netfilter if you need this.
> [2.078795] Key type dns_resolver registered
> [2.079317] cio: Channel measurement facility initialized using format
> extended (mode autodetected)
> [2.081494] Discipline DIAG cannot be used without z/VM
> [ 260.626363] random: crng init done
> qemu-system-s390x: terminating on signal 15 from pid 3815762 (timeout)
>
> We have a simple rootfs available if necessary:
>
> https://github.com/ClangBuiltLinux/boot-utils/raw/bc0d17785eb67f1edd0ee0a134970a807895f741/images/s390/rootfs.cpio.zst
>
> If there is any other information I can provide, please let me know!
Hmm... strange. Can you please try the following command line options
and tell me which of these has the issue and which don't?
1) deferred_probe_timeout=0
2) deferred_probe_timeout=1
3) deferred_probe_timeout=300
That should help me narrow down where the error might be.
Thanks,
Saravana
>
> Cheers,
> Nathan
>
> # bad: [18ecd30af1a8402c162cca1bd58771c0e5be7815] Add linux-next specific
> files for 20220520
> # good: [b015dcd62b86d298829990f8261d5d154b8d7af5] Merge tag
> 'for-5.18/parisc-4' of
> git://git.kernel.org/pub/scm/linux/kernel/git/deller/parisc-linux
> git bisect start '18ecd30af1a8402c162cca1bd58771c0e5be7815'
> 'b015dcd62b86d298829990f8261d5d154b8d7af5'
> # good: [f9b63740b666dd9887eb0282d21b5f65bb0cadd0] Merge branch 'master' of
> git://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git
> git bisect good f9b63740b666dd9887eb0282d21b5f65bb0cadd0
> # good: [1f5eb3e76303572f0318e8c50da51c516580aa03] Merge branch 'master' of
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
> git bisect good 1f5eb3e76303572f0318e8c50da51c516580aa03
> # bad: [4c1d9cc0363691893ef94fa0d798faca013e27d3] Merge branch 'staging-next'
> of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging.git
> git bisect bad 4c1d9cc0363691893ef94fa0d798faca013e27d3
> # bad: [dcb68304485c0ba5f84f1a54687c751b68263d93] Merge branch 'usb-next' of
> git://git.kernel.org/pub/scm/linux/kernel/git/johan/usb-serial.git
> git bisect bad dcb68304485c0ba5f84f1a54687c751b68263d93
> # good: [61271996dc46aecb40fd26f89a4ec0a6bd8f3a8f] Merge branch 'next' of
> git://git.kernel.org/pub/scm/virt/kvm/kvm.git
> git bisect good 61271996dc46aecb40fd26f89a4ec0a6bd8f3a8f
>
Re: [PATCH v1] driver core: Extend deferred probe timeout on driver registration
On Fri, May 13, 2022 at 6:58 AM Rob Herring wrote: > > On Fri, Apr 29, 2022 at 5:09 PM Saravana Kannan wrote: > > > > The deferred probe timer that's used for this currently starts at > > late_initcall and runs for driver_deferred_probe_timeout seconds. The > > assumption being that all available drivers would be loaded and > > registered before the timer expires. This means, the > > driver_deferred_probe_timeout has to be pretty large for it to cover the > > worst case. But if we set the default value for it to cover the worst > > case, it would significantly slow down the average case. For this > > reason, the default value is set to 0. > > > > Also, with CONFIG_MODULES=y and the current default values of > > driver_deferred_probe_timeout=0 and fw_devlink=on, devices with missing > > drivers will cause their consumer devices to always defer their probes. > > This is because device links created by fw_devlink defer the probe even > > before the consumer driver's probe() is called. > > > > Instead of a fixed timeout, if we extend an unexpired deferred probe > > timer on every successful driver registration, with the expectation more > > modules would be loaded in the near future, then the default value of > > driver_deferred_probe_timeout only needs to be as long as the worst case > > time difference between two consecutive module loads. > > > > So let's implement that and set the default value to 10 seconds when > > CONFIG_MODULES=y. > > We had to revert a non-zero timeout before (issue with NFS root IIRC). > Does fw_devlink=on somehow fix that? If it's the one where ip autoconfig was timing out, then John Stultz fixed it by fixing wait_for_device_probe(). https://lore.kernel.org/all/20200422203245.83244-4-john.stu...@linaro.org/ If you are referring to some other issue, then I'd need more details. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v1] driver core: Extend deferred probe timeout on driver registration
The deferred probe timer that's used for this currently starts at
late_initcall and runs for driver_deferred_probe_timeout seconds. The
assumption being that all available drivers would be loaded and
registered before the timer expires. This means, the
driver_deferred_probe_timeout has to be pretty large for it to cover the
worst case. But if we set the default value for it to cover the worst
case, it would significantly slow down the average case. For this
reason, the default value is set to 0.
Also, with CONFIG_MODULES=y and the current default values of
driver_deferred_probe_timeout=0 and fw_devlink=on, devices with missing
drivers will cause their consumer devices to always defer their probes.
This is because device links created by fw_devlink defer the probe even
before the consumer driver's probe() is called.
Instead of a fixed timeout, if we extend an unexpired deferred probe
timer on every successful driver registration, with the expectation more
modules would be loaded in the near future, then the default value of
driver_deferred_probe_timeout only needs to be as long as the worst case
time difference between two consecutive module loads.
So let's implement that and set the default value to 10 seconds when
CONFIG_MODULES=y.
Cc: Greg Kroah-Hartman
Cc: "Rafael J. Wysocki"
Cc: Rob Herring
Cc: Linus Walleij
Cc: Will Deacon
Cc: Ulf Hansson
Cc: Kevin Hilman
Cc: Thierry Reding
Cc: Mark Brown
Cc: Pavel Machek
Cc: Geert Uytterhoeven
Cc: Yoshihiro Shimoda
Cc: Paul Kocialkowski
Cc: linux-g...@vger.kernel.org
Cc: linux...@vger.kernel.org
Cc: iommu@lists.linux-foundation.org
Signed-off-by: Saravana Kannan
---
.../admin-guide/kernel-parameters.txt | 6 --
drivers/base/base.h | 1 +
drivers/base/dd.c | 19 +++
drivers/base/driver.c | 1 +
4 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt
b/Documentation/admin-guide/kernel-parameters.txt
index 7123524a86b8..fcc1dfc877a9 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -939,8 +939,10 @@
[KNL] Debugging option to set a timeout in seconds for
deferred probe to give up waiting on dependencies to
probe. Only specific dependencies (subsystems or
- drivers) that have opted in will be ignored. A timeout
of 0
- will timeout at the end of initcalls. This option will
also
+ drivers) that have opted in will be ignored. A timeout
+ of 0 will timeout at the end of initcalls. If the time
+ out hasn't expired, it'll be restarted by each
+ successful driver registration. This option will also
dump out devices still on the deferred probe list after
retrying.
diff --git a/drivers/base/base.h b/drivers/base/base.h
index 2882af26392a..ab71403d102f 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -159,6 +159,7 @@ extern char *make_class_name(const char *name, struct
kobject *kobj);
extern int devres_release_all(struct device *dev);
extern void device_block_probing(void);
extern void device_unblock_probing(void);
+extern void deferred_probe_extend_timeout(void);
/* /sys/devices directory */
extern struct kset *devices_kset;
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index f47cab21430f..603379b5f9dd 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -255,7 +255,12 @@ static int deferred_devs_show(struct seq_file *s, void
*data)
}
DEFINE_SHOW_ATTRIBUTE(deferred_devs);
+#ifdef CONFIG_MODULES
+int driver_deferred_probe_timeout = 10;
+#else
int driver_deferred_probe_timeout;
+#endif
+
EXPORT_SYMBOL_GPL(driver_deferred_probe_timeout);
static DECLARE_WAIT_QUEUE_HEAD(probe_timeout_waitqueue);
@@ -315,6 +320,20 @@ static void deferred_probe_timeout_work_func(struct
work_struct *work)
}
static DECLARE_DELAYED_WORK(deferred_probe_timeout_work,
deferred_probe_timeout_work_func);
+void deferred_probe_extend_timeout(void)
+{
+ /*
+* If the work hasn't been queued yet or if the work expired, don't
+* start a new one.
+*/
+ if (cancel_delayed_work(_probe_timeout_work)) {
+ schedule_delayed_work(_probe_timeout_work,
+ driver_deferred_probe_timeout * HZ);
+ pr_debug("Extended deferred probe timeout by %d secs\n",
+ driver_deferred_probe_timeout);
+ }
+}
+
/**
* deferred_probe_initcall() - Enable probing of deferred devices
*
diff --git a/drivers/base/driver.c b/drivers/base/driver.c
index 8c0d33e182fd..77a77b2095cd 100644
--- a/drivers/base/driver.c
+++ b/drivers/base/driver.c
@@ -177,6 +177,7 @@ int
Re: [PATCH] firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module
On Wed, Jul 21, 2021 at 1:23 PM Bjorn Andersson wrote: > > On Wed 21 Jul 13:00 CDT 2021, Saravana Kannan wrote: > > > On Wed, Jul 21, 2021 at 10:24 AM John Stultz wrote: > > > > > > On Wed, Jul 21, 2021 at 4:54 AM Greg Kroah-Hartman > > > wrote: > > > > > > > > On Wed, Jul 07, 2021 at 04:53:20AM +, John Stultz wrote: > > > > > Allow the qcom_scm driver to be loadable as a permenent module. > > > > > > > > This feels like a regression, it should be allowed to be a module. > > > > > > I'm sorry, I'm not sure I'm following you, Greg. This patch is trying > > > to enable the driver to be able to be loaded as a module. > > > > I think the mix up might be that Greg mentally read "permanent module" > > as "builtin"? > > > > "permanent module" is just something that can't be unloaded once it's > > loaded. It's not "builtin". > > > > Afaict there's nothing in this patch that makes it more or less > permanent. The lack of a module_exit() makes it a permanent module. If you do lsmod, it'll mark this as "[permanent]". -Saravana > The module will be quite permanent (in practice) because > several other core modules reference symbols in the qcom_scm module. > > But thanks to a previous patch, the qcom_scm device comes with > suppress_bind_attrs, to prevent that the device goes away from a simple > unbind operation - which the API and client drivers aren't designed to > handle. > > So, it would have been better in this case to omit the word "permanent" > from the commit message, but the change is good and I don't want to > rebase my tree to drop that word. > > Thanks, > Bjorn > > > -Saravana > > > > > > > > > > This still uses the "depends on QCOM_SCM || !QCOM_SCM" bit to > > > > > ensure that drivers that call into the qcom_scm driver are > > > > > also built as modules. While not ideal in some cases its the > > > > > only safe way I can find to avoid build errors without having > > > > > those drivers select QCOM_SCM and have to force it on (as > > > > > QCOM_SCM=n can be valid for those drivers). > > > > > > > > > > Reviving this now that Saravana's fw_devlink defaults to on, > > > > > which should avoid loading troubles seen before. > > > > > > > > fw_devlink was supposed to resolve these issues and _allow_ code to be > > > > built as modules and not forced to be built into the kernel. > > > > > > Right. I'm re-submitting this patch to enable a driver to work as a > > > module, because earlier attempts to submit it ran into boot trouble > > > because fw_devlink wasn't yet enabled. > > > > > > I worry something in my description made it seem otherwise, so let me > > > know how you read it and I'll try to avoid such confusion in the > > > future. > > > > > > thanks > > > -john ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module
On Wed, Jul 21, 2021 at 10:24 AM John Stultz wrote: > > On Wed, Jul 21, 2021 at 4:54 AM Greg Kroah-Hartman > wrote: > > > > On Wed, Jul 07, 2021 at 04:53:20AM +, John Stultz wrote: > > > Allow the qcom_scm driver to be loadable as a permenent module. > > > > This feels like a regression, it should be allowed to be a module. > > I'm sorry, I'm not sure I'm following you, Greg. This patch is trying > to enable the driver to be able to be loaded as a module. I think the mix up might be that Greg mentally read "permanent module" as "builtin"? "permanent module" is just something that can't be unloaded once it's loaded. It's not "builtin". -Saravana > > > > This still uses the "depends on QCOM_SCM || !QCOM_SCM" bit to > > > ensure that drivers that call into the qcom_scm driver are > > > also built as modules. While not ideal in some cases its the > > > only safe way I can find to avoid build errors without having > > > those drivers select QCOM_SCM and have to force it on (as > > > QCOM_SCM=n can be valid for those drivers). > > > > > > Reviving this now that Saravana's fw_devlink defaults to on, > > > which should avoid loading troubles seen before. > > > > fw_devlink was supposed to resolve these issues and _allow_ code to be > > built as modules and not forced to be built into the kernel. > > Right. I'm re-submitting this patch to enable a driver to work as a > module, because earlier attempts to submit it ran into boot trouble > because fw_devlink wasn't yet enabled. > > I worry something in my description made it seem otherwise, so let me > know how you read it and I'll try to avoid such confusion in the > future. > > thanks > -john ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] firmware: QCOM_SCM: Allow qcom_scm driver to be loadable as a permenent module
On Mon, Jul 19, 2021 at 12:36 PM Bjorn Andersson wrote: > > On Mon 19 Jul 14:00 CDT 2021, John Stultz wrote: > > > On Fri, Jul 16, 2021 at 10:01 PM Bjorn Andersson > > wrote: > > > On Tue 06 Jul 23:53 CDT 2021, John Stultz wrote: > > > > Allow the qcom_scm driver to be loadable as a permenent module. > > > > > > > > This still uses the "depends on QCOM_SCM || !QCOM_SCM" bit to > > > > ensure that drivers that call into the qcom_scm driver are > > > > also built as modules. While not ideal in some cases its the > > > > only safe way I can find to avoid build errors without having > > > > those drivers select QCOM_SCM and have to force it on (as > > > > QCOM_SCM=n can be valid for those drivers). > > > > > > > > Reviving this now that Saravana's fw_devlink defaults to on, > > > > which should avoid loading troubles seen before. > > > > > > > > > > Are you (in this last paragraph) saying that all those who have been > > > burnt by fw_devlink during the last months and therefor run with it > > > disabled will have a less fun experience once this is merged? > > > Bjorn, I jump in and help with any reports of issues with fw_devlink if I'm cc'ed. Please feel free to add me and I'll help fix any issues you have with fw_devlink=on. > > > > I guess potentially. So way back when this was originally submitted, > > some folks had trouble booting if it was set as a module due to it > > loading due to the deferred_probe_timeout expiring. > > My attempts to change the default timeout value to be larger ran into > > trouble, but Saravana's fw_devlink does manage to resolve things > > properly for this case. > > > > Unfortunately I see really weird things coming out of that, e.g. display > on my db845c is waiting for the USB hub on PCIe to load its firmware, > which typically times out after 60 seconds. > > I've stared at it quite a bit and I don't understand how they are > related. Can you please add me to any email thread with the details? I'd be happy to help. First step is to make sure all the devices probe as with fw_devlink=permissive. After that if you are still seeing issues, it's generally timing issues in the driver. But if the actual timing issue is identified (by you or whoever knows the driver seeing the issue), then I can help with fixes or suggestions for fixes. > > But if folks are having issues w/ fw_devlink, and have it disabled, > > and set QCOM_SCM=m they could still trip over the issue with the > > timeout firing before it is loaded (especially if they are loading > > modules from late mounted storage rather than ramdisk). > > > > I guess we'll have to force QCOM_SCM=y in the defconfig and hope people > don't make it =m. > > > > (I'm picking this up, but I don't fancy the idea that some people are > > > turning the boot process into a lottery) > > > > Me neither, and I definitely think the deferred_probe_timeout logic is > > way too fragile, which is why I'm eager for fw_devlink as it's a much > > less racy approach to handling module loading dependencies. > > Right, deferred_probe_timeout is the main issue here. Without it we > might get some weird probe deferral runs, but either some driver is > missing or it settles eventually. > > With deferred_probe_timeout it's rather common for me to see things > end up probe out of order (even more now with fw_devlink finding cyclic > dependencies) and deferred_probe_timeout just breaking things. Again, please CC me on these threads and I'd be happy to help. > > > So if you > > want to hold on this, while any remaining fw_devlink issues get > > sorted, that's fine. But I'd also not cast too much ire at > > fw_devlink, as the global probe timeout approach for handling optional > > links isn't great, and we need a better solution. > > > > There's no end to the possible and valid ways you can setup your > defconfig and run into the probe deferral issues, so I see no point in > holding this one back any longer. I just hope that one day it will be > possible to boot the upstream kernel in a reliable fashion. Might not be believable, but I'm hoping fw_devlink helps you meet this goal :) -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/6] iommu: Enable devices to request non-strict DMA, starting with QCom SD/MMC
On Tue, Jun 22, 2021 at 1:02 PM Rob Herring wrote: > > On Tue, Jun 22, 2021 at 09:06:02AM -0700, Doug Anderson wrote: > > Hi, > > > > On Tue, Jun 22, 2021 at 4:35 AM Robin Murphy wrote: > > > > > > Hi Doug, > > > > > > On 2021-06-22 00:52, Douglas Anderson wrote: > > > > > > > > This patch attempts to put forward a proposal for enabling non-strict > > > > DMA on a device-by-device basis. The patch series requests non-strict > > > > DMA for the Qualcomm SDHCI controller as a first device to enable, > > > > getting a nice bump in performance with what's believed to be a very > > > > small drop in security / safety (see the patch for the full argument). > > > > > > > > As part of this patch series I am end up slightly cleaning up some of > > > > the interactions between the PCI subsystem and the IOMMU subsystem but > > > > I don't go all the way to fully remove all the tentacles. Specifically > > > > this patch series only concerns itself with a single aspect: strict > > > > vs. non-strict mode for the IOMMU. I'm hoping that this will be easier > > > > to talk about / reason about for more subsystems compared to overall > > > > deciding what it means for a device to be "external" or "untrusted". > > > > > > > > If something like this patch series ends up being landable, it will > > > > undoubtedly need coordination between many maintainers to land. I > > > > believe it's fully bisectable but later patches in the series > > > > definitely depend on earlier ones. Sorry for the long CC list. :( > > > > > > Unfortunately, this doesn't work. In normal operation, the default > > > domains should be established long before individual drivers are even > > > loaded (if they are modules), let alone anywhere near probing. The fact > > > that iommu_probe_device() sometimes gets called far too late off the > > > back of driver probe is an unfortunate artefact of the original > > > probe-deferral scheme, and causes other problems like potentially > > > malformed groups - I've been forming a plan to fix that for a while now, > > > so I for one really can't condone anything trying to rely on it. > > > Non-deterministic behaviour based on driver probe order for multi-device > > > groups is part of the existing problem, and your proposal seems equally > > > vulnerable to that too. > > > > Doh! :( I definitely can't say I understand the iommu subsystem > > amazingly well. It was working for me, but I could believe that I was > > somehow violating a rule somewhere. > > > > I'm having a bit of a hard time understanding where the problem is > > though. Is there any chance that you missed the part of my series > > where I introduced a "pre_probe" step? Specifically, I see this: > > > > * really_probe() is called w/ a driver and a device. > > * -> calls dev->bus->dma_configure() w/ a "struct device *" > > * -> eventually calls iommu_probe_device() w/ the device. > > * -> calls iommu_alloc_default_domain() w/ the device > > * -> calls iommu_group_alloc_default_domain() > > * -> always allocates a new domain > > > > ...so we always have a "struct device" when a domain is allocated if > > that domain is going to be associated with a device. > > > > I will agree that iommu_probe_device() is called before the driver > > probe, but unless I missed something it's after the device driver is > > loaded. ...and assuming something like patch #1 in this series looks > > OK then iommu_probe_device() will be called after "pre_probe". > > > > So assuming I'm not missing something, I'm not actually relying the > > IOMMU getting init off the back of driver probe. > > > > > > > FWIW we already have a go-faster knob for people who want to tweak the > > > security/performance compromise for specific devices, namely the sysfs > > > interface for changing a group's domain type before binding the relevant > > > driver(s). Is that something you could use in your application, say from > > > an initramfs script? > > > > We've never had an initramfs script in Chrome OS. I don't know all the > > history of why (I'm trying to check), but I'm nearly certain it was a > > conscious decision. Probably it has to do with the fact that we're not > > trying to build a generic distribution where a single boot source can > > boot a huge variety of hardware. We generally have one kernel for a > > class of devices. I believe avoiding the initramfs just keeps things > > simpler. > > > > I think trying to revamp Chrome OS to switch to an initramfs type > > system would be a pretty big undertaking since (as I understand it) > > you can't just run a little command and then return to the normal boot > > flow. Once you switch to initramfs you're committing to finding / > > setting up the rootfs yourself and on Chrome OS I believe that means a > > whole bunch of dm-verity work. > > > > > > ...so probably the initramfs is a no-go for me, but I'm still crossing > > my fingers that the pre_probe() might be legit if you take a second > > look at it? > > Couldn't you have a driver
Re: [PATCH 4/6] iommu: Combine device strictness requests with the global default
On Tue, Jun 22, 2021 at 9:46 AM Doug Anderson wrote:
>
> Hi,
>
> On Mon, Jun 21, 2021 at 7:56 PM Saravana Kannan wrote:
> >
> > On Mon, Jun 21, 2021 at 4:53 PM Douglas Anderson
> > wrote:
> > >
> > > In the patch ("drivers: base: Add bits to struct device to control
> > > iommu strictness") we add the ability for devices to tell us about
> > > their IOMMU strictness requirements. Let's now take that into account
> > > in the IOMMU layer.
> > >
> > > A few notes here:
> > > * Presumably this is always how iommu_get_dma_strict() was intended to
> > > behave. Had this not been the intention then it never would have
> > > taken a domain as a parameter.
> > > * The iommu_set_dma_strict() feels awfully non-symmetric now. That
> > > function sets the _default_ strictness globally in the system
> > > whereas iommu_get_dma_strict() returns the value for a given domain
> > > (falling back to the default). Presumably, at least, the fact that
> > > iommu_set_dma_strict() doesn't take a domain makes this obvious.
> > >
> > > The function iommu_get_dma_strict() should now make it super obvious
> > > where strictness comes from and who overides who. Though the function
> > > changed a bunch to make the logic clearer, the only two new rules
> > > should be:
> > > * Devices can force strictness for themselves, overriding the cmdline
> > > "iommu.strict=0" or a call to iommu_set_dma_strict(false)).
> > > * Devices can request non-strictness for themselves, assuming there
> > > was no cmdline "iommu.strict=1" or a call to
> > > iommu_set_dma_strict(true).
> > >
> > > Signed-off-by: Douglas Anderson
> > > ---
> > >
> > > drivers/iommu/iommu.c | 56 +--
> > > include/linux/iommu.h | 2 ++
> > > 2 files changed, 45 insertions(+), 13 deletions(-)
> > >
> > > diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> > > index 808ab70d5df5..0c84a4c06110 100644
> > > --- a/drivers/iommu/iommu.c
> > > +++ b/drivers/iommu/iommu.c
> > > @@ -28,8 +28,19 @@
> > > static struct kset *iommu_group_kset;
> > > static DEFINE_IDA(iommu_group_ida);
> > >
> > > +enum iommu_strictness {
> > > + IOMMU_DEFAULT_STRICTNESS = -1,
> > > + IOMMU_NOT_STRICT = 0,
> > > + IOMMU_STRICT = 1,
> > > +};
> > > +static inline enum iommu_strictness bool_to_strictness(bool strictness)
> > > +{
> > > + return (enum iommu_strictness)strictness;
> > > +}
> > > +
> > > static unsigned int iommu_def_domain_type __read_mostly;
> > > -static bool iommu_dma_strict __read_mostly = true;
> > > +static enum iommu_strictness cmdline_dma_strict __read_mostly =
> > > IOMMU_DEFAULT_STRICTNESS;
> > > +static enum iommu_strictness driver_dma_strict __read_mostly =
> > > IOMMU_DEFAULT_STRICTNESS;
> > > static u32 iommu_cmd_line __read_mostly;
> > >
> > > struct iommu_group {
> > > @@ -69,7 +80,6 @@ static const char * const
> > > iommu_group_resv_type_string[] = {
> > > };
> > >
> > > #define IOMMU_CMD_LINE_DMA_API BIT(0)
> > > -#define IOMMU_CMD_LINE_STRICT BIT(1)
> > >
> > > static int iommu_alloc_default_domain(struct iommu_group *group,
> > > struct device *dev);
> > > @@ -336,25 +346,38 @@ early_param("iommu.passthrough",
> > > iommu_set_def_domain_type);
> > >
> > > static int __init iommu_dma_setup(char *str)
> > > {
> > > - int ret = kstrtobool(str, _dma_strict);
> > > + bool strict;
> > > + int ret = kstrtobool(str, );
> > >
> > > if (!ret)
> > > - iommu_cmd_line |= IOMMU_CMD_LINE_STRICT;
> > > + cmdline_dma_strict = bool_to_strictness(strict);
> > > return ret;
> > > }
> > > early_param("iommu.strict", iommu_dma_setup);
> > >
> > > void iommu_set_dma_strict(bool strict)
> > > {
> > > - if (strict || !(iommu_cmd_line & IOMMU_CMD_LINE_STRICT))
> > > - iommu_dma_strict = strict;
> > > + /* A driver can request strictness but not the other way around */
> > > + if (driver_dma_strict != IOMMU_STRICT)
> > > + driver_dma_strict = bool_to_strictness(strict);
> > > }
> > >
> > > bool iommu_get_dma_strict(struct iommu_domain *domain)
> > > {
> > > - /* only allow lazy flushing for DMA domains */
> > > - if (domain->type == IOMMU_DOMAIN_DMA)
> > > - return iommu_dma_strict;
> > > + /* Non-DMA domains or anyone forcing it to strict makes it strict
> > > */
> > > + if (domain->type != IOMMU_DOMAIN_DMA ||
> > > + cmdline_dma_strict == IOMMU_STRICT ||
> > > + driver_dma_strict == IOMMU_STRICT ||
> > > + domain->force_strict)
> > > + return true;
> > > +
> > > + /* Anyone requesting non-strict (if no forces) makes it
> > > non-strict */
> > > + if (cmdline_dma_strict == IOMMU_NOT_STRICT ||
> > > + driver_dma_strict == IOMMU_NOT_STRICT ||
> > > + domain->request_non_strict)
> > > + return
Re: [PATCH 4/6] iommu: Combine device strictness requests with the global default
On Mon, Jun 21, 2021 at 4:53 PM Douglas Anderson wrote:
>
> In the patch ("drivers: base: Add bits to struct device to control
> iommu strictness") we add the ability for devices to tell us about
> their IOMMU strictness requirements. Let's now take that into account
> in the IOMMU layer.
>
> A few notes here:
> * Presumably this is always how iommu_get_dma_strict() was intended to
> behave. Had this not been the intention then it never would have
> taken a domain as a parameter.
> * The iommu_set_dma_strict() feels awfully non-symmetric now. That
> function sets the _default_ strictness globally in the system
> whereas iommu_get_dma_strict() returns the value for a given domain
> (falling back to the default). Presumably, at least, the fact that
> iommu_set_dma_strict() doesn't take a domain makes this obvious.
>
> The function iommu_get_dma_strict() should now make it super obvious
> where strictness comes from and who overides who. Though the function
> changed a bunch to make the logic clearer, the only two new rules
> should be:
> * Devices can force strictness for themselves, overriding the cmdline
> "iommu.strict=0" or a call to iommu_set_dma_strict(false)).
> * Devices can request non-strictness for themselves, assuming there
> was no cmdline "iommu.strict=1" or a call to
> iommu_set_dma_strict(true).
>
> Signed-off-by: Douglas Anderson
> ---
>
> drivers/iommu/iommu.c | 56 +--
> include/linux/iommu.h | 2 ++
> 2 files changed, 45 insertions(+), 13 deletions(-)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 808ab70d5df5..0c84a4c06110 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -28,8 +28,19 @@
> static struct kset *iommu_group_kset;
> static DEFINE_IDA(iommu_group_ida);
>
> +enum iommu_strictness {
> + IOMMU_DEFAULT_STRICTNESS = -1,
> + IOMMU_NOT_STRICT = 0,
> + IOMMU_STRICT = 1,
> +};
> +static inline enum iommu_strictness bool_to_strictness(bool strictness)
> +{
> + return (enum iommu_strictness)strictness;
> +}
> +
> static unsigned int iommu_def_domain_type __read_mostly;
> -static bool iommu_dma_strict __read_mostly = true;
> +static enum iommu_strictness cmdline_dma_strict __read_mostly =
> IOMMU_DEFAULT_STRICTNESS;
> +static enum iommu_strictness driver_dma_strict __read_mostly =
> IOMMU_DEFAULT_STRICTNESS;
> static u32 iommu_cmd_line __read_mostly;
>
> struct iommu_group {
> @@ -69,7 +80,6 @@ static const char * const iommu_group_resv_type_string[] = {
> };
>
> #define IOMMU_CMD_LINE_DMA_API BIT(0)
> -#define IOMMU_CMD_LINE_STRICT BIT(1)
>
> static int iommu_alloc_default_domain(struct iommu_group *group,
> struct device *dev);
> @@ -336,25 +346,38 @@ early_param("iommu.passthrough",
> iommu_set_def_domain_type);
>
> static int __init iommu_dma_setup(char *str)
> {
> - int ret = kstrtobool(str, _dma_strict);
> + bool strict;
> + int ret = kstrtobool(str, );
>
> if (!ret)
> - iommu_cmd_line |= IOMMU_CMD_LINE_STRICT;
> + cmdline_dma_strict = bool_to_strictness(strict);
> return ret;
> }
> early_param("iommu.strict", iommu_dma_setup);
>
> void iommu_set_dma_strict(bool strict)
> {
> - if (strict || !(iommu_cmd_line & IOMMU_CMD_LINE_STRICT))
> - iommu_dma_strict = strict;
> + /* A driver can request strictness but not the other way around */
> + if (driver_dma_strict != IOMMU_STRICT)
> + driver_dma_strict = bool_to_strictness(strict);
> }
>
> bool iommu_get_dma_strict(struct iommu_domain *domain)
> {
> - /* only allow lazy flushing for DMA domains */
> - if (domain->type == IOMMU_DOMAIN_DMA)
> - return iommu_dma_strict;
> + /* Non-DMA domains or anyone forcing it to strict makes it strict */
> + if (domain->type != IOMMU_DOMAIN_DMA ||
> + cmdline_dma_strict == IOMMU_STRICT ||
> + driver_dma_strict == IOMMU_STRICT ||
> + domain->force_strict)
> + return true;
> +
> + /* Anyone requesting non-strict (if no forces) makes it non-strict */
> + if (cmdline_dma_strict == IOMMU_NOT_STRICT ||
> + driver_dma_strict == IOMMU_NOT_STRICT ||
> + domain->request_non_strict)
> + return false;
> +
> + /* Nobody said anything, so it's strict by default */
If iommu.strict is not set in the command line, upstream treats it as
iommu.strict=1. Meaning, no drivers can override it.
If I understand it correctly, with your series, if iommu.strict=1 is
not set, drivers can override the "default strict mode" and ask for
non-strict mode for their domain. So if this series gets in and future
driver changes start asking for non-strict mode, systems that are
expected to operate in fully strict mode will now have devices
operating in non-strict mode.
That's breaking backward
Re: [PATCH v2 5/7] driver core: Add device location to "struct device" and expose it in sysfs
On Mon, Jun 29, 2020 at 9:49 PM Rajat Jain wrote:
>
> Add a new (optional) field to denote the physical location of a device
> in the system, and expose it in sysfs. This was discussed here:
> https://lore.kernel.org/linux-acpi/20200618184621.ga446...@kroah.com/
>
> (The primary choice for attribute name i.e. "location" is already
> exposed as an ABI elsewhere, so settled for "site"). Individual buses
> that want to support this new attribute can opt-in by setting a flag in
> bus_type, and then populating the location of device while enumerating
> it.
>
> Signed-off-by: Rajat Jain
> ---
> v2: (Initial version)
>
> drivers/base/core.c| 35 +++
> include/linux/device.h | 42 ++
> include/linux/device/bus.h | 8
> 3 files changed, 85 insertions(+)
>
I'm not CC'ed in 4/7, so just replying
> diff --git a/include/linux/device.h b/include/linux/device.h
> index 15460a5ac024a..a4143735ae712 100644
> --- a/include/linux/device.h
> +++ b/include/linux/device.h
> @@ -428,6 +428,31 @@ enum dl_dev_state {
> DL_DEV_UNBINDING,
> };
>
> +/**
> + * enum device_site - Physical location of the device in the system.
> + * The semantics of values depend on subsystem / bus:
> + *
> + * @SITE_UNKNOWN: Location is Unknown (default)
> + *
> + * @SITE_INTERNAL: Device is internal to the system, and cannot be (easily)
> + * removed. E.g. SoC internal devices, onboard soldered
> + * devices, internal M.2 cards (that cannot be removed
> + * without opening the chassis).
> + * @SITE_EXTENDED: Device sits an extension of the system. E.g. devices
> + * on external PCIe trays, docking stations etc. These
> + * devices may be removable, but are generally housed
> + * internally on an extension board, so they are removed
> + * only when that whole extension board is removed.
> + * @SITE_EXTERNAL: Devices truly external to the system (i.e. plugged on
> + * an external port) that may be removed or added frequently.
> + */
> +enum device_site {
> + SITE_UNKNOWN = 0,
> + SITE_INTERNAL,
> + SITE_EXTENDED,
> + SITE_EXTERNAL,
> +};
> +
> /**
> * struct dev_links_info - Device data related to device links.
> * @suppliers: List of links to supplier devices.
> @@ -513,6 +538,7 @@ struct dev_links_info {
> * device (i.e. the bus driver that discovered the device).
> * @iommu_group: IOMMU group the device belongs to.
> * @iommu: Per device generic IOMMU runtime data
> + * @site: Physical location of the device w.r.t. the system
> *
> * @offline_disabled: If set, the device is permanently online.
> * @offline: Set after successful invocation of bus type's .offline().
> @@ -613,6 +639,8 @@ struct device {
> struct iommu_group *iommu_group;
> struct dev_iommu*iommu;
>
> + enum device_sitesite; /* Device physical location */
> +
> booloffline_disabled:1;
> booloffline:1;
> boolof_node_reused:1;
> @@ -806,6 +834,20 @@ static inline bool dev_has_sync_state(struct device *dev)
> return false;
> }
>
> +static inline int dev_set_site(struct device *dev, enum device_site site)
> +{
> + if (site < SITE_UNKNOWN || site > SITE_EXTERNAL)
> + return -EINVAL;
> +
> + dev->site = site;
> + return 0;
> +}
> +
> +static inline bool dev_is_external(struct device *dev)
> +{
> + return dev->site == SITE_EXTERNAL;
> +}
I'm not CC'ed in the rest of the patches in this series, so just
responding here. I see you use this function in patch 6/7 to decide if
the PCI device is trusted. Anything other than EXTERNAL is being
treated as trusted. I'd argue that anything that's not internal should
be distrusted. For example, I can have a hacked up laptop dock that I
can share with you when you visit my home/office and now you are
trusting it when you shouldn't be.
Also, "UNKNOWN" is treated as trusted in patch 6/7. I'm guessing this
is because some of the devices might not have the info in their
firmware? At which point, this feature isn't even protecting all the
PCI ports properly? This adds to Greg point that this should be a
userspace policy so that it can override whatever is wrong/missing in
the firmware.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RFC 0/2] iommu: arm-smmu: Add support for early direct mappings
I added everyone from the other thread, but somehow managed to miss the Bjorn who sent the emails! Fixing that now. On Mon, Jan 13, 2020 at 6:07 AM Thierry Reding wrote: > > On Fri, Jan 10, 2020 at 08:56:39PM -0800, Saravana Kannan wrote: > > Hi Thierry, > > > > I happened upon this thread while looking into another thread [1]. > > > > > From: Thierry Reding > > > > > > On some platforms, the firmware will setup hardware to read from a given > > > region of memory. One such example is a display controller that is > > > scanning out a splash screen from physical memory. > > > > > > During Linux' boot process, the ARM SMMU will configure all contexts to > > > fault by default. This means that memory accesses that happen by an SMMU > > > master before its driver has had a chance to properly set up the IOMMU > > > will cause a fault. This is especially annoying for something like the > > > display controller scanning out a splash screen because the faults will > > > result in the display controller getting bogus data (all-ones on Tegra) > > > and since it repeatedly scans that framebuffer, it will keep triggering > > > such faults and spam the boot log with them. > > > > While I'm not an expert on IOMMUs, I have a decent high level > > understanding of the problem you are trying to solve. > > > > > In order to work around such problems, scan the device tree for IOMMU > > > masters and set up a special identity domain that will map 1:1 all of > > > the reserved regions associated with them. This happens before the SMMU > > > is enabled, so that the mappings are already set up before translations > > > begin. > > > > I'm not sure if this RFC will solve the splash screen issue across SoCs > > ([1] seems to have a different issue and might not have memory-regions). > > Looking at the proposed patches, they look like they're solving a > different, although related, problem. In your case you seem to have a > bootloader that already sets up the SMMU to translate for a given > master. The case that I'm trying to solve here is where the bootloader > has not yet setup the SMMU but has instead pointed some device to read > memory from a physical address. Ah, thanks for explaining your scenario. > So what this patch is trying to solve is to create the mappings that a > given device needs in order to transparently keep scanning out from an > address region that it's using, even when the kernel enables address > translation. Ok, makes sense. > In the case where you're trying to inherit the bootloader configuration > of the SMMU, how do you solve the problem of passing the page tables to > the kernel? You must have some way of reserving that memory in order to > prevent the kernel from reusing it. Maybe "inherit" makes it sound a lot more complicated than it is? Bjorn will know the details of what the bootloader sets up. But AFAICT, it looks like a simple "bypass"/identity mapping too. I don't think it actually sets up page tables. > > > One thing that was pointed out earlier, and which I don't have a good > > > idea on how to solve it, is that the early identity domain is not > > > discarded. The assumption is that the standard direct mappings code of > > > the IOMMU framework will replace the early identity domain once devices > > > are properly attached to domains, but we don't have a good point in time > > > when it would be safe to remove the early identity domain. > > > > You are in luck! I added sync_state() driver callbacks [2] exactly for > > cases like this. Heck, I even listed IOMMUs as an example use case. :) > > sync_state() works even with modules if one enables of_devlink [3] kernel > > parameter (which already supports iommus DT bindings). I'd be happy to > > answer any question you have on sync_state() and of_devlink. > > I wasn't aware of of_devlink, but I like it! It does have the drawback > that you need to reimplement a lot of the (phandle, specifier) parsing > code, but I don't think anybody was ever able to solve anyway. > > Looking at struct supplier_bindings, I think it might be possible to > share the property parsing code with the subsystems, though. But I > digress... Yeah, I don't want to digress either. But as of now, iommus are already supported. > Regarding sync_state(), I'm not sure it would be useful in my case. One > of the drivers I'm dealing with, for example, is a composite driver that > is created by tying together multiple devices. If you can give additional details about this, I can give a better answer. But with the limited info, there's one way I can think of to handle this. To make the explanation easier, let's call the device that references the IOMMU in DT as the "direct consumer device". The driver __probe function__ for the direct consumer device can add a device link from the composite device to the iommu device. This will ensure the iommu device doesn't get the sync_state() callback before the composite device probes. Keep in mind that devices do not need to be
Re: [RFC 0/2] iommu: arm-smmu: Add support for early direct mappings
Hi Thierry, I happened upon this thread while looking into another thread [1]. > From: Thierry Reding > > On some platforms, the firmware will setup hardware to read from a given > region of memory. One such example is a display controller that is > scanning out a splash screen from physical memory. > > During Linux' boot process, the ARM SMMU will configure all contexts to > fault by default. This means that memory accesses that happen by an SMMU > master before its driver has had a chance to properly set up the IOMMU > will cause a fault. This is especially annoying for something like the > display controller scanning out a splash screen because the faults will > result in the display controller getting bogus data (all-ones on Tegra) > and since it repeatedly scans that framebuffer, it will keep triggering > such faults and spam the boot log with them. While I'm not an expert on IOMMUs, I have a decent high level understanding of the problem you are trying to solve. > In order to work around such problems, scan the device tree for IOMMU > masters and set up a special identity domain that will map 1:1 all of > the reserved regions associated with them. This happens before the SMMU > is enabled, so that the mappings are already set up before translations > begin. I'm not sure if this RFC will solve the splash screen issue across SoCs ([1] seems to have a different issue and might not have memory-regions). > One thing that was pointed out earlier, and which I don't have a good > idea on how to solve it, is that the early identity domain is not > discarded. The assumption is that the standard direct mappings code of > the IOMMU framework will replace the early identity domain once devices > are properly attached to domains, but we don't have a good point in time > when it would be safe to remove the early identity domain. You are in luck! I added sync_state() driver callbacks [2] exactly for cases like this. Heck, I even listed IOMMUs as an example use case. :) sync_state() works even with modules if one enables of_devlink [3] kernel parameter (which already supports iommus DT bindings). I'd be happy to answer any question you have on sync_state() and of_devlink. > One option that I can think of would be to create an early identity > domain for each master and inherit it when that master is attached to > the domain later on, but that seems rather complicated from an book- > keeping point of view and tricky because we need to be careful not to > map regions twice, etc. > > Any good ideas on how to solve this? It'd also be interesting to see if > there's a more generic way of doing this. I know that something like > this isn't necessary on earlier Tegra SoCs with the custom Tegra SMMU > because translations are only enabled when the devices are attached to a > domain. Good foresight. As [1] shows, identity mapping doesn't solve it in a generic way. How about actually reading the current settings/mappings and just inheriting that instead of always doing a 1:1 identity mapping? And then those "inherited" mappings can be dropped when you get a sync_state(). What's wrong with that option? Cheers, Saravana [1] https://lore.kernel.org/linux-arm-msm/20200108091641.GA15147@willie-the-truck/ [2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/driver-api/driver-model/driver.rst#n172 [3] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/admin-guide/kernel-parameters.txt#n3239 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 09/14] iommu/arm-smmu: Prevent forced unbinding of Arm SMMU drivers
On Tue, Nov 26, 2019 at 1:13 AM John Garry wrote: > > On 21/11/2019 11:49, Will Deacon wrote: > > Forcefully unbinding the Arm SMMU drivers is a pretty dangerous operation, > > since it will likely lead to catastrophic failure for any DMA devices > > mastering through the SMMU being unbound. When the driver then attempts > > to "handle" the fatal faults, it's very easy to trip over dead data > > structures, leading to use-after-free. > > > > On John's machine, he reports that the machine was "unusable" due to > > loss of the storage controller following a forced unbind of the SMMUv3 > > driver: > > > >| # cd ./bus/platform/drivers/arm-smmu-v3 > >| # echo arm-smmu-v3.0.auto > unbind > >| hisi_sas_v2_hw HISI0162:01: CQE_AXI_W_ERR (0x800) found! > >| platform arm-smmu-v3.0.auto: CMD_SYNC timeout at 0x0146 > >| [hwprod 0x0146, hwcons 0x] > > > > Prevent this forced unbinding of the drivers by setting > > "suppress_bind_attrs" > > to true. > > This seems a reasonable approach for now. > > BTW, I'll give this series a spin this week, which again looks to be > your iommu/module branch, excluding the new IORT patch. Is this on a platform where of_devlink creates device links between the iommu device and its suppliers? I'm guessing no? Because device links should for unbinding of all the consumers before unbinding the supplier. Looks like it'll still allow the supplier to unbind if the consumers don't allow unbinding. Is that the case here? -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH] of: property: Add device link support for "iommu-map"
On Wed, Nov 20, 2019 at 11:00 AM Will Deacon wrote:
>
> Commit 8e12257dead7 ("of: property: Add device link support for iommus,
> mboxes and io-channels") added device link support for IOMMU linkages
> described using the "iommus" property. For PCI devices, this property
> is not present and instead the "iommu-map" property is used on the host
> bridge node to map the endpoint RequesterIDs to their corresponding
> IOMMU instance.
>
> Add support for "iommu-map" to the device link supplier bindings so that
> probing of PCI devices can be deferred until after the IOMMU is
> available.
>
> Cc: Greg Kroah-Hartman
> Cc: Rob Herring
> Cc: Saravana Kannan
> Cc: Robin Murphy
> Signed-off-by: Will Deacon
> ---
>
> Applies against driver-core/driver-core-next.
> Tested on AMD Seattle (arm64).
>
> drivers/of/property.c | 10 ++
> 1 file changed, 10 insertions(+)
>
> diff --git a/drivers/of/property.c b/drivers/of/property.c
> index 0fa04692e3cc..37e0d408430d 100644
> --- a/drivers/of/property.c
> +++ b/drivers/of/property.c
> @@ -1190,10 +1190,20 @@ DEFINE_SIMPLE_PROP(mboxes, "mboxes", "#mbox-cells")
> DEFINE_SIMPLE_PROP(io_channels, "io-channel", "#io-channel-cells")
> DEFINE_SUFFIX_PROP(regulators, "-supply", NULL)
>
> +static struct device_node *parse_iommu_maps(struct device_node *np,
> + const char *prop_name, int index)
> +{
> + if (strcmp(prop_name, "iommu-map"))
> + return NULL;
> +
> + return of_parse_phandle(np, prop_name, (index * 4) + 1);
> +}
> +
> static const struct supplier_bindings of_supplier_bindings[] = {
> { .parse_prop = parse_clocks, },
> { .parse_prop = parse_interconnects, },
> { .parse_prop = parse_iommus, },
> + { .parse_prop = parse_iommu_maps, },
> { .parse_prop = parse_mboxes, },
> { .parse_prop = parse_io_channels, },
> { .parse_prop = parse_regulators, },
Heh... a lot smaller than I thought this would be.
Acked-by: Saravana Kannan
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Wed, Oct 30, 2019 at 5:57 PM Saravana Kannan wrote:
>
> On Wed, Oct 30, 2019 at 8:54 AM Will Deacon wrote:
> >
> > Hi Robin,
> >
> > On Wed, Oct 30, 2019 at 03:35:55PM +, Robin Murphy wrote:
> > > On 30/10/2019 14:51, Will Deacon wrote:
> > > > As part of the work to enable a "Generic Kernel Image" across multiple
> > > > Android devices, there is a need to seperate shared, core kernel code
> > > > from modular driver code that may not be needed by all SoCs. This means
> > > > building IOMMU drivers as modules.
> > > >
> > > > It turns out that most of the groundwork has already been done to enable
> > > > the ARM SMMU drivers to be 'tristate' options in drivers/iommu/Kconfig;
> > > > with a few symbols exported from the IOMMU/PCI core, everything builds
> > > > nicely out of the box. The one exception is support for the legacy SMMU
> > > > DT binding, which is not in widespread use and has never worked with
> > > > modules, so we can simply remove that when building as a module rather
> > > > than try to paper over it with even more hacks.
> > > >
> > > > Obviously you need to be careful about using IOMMU drivers as modules,
> > > > since late loading of the driver for an IOMMU serving active DMA masters
> > > > is going to end badly in many cases. On Android, we're using device
> > > > links
> > > > to ensure that the IOMMU probes first.
> > >
> > > Out of curiosity, which device links are those? Clearly not the RPM links
> > > created by the IOMMU drivers themselves... Is this some special Android
> > > magic, or is there actually a chance of replacing all the
> > > of_iommu_configure() machinery with something more generic?
> >
> > I'll admit that I haven't used them personally yet, but I'm referring to
> > this series from Saravana [CC'd]:
> >
> > https://lore.kernel.org/linux-acpi/20190904211126.47518-1-sarava...@google.com/
> >
> > which is currently sitting in linux-next now that we're upstreaming the
> > "special Android magic" ;)
>
> Hi Robin,
>
> Actually, none of this is special Android magic. Will is talking about
> the of_devlink feature that's been merged into driver-core-next.
>
> A one line summary of of_devlink: the driver core + firmware (DT in
> this case) automatically add the device links during device addition
> based on the firmware properties of each device. The link that Will
> gave has more details.
>
> Wrt IOMMUs, the only missing piece in upstream is a trivial change
> that does something like this in drivers/of/property.c
>
> +static struct device_node *parse_iommus(struct device_node *np,
> +const char *prop_name, int index)
> +{
> +return parse_prop_cells(np, prop_name, index, "iommus",
> +"#iommu-cells");
> +}
>
> static const struct supplier_bindings of_supplier_bindings[] = {
> { .parse_prop = parse_clocks, },
> { .parse_prop = parse_interconnects, },
> { .parse_prop = parse_regulators, },
> +{ .parse_prop = parse_iommus, },
> {},
> };
>
> I plan to upstream this pretty soon, but I have other patches in
> flight that touch the same file and I'm waiting for those to get
> accepted. I also want to clean up the code a bit to reduce some
> repetition before I add support for more bindings.
As promised:
https://lore.kernel.org/lkml/20191105065000.50407-1-sarava...@google.com/
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Mon, Nov 4, 2019 at 5:29 AM Robin Murphy wrote:
>
> On 04/11/2019 12:16, John Garry wrote:
> > On 01/11/2019 21:13, Saravana Kannan wrote:
> >> On Fri, Nov 1, 2019 at 3:28 AM John Garry wrote:
> >>>
> >>> On 31/10/2019 23:34, Saravana Kannan via iommu wrote:
> >>>> I looked into the iommu-map property and it shouldn't be too hard to
> >>>> add support for it. Looks like we can simply hold off on probing the
> >>>> root bridge device till all the iommus in its iommu-map are probed and
> >>>> we should be fine.
> >>>>
> >>>>> I'm also unsure about distro vendors agreeing to a mandatory kernel
> >>>>> parameter (of_devlink). Do you plan to eventually enable it by
> >>>>> default?
> >>>>>
> >>>>>> static const struct supplier_bindings of_supplier_bindings[] = {
> >>>>>> { .parse_prop = parse_clocks, },
> >>>>>> { .parse_prop = parse_interconnects, },
> >>>>>> { .parse_prop = parse_regulators, },
> >>>>>> +{ .parse_prop = parse_iommus, },
> >>>>>> {},
> >>>>>> };
> >>>>>>
> >>>>>> I plan to upstream this pretty soon, but I have other patches in
> >>>>>> flight that touch the same file and I'm waiting for those to get
> >>>>>> accepted. I also want to clean up the code a bit to reduce some
> >>>>>> repetition before I add support for more bindings.
> >>>>> I'm also wondering about ACPI support.
> >>>> I'd love to add ACPI support too, but I have zero knowledge of ACPI.
> >>>> I'd be happy to help anyone who wants to add ACPI support that allows
> >>>> ACPI to add device links.
> >>>
> >>> If possible to add, that may be useful for remedying this:
> >>>
> >>> https://lore.kernel.org/linux-iommu/9625faf4-48ef-2dd3-d82f-931d9cf26...@huawei.com/
> >>>
> >>
> >> I'm happy that this change might fix that problem, but isn't the
> >> problem reported in that thread more to do with child devices getting
> >> added before the parent probes successfully? That doesn't make sense
> >> to me.
> >
> > So the pcieport device and then the child device are added in the PCI
> > scan, but only some time later do the device drivers probe for these
> > devices; so it's not that the that pcieport driver creates the child
> > device.
> >
> > The problem then occurs in that the ordering the of device driver probe
> > is such that we have this: pcieport probe + defer (as no IOMMU group
> > registered), SMMU probe (registers the IOMMU group), child device probe,
> > pcieport really probe.
> >
> > Can't the piceport driver not add its child devices before it
> >> probes successfully? Or more specifically, who adds the child devices
> >> of the pcieport before the pcieport itself probes?
> >
> > The devices are actually added in order pcieport, child device, but not
> > really probed in that same order, as above.
>
> Right, in short the fundamental problem is that of_iommu_configure() now
> does the wrong thing. Deferring probe of the entire host bridge/root
> complex based on "iommu-map" would indeed happen to solve the problem by
> brute force, I think, but could lead to a dependency cycle for PCI-based
> IOMMUs as Jean points out.
Sorry for the late reply. Got caught up on other work.
I didn't think the SMMU itself was PCI based in the example Jean gave.
I thought it just happened to be the case where the SMMU probes after
the pcieport but before the other children. If the SMMU itself is a
child of the pcieport, how can it be required for the parent to
function? How will suspend/resume even work?! I feel like I'm missing
some context wrt to PCI that everyone else seems to know (which isn't
surprising).
> I hope to have time this week to work a bit
> more on pulling of_iommu_configure() apart to fix it properly, after
> which of_devlink *should* only have to worry about the child devices
> themselves...
Worry about child devices in what sense? From a non-iommu perspective?
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Mon, Nov 4, 2019 at 4:16 AM John Garry wrote:
>
> On 01/11/2019 21:13, Saravana Kannan wrote:
> > On Fri, Nov 1, 2019 at 3:28 AM John Garry wrote:
> >>
> >> On 31/10/2019 23:34, Saravana Kannan via iommu wrote:
> >>> I looked into the iommu-map property and it shouldn't be too hard to
> >>> add support for it. Looks like we can simply hold off on probing the
> >>> root bridge device till all the iommus in its iommu-map are probed and
> >>> we should be fine.
> >>>
> >>>> I'm also unsure about distro vendors agreeing to a mandatory kernel
> >>>> parameter (of_devlink). Do you plan to eventually enable it by default?
> >>>>
> >>>>> static const struct supplier_bindings of_supplier_bindings[] = {
> >>>>> { .parse_prop = parse_clocks, },
> >>>>> { .parse_prop = parse_interconnects, },
> >>>>> { .parse_prop = parse_regulators, },
> >>>>> +{ .parse_prop = parse_iommus, },
> >>>>> {},
> >>>>> };
> >>>>>
> >>>>> I plan to upstream this pretty soon, but I have other patches in
> >>>>> flight that touch the same file and I'm waiting for those to get
> >>>>> accepted. I also want to clean up the code a bit to reduce some
> >>>>> repetition before I add support for more bindings.
> >>>> I'm also wondering about ACPI support.
> >>> I'd love to add ACPI support too, but I have zero knowledge of ACPI.
> >>> I'd be happy to help anyone who wants to add ACPI support that allows
> >>> ACPI to add device links.
> >>
> >> If possible to add, that may be useful for remedying this:
> >>
> >> https://lore.kernel.org/linux-iommu/9625faf4-48ef-2dd3-d82f-931d9cf26...@huawei.com/
> >
> > I'm happy that this change might fix that problem, but isn't the
> > problem reported in that thread more to do with child devices getting
> > added before the parent probes successfully? That doesn't make sense
> > to me.
>
> So the pcieport device and then the child device are added in the PCI
> scan, but only some time later do the device drivers probe for these
> devices; so it's not that the that pcieport driver creates the child device.
My lack of PCI knowledge might be showing, but without the pcieport
device itself being functional (I'm assuming this is the PCIe
controller) how does a PCI scan work? In anyway, I understand that all
devices are added at the same time as of today.
> The problem then occurs in that the ordering the of device driver probe
> is such that we have this: pcieport probe + defer (as no IOMMU group
> registered), SMMU probe (registers the IOMMU group), child device probe,
> pcieport really probe.
>
> Can't the piceport driver not add its child devices before it
> > probes successfully? Or more specifically, who adds the child devices
> > of the pcieport before the pcieport itself probes?
>
> The devices are actually added in order pcieport, child device, but not
> really probed in that same order, as above.
Thanks. Got it.
I guess one nice thing that came out of devicetree world is that the
child devices are never probed without the parent probing first (I'm
ignoring the simple-bus devices). It also kinda makes sense to me --
without a USB port device being added and probed, it doesn't make
sense to add the (keyboard, mouse, USB-HDD, etc) devices connected to
the USB port. But this is an orthogonal debate.
-Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Mon, Nov 4, 2019 at 3:43 AM Lorenzo Pieralisi wrote: > > On Fri, Nov 01, 2019 at 02:26:05PM -0700, Saravana Kannan wrote: > > On Fri, Nov 1, 2019 at 5:28 AM Lorenzo Pieralisi > > wrote: > > > > > > On Fri, Nov 01, 2019 at 12:41:48PM +0100, Jean-Philippe Brucker wrote: > > > > > > [...] > > > > > > > > > I'm also wondering about ACPI support. > > > > > > > > > > I'd love to add ACPI support too, but I have zero knowledge of ACPI. > > > > > I'd be happy to help anyone who wants to add ACPI support that allows > > > > > ACPI to add device links. > > > > > > > > It's not as generic as device-tree, each vendor has their own table to > > > > describe the IOMMU topology. I don't see a nice way to transpose the > > > > add_links() callback there. Links need to be created either in a common > > > > path (iommu_probe_device()) or in the APCI IORT driver. > > > > > > We can create a generic stub that calls into respective firmware > > > handling paths (eg iort_dma_setup() in acpi_dma_configure()). > > > > > > There are three arches booting with ACPI so stubbing it out in > > > specific firmware handlers is not such a big deal, less generic > > > sure, but not catastrophically bad. > > > > Ok, good to know. > > > > > Obviously this works for IOMMU masters links > > > > It's unclear to me what you are referring to here and it's throwing me > > off on the rest of the email. > > > > Did you mean to say "IOMMU master's links"? As in the bus masters > > whose accesses go through IOMMUs? And "links" as in device links? > > > > OR > > > > Do you mean device links from bus master devices to IOMMUs here? > > I meant associating endpoints devices to the IOMMU they are connected > to. > > In DT you do it through "iommus", "iommu-map" properties, in ACPI > it is arch specific, doable nonetheless through ACPI (IORT on ARM) > static tables data. > > > > - for resources > > > dependencies (eg power domains) it deserves some thought, keeping in > > > mind that IOMMUs are static table entries in ACPI and not device objects > > > so they are not even capable of expressing eg power resources and > > > suchlike. > > > > If you can reword this sentence for me with more context or split it > > into separate sentences, I'd appreciate that very much. I'd help me > > understand this better and allow me to try to help out. > > In ACPI (at least on ARM but on x86 I suspect that's the same story with > the DMAR table) an SMMU is presented in FW as an entry in a static > table (eg IORT on ARM). I noticed that your patch series takes into > account for instance eg clock dependencies in DT; this way the OS knows > the clock(s) the SMMU depends on to be activated. > > In ACPI there is not a notion of "clock" (hopefully - unless someone > sneaked that in using _DSD properties) but rather every device in the > ACPI namespace (which is part of tables containing code that needs the > ACPI interpreter to be used such as SSDT/DSDT - it is AML code) has ACPI > objects describing power resources (ie ACPI specification 6.3, 7.2). > > The SMMU, since it is not itself an ACPI object in the ACPI namespace > but rather an entry in a static ACPI table (IORT on ARM), can't have > PowerResource object in it which means that at the moment there is no > way you can detect a dependency on other power resources to be ON to > build the device links you require to sort out the probe dependencies, > which I *assume* that's the reason why you require to detect > clock dependencies in DT. > > Maybe it is not even needed at all but in case it is I was giving > a heads-up to say that clocks (or rather an all encompassing "power > resource" dependency) dependencies in ACPI to build an SMMU as > a module are not straightforward and most certainly will require > firmware specifications updates. > > *Hopefully* in the short term all you need to detect is how endpoint > devices are connected to an IOMMU and build device links to describe > that probe dependency, if we need to throw power management into > the picture there is more work to be done. > > I hope that's clearer, if it is not please let me know and I will > try to be more precise. Thanks for the detailed explanation and context Lorenzo. I understand it better now. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Fri, Nov 1, 2019 at 5:28 AM Lorenzo Pieralisi wrote: > > On Fri, Nov 01, 2019 at 12:41:48PM +0100, Jean-Philippe Brucker wrote: > > [...] > > > > > I'm also wondering about ACPI support. > > > > > > I'd love to add ACPI support too, but I have zero knowledge of ACPI. > > > I'd be happy to help anyone who wants to add ACPI support that allows > > > ACPI to add device links. > > > > It's not as generic as device-tree, each vendor has their own table to > > describe the IOMMU topology. I don't see a nice way to transpose the > > add_links() callback there. Links need to be created either in a common > > path (iommu_probe_device()) or in the APCI IORT driver. > > We can create a generic stub that calls into respective firmware > handling paths (eg iort_dma_setup() in acpi_dma_configure()). > > There are three arches booting with ACPI so stubbing it out in > specific firmware handlers is not such a big deal, less generic > sure, but not catastrophically bad. Ok, good to know. > Obviously this works for IOMMU masters links It's unclear to me what you are referring to here and it's throwing me off on the rest of the email. Did you mean to say "IOMMU master's links"? As in the bus masters whose accesses go through IOMMUs? And "links" as in device links? OR Do you mean device links from bus master devices to IOMMUs here? > - for resources > dependencies (eg power domains) it deserves some thought, keeping in > mind that IOMMUs are static table entries in ACPI and not device objects > so they are not even capable of expressing eg power resources and > suchlike. If you can reword this sentence for me with more context or split it into separate sentences, I'd appreciate that very much. I'd help me understand this better and allow me to try to help out. > Long story short: adding IOMMU masters links in ACPI should be > reasonably simple, everything else requires further thought. -Saravana ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Fri, Nov 1, 2019 at 3:28 AM John Garry wrote:
>
> On 31/10/2019 23:34, Saravana Kannan via iommu wrote:
> > I looked into the iommu-map property and it shouldn't be too hard to
> > add support for it. Looks like we can simply hold off on probing the
> > root bridge device till all the iommus in its iommu-map are probed and
> > we should be fine.
> >
> >> I'm also unsure about distro vendors agreeing to a mandatory kernel
> >> parameter (of_devlink). Do you plan to eventually enable it by default?
> >>
> >>> static const struct supplier_bindings of_supplier_bindings[] = {
> >>> { .parse_prop = parse_clocks, },
> >>> { .parse_prop = parse_interconnects, },
> >>> { .parse_prop = parse_regulators, },
> >>> +{ .parse_prop = parse_iommus, },
> >>> {},
> >>> };
> >>>
> >>> I plan to upstream this pretty soon, but I have other patches in
> >>> flight that touch the same file and I'm waiting for those to get
> >>> accepted. I also want to clean up the code a bit to reduce some
> >>> repetition before I add support for more bindings.
> >> I'm also wondering about ACPI support.
> > I'd love to add ACPI support too, but I have zero knowledge of ACPI.
> > I'd be happy to help anyone who wants to add ACPI support that allows
> > ACPI to add device links.
>
> If possible to add, that may be useful for remedying this:
>
> https://lore.kernel.org/linux-iommu/9625faf4-48ef-2dd3-d82f-931d9cf26...@huawei.com/
I'm happy that this change might fix that problem, but isn't the
problem reported in that thread more to do with child devices getting
added before the parent probes successfully? That doesn't make sense
to me. Can't the piceport driver not add its child devices before it
probes successfully? Or more specifically, who adds the child devices
of the pcieport before the pcieport itself probes?
Thanks,
Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Thu, Oct 31, 2019 at 12:38 PM Jean-Philippe Brucker
wrote:
>
> Hi Saravana, Will,
>
> On Wed, Oct 30, 2019 at 05:57:44PM -0700, Saravana Kannan via iommu wrote:
> > > > > Obviously you need to be careful about using IOMMU drivers as modules,
> > > > > since late loading of the driver for an IOMMU serving active DMA
> > > > > masters
> > > > > is going to end badly in many cases. On Android, we're using device
> > > > > links
> > > > > to ensure that the IOMMU probes first.
> > > >
> > > > Out of curiosity, which device links are those? Clearly not the RPM
> > > > links
> > > > created by the IOMMU drivers themselves... Is this some special Android
> > > > magic, or is there actually a chance of replacing all the
> > > > of_iommu_configure() machinery with something more generic?
> > >
> > > I'll admit that I haven't used them personally yet, but I'm referring to
> > > this series from Saravana [CC'd]:
> > >
> > > https://lore.kernel.org/linux-acpi/20190904211126.47518-1-sarava...@google.com/
> > >
> > > which is currently sitting in linux-next now that we're upstreaming the
> > > "special Android magic" ;)
>
> Neat, I'm trying to do the same for virtio-iommu. It needs to be modular
> because it depends on the virtio transport, which distributions usually
> build as a module. So far I've been managing the device links in
> virtio-iommu's add_device() and remove_device() callbacks [1]. Since it
> relies on the existing probe deferral, I had to make a special case for
> virtio-iommu to avoid giving up after initcalls_done [2].
>
> Currently buggy, it explodes on the second modprobe.
>
> [1]
> http://jpbrucker.net/git/linux/commit/?h=virtio-iommu/module-2019-10-31=f72978be18cb52eaa2d46dc762711bacbfab5039
> [2]
> http://jpbrucker.net/git/linux/commit/?h=virtio-iommu/module-2019-10-31=f5fe188bb7fde33422ef08b9aad956dc3c77ec39
>
> [...]
> > Wrt IOMMUs, the only missing piece in upstream is a trivial change
> > that does something like this in drivers/of/property.c
> >
> > +static struct device_node *parse_iommus(struct device_node *np,
> > +const char *prop_name, int index)
> > +{
> > +return parse_prop_cells(np, prop_name, index, "iommus",
> > +"#iommu-cells");
> > +}
>
> The 'iommus' property only applies to platform devices,
An early version of this patch series was limited to platform device,
but that's not true with the version that Will pointed to and was
merged into driver-core-next. The iommu parsing and creating device
links applies to all devices that use DT. That's why this code is in
of/property.c opposed to of/platform.c.
> do you have any
> plan for PCI? PCI devices generally don't have a DT node. Only their root
> bridge has a node, with an 'iommu-map' property instead of 'iommus', so
> I don't think add_links() would get called for them.
I looked into the iommu-map property and it shouldn't be too hard to
add support for it. Looks like we can simply hold off on probing the
root bridge device till all the iommus in its iommu-map are probed and
we should be fine.
> I'm also unsure about distro vendors agreeing to a mandatory kernel
> parameter (of_devlink). Do you plan to eventually enable it by default?
>
> > static const struct supplier_bindings of_supplier_bindings[] = {
> > { .parse_prop = parse_clocks, },
> > { .parse_prop = parse_interconnects, },
> > { .parse_prop = parse_regulators, },
> > +{ .parse_prop = parse_iommus, },
> > {},
> > };
> >
> > I plan to upstream this pretty soon, but I have other patches in
> > flight that touch the same file and I'm waiting for those to get
> > accepted. I also want to clean up the code a bit to reduce some
> > repetition before I add support for more bindings.
>
> I'm also wondering about ACPI support.
I'd love to add ACPI support too, but I have zero knowledge of ACPI.
I'd be happy to help anyone who wants to add ACPI support that allows
ACPI to add device links.
> IOMMU already has a sort of
> canonical code path that links endpoints to their IOMMU
> (iommu_probe_device()), after the firmware descriptions have been parsed.
> So if we created the device links in the iommu core, for example
> iommu_bus_notifier(), we would support all firmware interface flavors.
> Otherwise we'll have to create those device links in the IORT driver as
> well (plus DMAR and IVRS if they want it).
IOMMU driver/framework or whoever else ca
Re: [PATCH 0/7] iommu: Permit modular builds of ARM SMMU[v3] drivers
On Wed, Oct 30, 2019 at 8:54 AM Will Deacon wrote:
>
> Hi Robin,
>
> On Wed, Oct 30, 2019 at 03:35:55PM +, Robin Murphy wrote:
> > On 30/10/2019 14:51, Will Deacon wrote:
> > > As part of the work to enable a "Generic Kernel Image" across multiple
> > > Android devices, there is a need to seperate shared, core kernel code
> > > from modular driver code that may not be needed by all SoCs. This means
> > > building IOMMU drivers as modules.
> > >
> > > It turns out that most of the groundwork has already been done to enable
> > > the ARM SMMU drivers to be 'tristate' options in drivers/iommu/Kconfig;
> > > with a few symbols exported from the IOMMU/PCI core, everything builds
> > > nicely out of the box. The one exception is support for the legacy SMMU
> > > DT binding, which is not in widespread use and has never worked with
> > > modules, so we can simply remove that when building as a module rather
> > > than try to paper over it with even more hacks.
> > >
> > > Obviously you need to be careful about using IOMMU drivers as modules,
> > > since late loading of the driver for an IOMMU serving active DMA masters
> > > is going to end badly in many cases. On Android, we're using device links
> > > to ensure that the IOMMU probes first.
> >
> > Out of curiosity, which device links are those? Clearly not the RPM links
> > created by the IOMMU drivers themselves... Is this some special Android
> > magic, or is there actually a chance of replacing all the
> > of_iommu_configure() machinery with something more generic?
>
> I'll admit that I haven't used them personally yet, but I'm referring to
> this series from Saravana [CC'd]:
>
> https://lore.kernel.org/linux-acpi/20190904211126.47518-1-sarava...@google.com/
>
> which is currently sitting in linux-next now that we're upstreaming the
> "special Android magic" ;)
Hi Robin,
Actually, none of this is special Android magic. Will is talking about
the of_devlink feature that's been merged into driver-core-next.
A one line summary of of_devlink: the driver core + firmware (DT in
this case) automatically add the device links during device addition
based on the firmware properties of each device. The link that Will
gave has more details.
Wrt IOMMUs, the only missing piece in upstream is a trivial change
that does something like this in drivers/of/property.c
+static struct device_node *parse_iommus(struct device_node *np,
+const char *prop_name, int index)
+{
+return parse_prop_cells(np, prop_name, index, "iommus",
+"#iommu-cells");
+}
static const struct supplier_bindings of_supplier_bindings[] = {
{ .parse_prop = parse_clocks, },
{ .parse_prop = parse_interconnects, },
{ .parse_prop = parse_regulators, },
+{ .parse_prop = parse_iommus, },
{},
};
I plan to upstream this pretty soon, but I have other patches in
flight that touch the same file and I'm waiting for those to get
accepted. I also want to clean up the code a bit to reduce some
repetition before I add support for more bindings.
Thanks,
Saravana
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
