Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Sun, Aug 30, 2015 at 11:11 AM, Linus Torvalds
 wrote:
> On Sun, Aug 30, 2015 at 1:25 AM, Arend van Spriel  wrote:
>> On 08/29/2015 12:38 PM, Ming Lei wrote:
>>
>> Does this mean a built-in driver can not get firmware from initramfs or
>> built in the kernel early. Seems a bit too aggressive.
>
> Yeah, that seems wrong. Loading firmware from initramfs is required
> for some things, like disk drivers. Of course, depending on how it's
> done, it's all after the SYSTEM_BOOTING phase, but ..
>
> What we *might* do is to not allow it for the user-mode helper
> fallback,

FWIW, that's what we did, request_firmware_direct() now skips the
silly usermode helper. I'll note that Greg pointed out to me that
Daniel (was this right?) might have some use cases for the usermode
helper in the future on graphics though. Daniel is that right? Can you
clarify the use case, I'd just like to document this and keep it in
mind for future design changes on firmware_class. Also, it occurs to
me that if you have a need for it, perhaps others might and if we can
avoid *others* from coming up with their own solution that'd be best,
specifically as this is related to file loading -- more on this later
generalized possible use cases in another thread I'll Cc you folks on.

> but I think it's more likely that we'll just deprecate the
> usermode helper fw loader entirely, so adding new error cases for it
> seems pointless.

I was shooting hard to deprecate the usermodehelper, Greg has noted
that we can only phase it out though, so that is what I will be
shooting for: a sysdata API, what will have firmware signing support
later will *not* make use of the usermode helper. The old APIs will
still use it.

[0] http://lkml.kernel.org/r/20151006090821.gb9...@kroah.com

 Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Sun, Aug 30, 2015 at 11:11 AM, Linus Torvalds
 wrote:
> On Sun, Aug 30, 2015 at 1:25 AM, Arend van Spriel  wrote:
>> On 08/29/2015 12:38 PM, Ming Lei wrote:
>>
>> Does this mean a built-in driver can not get firmware from initramfs or
>> built in the kernel early. Seems a bit too aggressive.
>
> Yeah, that seems wrong. Loading firmware from initramfs is required
> for some things, like disk drivers. Of course, depending on how it's
> done, it's all after the SYSTEM_BOOTING phase, but ..
>
> What we *might* do is to not allow it for the user-mode helper
> fallback,

FWIW, that's what we did, request_firmware_direct() now skips the
silly usermode helper. I'll note that Greg pointed out to me that
Daniel (was this right?) might have some use cases for the usermode
helper in the future on graphics though. Daniel is that right? Can you
clarify the use case, I'd just like to document this and keep it in
mind for future design changes on firmware_class. Also, it occurs to
me that if you have a need for it, perhaps others might and if we can
avoid *others* from coming up with their own solution that'd be best,
specifically as this is related to file loading -- more on this later
generalized possible use cases in another thread I'll Cc you folks on.

> but I think it's more likely that we'll just deprecate the
> usermode helper fw loader entirely, so adding new error cases for it
> seems pointless.

I was shooting hard to deprecate the usermodehelper, Greg has noted
that we can only phase it out though, so that is what I will be
shooting for: a sysdata API, what will have firmware signing support
later will *not* make use of the usermode helper. The old APIs will
still use it.

[0] http://lkml.kernel.org/r/20151006090821.gb9...@kroah.com

 Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 10/16/2015 09:35 PM, Luis R. Rodriguez wrote:

On Thu, Sep 03, 2015 at 10:33:51AM -0700, Dmitry Torokhov wrote:

On Thu, Sep 3, 2015 at 10:23 AM, Arend van Spriel  wrote:

On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:


On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
 wrote:


On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez 
wrote:

IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
preemptively hacky. Sadly I can't think of clear and clever way for the
kernel
to know when firmware will be ready either...  Would userspace know?
Should the
kernel learn this from userspace ?



Yes. Given only userspace knows when firmware is available (I could
have it on a separate device and mount it at some time). So maybe
userpsace should simply try and scan busses for unbound devices and
tell them to re-probe when it decides that firmware is finally
available.



OK, the folks wanting this mechanism can implement it then. Short of
that we only have hacks.



So what does "userspace knows when firmware is available" mean here. The
specific firmware file the driver wants or the collection of firmware files
which may or may not have the specific firmware file the driver wants. I
assume the latter and re-probe will fail as expected.


Right, the latter.


Arend, curious are you working on this? Me and Julia did some hunting and
have found quite a bit of users that could use this. I'll provide results
in another thread but figured I'd follow up to see if anyone is working
to address this. Otherwise we just have hacks for now.


Not doing much else than diaper changes these days. Seems to me based on 
the above that from kernel perspective we can not do much when firmware 
mounts are controlled by user-space.


Regards,
Arend

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 10/16/2015 09:35 PM, Luis R. Rodriguez wrote:

On Thu, Sep 03, 2015 at 10:33:51AM -0700, Dmitry Torokhov wrote:

On Thu, Sep 3, 2015 at 10:23 AM, Arend van Spriel  wrote:

On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:


On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
 wrote:


On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez 
wrote:

IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
preemptively hacky. Sadly I can't think of clear and clever way for the
kernel
to know when firmware will be ready either...  Would userspace know?
Should the
kernel learn this from userspace ?



Yes. Given only userspace knows when firmware is available (I could
have it on a separate device and mount it at some time). So maybe
userpsace should simply try and scan busses for unbound devices and
tell them to re-probe when it decides that firmware is finally
available.



OK, the folks wanting this mechanism can implement it then. Short of
that we only have hacks.



So what does "userspace knows when firmware is available" mean here. The
specific firmware file the driver wants or the collection of firmware files
which may or may not have the specific firmware file the driver wants. I
assume the latter and re-probe will fail as expected.


Right, the latter.


Arend, curious are you working on this? Me and Julia did some hunting and
have found quite a bit of users that could use this. I'll provide results
in another thread but figured I'd follow up to see if anyone is working
to address this. Otherwise we just have hacks for now.


Not doing much else than diaper changes these days. Seems to me based on 
the above that from kernel perspective we can not do much when firmware 
mounts are controlled by user-space.


Regards,
Arend

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Fri, Oct 16, 2015 at 09:35:12PM +0200, Luis R. Rodriguez wrote:
> Arend, curious are you working on this? Me and Julia did some hunting and
> have found quite a bit of users that could use this. I'll provide results
> in another thread but figured I'd follow up to see if anyone is working
> to address this. Otherwise we just have hacks for now.

A lot of things are going on with firmware_class enhancements so lets
jot things down here:

http://kernelnewbies.org/KernelProjects/firmware-class-enhancements

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Thu, Sep 03, 2015 at 10:33:51AM -0700, Dmitry Torokhov wrote:
> On Thu, Sep 3, 2015 at 10:23 AM, Arend van Spriel  wrote:
> > On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:
> >>
> >> On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
> >>  wrote:
> >>>
> >>> On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez 
> >>> wrote:
>  IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
>  preemptively hacky. Sadly I can't think of clear and clever way for the
>  kernel
>  to know when firmware will be ready either...  Would userspace know?
>  Should the
>  kernel learn this from userspace ?
> >>>
> >>>
> >>> Yes. Given only userspace knows when firmware is available (I could
> >>> have it on a separate device and mount it at some time). So maybe
> >>> userpsace should simply try and scan busses for unbound devices and
> >>> tell them to re-probe when it decides that firmware is finally
> >>> available.
> >>
> >>
> >> OK, the folks wanting this mechanism can implement it then. Short of
> >> that we only have hacks.
> >
> >
> > So what does "userspace knows when firmware is available" mean here. The
> > specific firmware file the driver wants or the collection of firmware files
> > which may or may not have the specific firmware file the driver wants. I
> > assume the latter and re-probe will fail as expected.
> 
> Right, the latter.

Arend, curious are you working on this? Me and Julia did some hunting and
have found quite a bit of users that could use this. I'll provide results
in another thread but figured I'd follow up to see if anyone is working
to address this. Otherwise we just have hacks for now.

 Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Thu, Sep 03, 2015 at 10:33:51AM -0700, Dmitry Torokhov wrote:
> On Thu, Sep 3, 2015 at 10:23 AM, Arend van Spriel  wrote:
> > On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:
> >>
> >> On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
> >>  wrote:
> >>>
> >>> On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez 
> >>> wrote:
>  IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
>  preemptively hacky. Sadly I can't think of clear and clever way for the
>  kernel
>  to know when firmware will be ready either...  Would userspace know?
>  Should the
>  kernel learn this from userspace ?
> >>>
> >>>
> >>> Yes. Given only userspace knows when firmware is available (I could
> >>> have it on a separate device and mount it at some time). So maybe
> >>> userpsace should simply try and scan busses for unbound devices and
> >>> tell them to re-probe when it decides that firmware is finally
> >>> available.
> >>
> >>
> >> OK, the folks wanting this mechanism can implement it then. Short of
> >> that we only have hacks.
> >
> >
> > So what does "userspace knows when firmware is available" mean here. The
> > specific firmware file the driver wants or the collection of firmware files
> > which may or may not have the specific firmware file the driver wants. I
> > assume the latter and re-probe will fail as expected.
> 
> Right, the latter.

Arend, curious are you working on this? Me and Julia did some hunting and
have found quite a bit of users that could use this. I'll provide results
in another thread but figured I'd follow up to see if anyone is working
to address this. Otherwise we just have hacks for now.

 Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Fri, Oct 16, 2015 at 09:35:12PM +0200, Luis R. Rodriguez wrote:
> Arend, curious are you working on this? Me and Julia did some hunting and
> have found quite a bit of users that could use this. I'll provide results
> in another thread but figured I'd follow up to see if anyone is working
> to address this. Otherwise we just have hacks for now.

A lot of things are going on with firmware_class enhancements so lets
jot things down here:

http://kernelnewbies.org/KernelProjects/firmware-class-enhancements

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Thu, Sep 3, 2015 at 10:23 AM, Arend van Spriel  wrote:
> On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:
>>
>> On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
>>  wrote:
>>>
>>> On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez 
>>> wrote:
 IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
 preemptively hacky. Sadly I can't think of clear and clever way for the
 kernel
 to know when firmware will be ready either...  Would userspace know?
 Should the
 kernel learn this from userspace ?
>>>
>>>
>>> Yes. Given only userspace knows when firmware is available (I could
>>> have it on a separate device and mount it at some time). So maybe
>>> userpsace should simply try and scan busses for unbound devices and
>>> tell them to re-probe when it decides that firmware is finally
>>> available.
>>
>>
>> OK, the folks wanting this mechanism can implement it then. Short of
>> that we only have hacks.
>
>
> So what does "userspace knows when firmware is available" mean here. The
> specific firmware file the driver wants or the collection of firmware files
> which may or may not have the specific firmware file the driver wants. I
> assume the latter and re-probe will fail as expected.

Right, the latter.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:

On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
 wrote:

On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez  wrote:

On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:

On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:

Ok. So some background why we need it in brcm80211 drivers. So as a wireless
network device driver the answer we got when asking for an event to load
firware is upon IF_UP for a registered net device. Because we try to do
things smart we query the firmware running on the device for capabilities
before we can register the net device hence we request the firmware during
probe. This may be specific to wireless drivers (Intel has same approach if
not mistaken) but I suspect there may be more.


We have the same issue with input devices: before we can register one
we need to set their capabilities and to know their capabilities we
quite often need to load their firmware/config and query the device.


Should Arend's driver use async probe then?


What has async probe have to do with anything? We will still be
waiting for async probes to finish before we mount rootfs so it will
not change absolutely anything.


:) Right, its what I was alluding to as well.


Indeed. However, upon module_init we schedule a worker in which the 
driver are registered. We do that to make sure the probe is not done 
within module_init context. That could now be done with async probe. 
This is not the problem discussed here so let's not spend more time on this.



IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
preemptively hacky. Sadly I can't think of clear and clever way for the kernel
to know when firmware will be ready either...  Would userspace know? Should the
kernel learn this from userspace ?


Yes. Given only userspace knows when firmware is available (I could
have it on a separate device and mount it at some time). So maybe
userpsace should simply try and scan busses for unbound devices and
tell them to re-probe when it decides that firmware is finally
available.


OK, the folks wanting this mechanism can implement it then. Short of
that we only have hacks.


So what does "userspace knows when firmware is available" mean here. The 
specific firmware file the driver wants or the collection of firmware 
files which may or may not have the specific firmware file the driver 
wants. I assume the latter and re-probe will fail as expected.


Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:

On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
 wrote:

On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez  wrote:

On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:

On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:

Ok. So some background why we need it in brcm80211 drivers. So as a wireless
network device driver the answer we got when asking for an event to load
firware is upon IF_UP for a registered net device. Because we try to do
things smart we query the firmware running on the device for capabilities
before we can register the net device hence we request the firmware during
probe. This may be specific to wireless drivers (Intel has same approach if
not mistaken) but I suspect there may be more.


We have the same issue with input devices: before we can register one
we need to set their capabilities and to know their capabilities we
quite often need to load their firmware/config and query the device.


Should Arend's driver use async probe then?


What has async probe have to do with anything? We will still be
waiting for async probes to finish before we mount rootfs so it will
not change absolutely anything.


:) Right, its what I was alluding to as well.


Indeed. However, upon module_init we schedule a worker in which the 
driver are registered. We do that to make sure the probe is not done 
within module_init context. That could now be done with async probe. 
This is not the problem discussed here so let's not spend more time on this.



IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
preemptively hacky. Sadly I can't think of clear and clever way for the kernel
to know when firmware will be ready either...  Would userspace know? Should the
kernel learn this from userspace ?


Yes. Given only userspace knows when firmware is available (I could
have it on a separate device and mount it at some time). So maybe
userpsace should simply try and scan busses for unbound devices and
tell them to re-probe when it decides that firmware is finally
available.


OK, the folks wanting this mechanism can implement it then. Short of
that we only have hacks.


So what does "userspace knows when firmware is available" mean here. The 
specific firmware file the driver wants or the collection of firmware 
files which may or may not have the specific firmware file the driver 
wants. I assume the latter and re-probe will fail as expected.


Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Thu, Sep 3, 2015 at 10:23 AM, Arend van Spriel  wrote:
> On 09/03/2015 01:46 AM, Luis R. Rodriguez wrote:
>>
>> On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
>>  wrote:
>>>
>>> On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez 
>>> wrote:
 IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
 preemptively hacky. Sadly I can't think of clear and clever way for the
 kernel
 to know when firmware will be ready either...  Would userspace know?
 Should the
 kernel learn this from userspace ?
>>>
>>>
>>> Yes. Given only userspace knows when firmware is available (I could
>>> have it on a separate device and mount it at some time). So maybe
>>> userpsace should simply try and scan busses for unbound devices and
>>> tell them to re-probe when it decides that firmware is finally
>>> available.
>>
>>
>> OK, the folks wanting this mechanism can implement it then. Short of
>> that we only have hacks.
>
>
> So what does "userspace knows when firmware is available" mean here. The
> specific firmware file the driver wants or the collection of firmware files
> which may or may not have the specific firmware file the driver wants. I
> assume the latter and re-probe will fail as expected.

Right, the latter.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
 wrote:
> On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez  wrote:
>> On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
>>> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
>>> > Ok. So some background why we need it in brcm80211 drivers. So as a 
>>> > wireless
>>> > network device driver the answer we got when asking for an event to load
>>> > firware is upon IF_UP for a registered net device. Because we try to do
>>> > things smart we query the firmware running on the device for capabilities
>>> > before we can register the net device hence we request the firmware during
>>> > probe. This may be specific to wireless drivers (Intel has same approach 
>>> > if
>>> > not mistaken) but I suspect there may be more.
>>>
>>> We have the same issue with input devices: before we can register one
>>> we need to set their capabilities and to know their capabilities we
>>> quite often need to load their firmware/config and query the device.
>>
>> Should Arend's driver use async probe then?
>
> What has async probe have to do with anything? We will still be
> waiting for async probes to finish before we mount rootfs so it will
> not change absolutely anything.

:) Right, its what I was alluding to as well.

>> IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
>> preemptively hacky. Sadly I can't think of clear and clever way for the 
>> kernel
>> to know when firmware will be ready either...  Would userspace know? Should 
>> the
>> kernel learn this from userspace ?
>
> Yes. Given only userspace knows when firmware is available (I could
> have it on a separate device and mount it at some time). So maybe
> userpsace should simply try and scan busses for unbound devices and
> tell them to re-probe when it decides that firmware is finally
> available.

OK, the folks wanting this mechanism can implement it then. Short of
that we only have hacks.

 Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez  wrote:
> On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
>> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
>> > On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
>> >>
>> >> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
>> >>>
>> >>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
>> 
>>  On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>> >
>> > On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
>> >>
>> >> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
>> >>  wrote:
>> >>>
>> >>> Does this mean a built-in driver can not get firmware from initramfs
>> >>> or
>> >>> built in the kernel early. Seems a bit too aggressive. The problem
>> >>> stated in
>> >>> this thread is when the firmware is not on initramfs but only on the
>> >>> rootfs.
>> >>
>> >>
>> >> Yes, strictly speaking, user mode request can't be handled with defer
>> >> probe
>> >> during booting because we don't know how the user helper handles the
>> >> request,
>> >
>> >
>> > FWIW I have a strategy in mind to help us compartamentalize the user
>> > mode
>> > helper only to the dell-rbu driver, and as such phase out that code
>> > eventually
>> > completely. Its part of the goals I have with the extensible firmware
>> > API I've
>> > been proposing.
>> >
>> >> that said even checking if the firmware exists in current path doesn't
>> >> make sense for user mode request.
>> >>
>> >> So the patch should have used defer proble for direct load only
>> >> during booting.
>> >
>> >
>> > What exact guarantees would we be giving to callers if they follow up
>> > on probe
>> > with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
>> > / probe
>> > (note that unless you're using async probe since we batch both so it
>> > doesn't really
>> > matter where you place your code) all together and then for the few
>> > remaining
>> > stragglers understand the requirements and provide an interface that
>> > lets them
>> > claim their requirements and try to meets them.
>> >
>> > A grammatical hunt for drivers who call fw API on init / probe can be
>> > completed, although I know the hunt needs a bit more fine tuning it
>> > surely can
>> > be completed. If we don't have many callers the compexity added for
>> > only a
>> > few callers with rather loose criteria seems rather unnecessary,
>> > specially if
>> > we can change the drivers and make these driver sthe exception rather
>> > than
>> > a norm.
>> >
>> > Then as for drivers *needing* the fw at probe why not have a proper
>> > interface
>> > that does guarantee they get the requirements they ask for first ? For
>> > instance
>> > a new probe type specified by the driver could enable the core to wait
>> > for say
>> > an event and then tirgger a probe, kind of how we ended up defining
>> > the async
>> > probe type preference:
>> >
>> > static struct some_bus_driver some_driver = {
>> >  .probe = some_probe,
>> >  .id_table = some_id,
>> >  .driver = {
>> >  .name = DEVICE_NAME,
>> >  .pm = _pm_ops,
>> >  .probe_type = PROBE_PREFER_POST_FOO,
>> >  },
>> > };
>> >
>> > Then we just don't try just hoping for completion but rather can do
>> > something
>> > about the criteria passed.
>> >>>
>> >>>
>> >>> So should the probe type indicate some event or should it just
>> >>> indicate what the driver needs, ie. .probe_type =
>> >>> PROBE_TYPE_NEED_FW.
>> >>
>> >>
>> >> Right so this is an open question. I suggested something like the above
>> >> since the deferred probe documentation on drivers/base/dd.c states:
>> >>
>> >>   * Sometimes driver probe order matters, but the kernel doesn't always
>> >> have
>> >>   * dependency information
>> >>
>> >> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
>> >> kernel where some work can be done to not only list the dependency
>> >> the information from the driver but also we know we can get it from
>> >> the kernel. In this case I do believe we could not only express the
>> >> requirement but also wait for it in the kernel. Before we do that
>> >> though I think it'd be good to do a grammar hunt to determine exactly
>> >> how popular all this fw on probe needed really is.
>> >
>> >
>> > Ok. So some background why we need it in brcm80211 drivers. So as a 
>> > wireless
>> > network device driver the answer we got when asking for an event to load
>> > firware is upon IF_UP for a registered net device. Because we try to do
>> > things smart we query the firmware running on the device for capabilities
>> > before we can 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
> > On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
> >>
> >> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
> >>>
> >>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
> 
>  On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
> >
> > On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
> >>
> >> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
> >>  wrote:
> >>>
> >>> Does this mean a built-in driver can not get firmware from initramfs
> >>> or
> >>> built in the kernel early. Seems a bit too aggressive. The problem
> >>> stated in
> >>> this thread is when the firmware is not on initramfs but only on the
> >>> rootfs.
> >>
> >>
> >> Yes, strictly speaking, user mode request can't be handled with defer
> >> probe
> >> during booting because we don't know how the user helper handles the
> >> request,
> >
> >
> > FWIW I have a strategy in mind to help us compartamentalize the user
> > mode
> > helper only to the dell-rbu driver, and as such phase out that code
> > eventually
> > completely. Its part of the goals I have with the extensible firmware
> > API I've
> > been proposing.
> >
> >> that said even checking if the firmware exists in current path doesn't
> >> make sense for user mode request.
> >>
> >> So the patch should have used defer proble for direct load only
> >> during booting.
> >
> >
> > What exact guarantees would we be giving to callers if they follow up
> > on probe
> > with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
> > / probe
> > (note that unless you're using async probe since we batch both so it
> > doesn't really
> > matter where you place your code) all together and then for the few
> > remaining
> > stragglers understand the requirements and provide an interface that
> > lets them
> > claim their requirements and try to meets them.
> >
> > A grammatical hunt for drivers who call fw API on init / probe can be
> > completed, although I know the hunt needs a bit more fine tuning it
> > surely can
> > be completed. If we don't have many callers the compexity added for
> > only a
> > few callers with rather loose criteria seems rather unnecessary,
> > specially if
> > we can change the drivers and make these driver sthe exception rather
> > than
> > a norm.
> >
> > Then as for drivers *needing* the fw at probe why not have a proper
> > interface
> > that does guarantee they get the requirements they ask for first ? For
> > instance
> > a new probe type specified by the driver could enable the core to wait
> > for say
> > an event and then tirgger a probe, kind of how we ended up defining
> > the async
> > probe type preference:
> >
> > static struct some_bus_driver some_driver = {
> >  .probe = some_probe,
> >  .id_table = some_id,
> >  .driver = {
> >  .name = DEVICE_NAME,
> >  .pm = _pm_ops,
> >  .probe_type = PROBE_PREFER_POST_FOO,
> >  },
> > };
> >
> > Then we just don't try just hoping for completion but rather can do
> > something
> > about the criteria passed.
> >>>
> >>>
> >>> So should the probe type indicate some event or should it just
> >>> indicate what the driver needs, ie. .probe_type =
> >>> PROBE_TYPE_NEED_FW.
> >>
> >>
> >> Right so this is an open question. I suggested something like the above
> >> since the deferred probe documentation on drivers/base/dd.c states:
> >>
> >>   * Sometimes driver probe order matters, but the kernel doesn't always
> >> have
> >>   * dependency information
> >>
> >> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
> >> kernel where some work can be done to not only list the dependency
> >> the information from the driver but also we know we can get it from
> >> the kernel. In this case I do believe we could not only express the
> >> requirement but also wait for it in the kernel. Before we do that
> >> though I think it'd be good to do a grammar hunt to determine exactly
> >> how popular all this fw on probe needed really is.
> >
> >
> > Ok. So some background why we need it in brcm80211 drivers. So as a wireless
> > network device driver the answer we got when asking for an event to load
> > firware is upon IF_UP for a registered net device. Because we try to do
> > things smart we query the firmware running on the device for capabilities
> > before we can register the net device hence we request the firmware during
> > probe. This may be specific to wireless drivers (Intel has same approach if
> > not mistaken) but I suspect there may be 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
> On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
>>
>> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
>>>
>>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:

 On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>
> On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
>>
>> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
>>  wrote:
>>>
>>> Does this mean a built-in driver can not get firmware from initramfs
>>> or
>>> built in the kernel early. Seems a bit too aggressive. The problem
>>> stated in
>>> this thread is when the firmware is not on initramfs but only on the
>>> rootfs.
>>
>>
>> Yes, strictly speaking, user mode request can't be handled with defer
>> probe
>> during booting because we don't know how the user helper handles the
>> request,
>
>
> FWIW I have a strategy in mind to help us compartamentalize the user
> mode
> helper only to the dell-rbu driver, and as such phase out that code
> eventually
> completely. Its part of the goals I have with the extensible firmware
> API I've
> been proposing.
>
>> that said even checking if the firmware exists in current path doesn't
>> make sense for user mode request.
>>
>> So the patch should have used defer proble for direct load only
>> during booting.
>
>
> What exact guarantees would we be giving to callers if they follow up
> on probe
> with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
> / probe
> (note that unless you're using async probe since we batch both so it
> doesn't really
> matter where you place your code) all together and then for the few
> remaining
> stragglers understand the requirements and provide an interface that
> lets them
> claim their requirements and try to meets them.
>
> A grammatical hunt for drivers who call fw API on init / probe can be
> completed, although I know the hunt needs a bit more fine tuning it
> surely can
> be completed. If we don't have many callers the compexity added for
> only a
> few callers with rather loose criteria seems rather unnecessary,
> specially if
> we can change the drivers and make these driver sthe exception rather
> than
> a norm.
>
> Then as for drivers *needing* the fw at probe why not have a proper
> interface
> that does guarantee they get the requirements they ask for first ? For
> instance
> a new probe type specified by the driver could enable the core to wait
> for say
> an event and then tirgger a probe, kind of how we ended up defining
> the async
> probe type preference:
>
> static struct some_bus_driver some_driver = {
>  .probe = some_probe,
>  .id_table = some_id,
>  .driver = {
>  .name = DEVICE_NAME,
>  .pm = _pm_ops,
>  .probe_type = PROBE_PREFER_POST_FOO,
>  },
> };
>
> Then we just don't try just hoping for completion but rather can do
> something
> about the criteria passed.
>>>
>>>
>>> So should the probe type indicate some event or should it just
>>> indicate what the driver needs, ie. .probe_type =
>>> PROBE_TYPE_NEED_FW.
>>
>>
>> Right so this is an open question. I suggested something like the above
>> since the deferred probe documentation on drivers/base/dd.c states:
>>
>>   * Sometimes driver probe order matters, but the kernel doesn't always
>> have
>>   * dependency information
>>
>> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
>> kernel where some work can be done to not only list the dependency
>> the information from the driver but also we know we can get it from
>> the kernel. In this case I do believe we could not only express the
>> requirement but also wait for it in the kernel. Before we do that
>> though I think it'd be good to do a grammar hunt to determine exactly
>> how popular all this fw on probe needed really is.
>
>
> Ok. So some background why we need it in brcm80211 drivers. So as a wireless
> network device driver the answer we got when asking for an event to load
> firware is upon IF_UP for a registered net device. Because we try to do
> things smart we query the firmware running on the device for capabilities
> before we can register the net device hence we request the firmware during
> probe. This may be specific to wireless drivers (Intel has same approach if
> not mistaken) but I suspect there may be more.

We have the same issue with input devices: before we can register one
we need to set their capabilities and to know their capabilities we
quite often need to load their firmware/config and query the device.

Thanks.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:

On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:

On 09/02/2015 02:09 PM, Arend van Spriel wrote:

On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
 wrote:

Does this mean a built-in driver can not get firmware from initramfs or
built in the kernel early. Seems a bit too aggressive. The problem
stated in
this thread is when the firmware is not on initramfs but only on the
rootfs.


Yes, strictly speaking, user mode request can't be handled with defer
probe
during booting because we don't know how the user helper handles the
request,


FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code
eventually
completely. Its part of the goals I have with the extensible firmware
API I've
been proposing.


that said even checking if the firmware exists in current path doesn't
make sense for user mode request.

So the patch should have used defer proble for direct load only
during booting.


What exact guarantees would we be giving to callers if they follow up
on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
/ probe
(note that unless you're using async probe since we batch both so it
doesn't really
matter where you place your code) all together and then for the few
remaining
stragglers understand the requirements and provide an interface that
lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it
surely can
be completed. If we don't have many callers the compexity added for
only a
few callers with rather loose criteria seems rather unnecessary,
specially if
we can change the drivers and make these driver sthe exception rather
than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper
interface
that does guarantee they get the requirements they ask for first ? For
instance
a new probe type specified by the driver could enable the core to wait
for say
an event and then tirgger a probe, kind of how we ended up defining
the async
probe type preference:

static struct some_bus_driver some_driver = {
 .probe = some_probe,
 .id_table = some_id,
 .driver = {
 .name = DEVICE_NAME,
 .pm = _pm_ops,
 .probe_type = PROBE_PREFER_POST_FOO,
 },
};

Then we just don't try just hoping for completion but rather can do
something
about the criteria passed.


So should the probe type indicate some event or should it just
indicate what the driver needs, ie. .probe_type =
PROBE_TYPE_NEED_FW.


Right so this is an open question. I suggested something like the above
since the deferred probe documentation on drivers/base/dd.c states:

  * Sometimes driver probe order matters, but the kernel doesn't always have
  * dependency information

I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
kernel where some work can be done to not only list the dependency
the information from the driver but also we know we can get it from
the kernel. In this case I do believe we could not only express the
requirement but also wait for it in the kernel. Before we do that
though I think it'd be good to do a grammar hunt to determine exactly
how popular all this fw on probe needed really is.


Ok. So some background why we need it in brcm80211 drivers. So as a 
wireless network device driver the answer we got when asking for an 
event to load firware is upon IF_UP for a registered net device. Because 
we try to do things smart we query the firmware running on the device 
for capabilities before we can register the net device hence we request 
the firmware during probe. This may be specific to wireless drivers 
(Intel has same approach if not mistaken) but I suspect there may be more.


Regards,
Arend



Greg, provided we find sufficient drivers that really do need firmware
on probe, any thoughts on this line of approach to address this?

   Luis



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 5:13 AM, Arend van Spriel  wrote:
> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
>>
>> On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>>>
>>> On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

 On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
  wrote:
>
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive. The problem
> stated in
> this thread is when the firmware is not on initramfs but only on the
> rootfs.


 Yes, strictly speaking, user mode request can't be handled with defer
 probe
 during booting because we don't know how the user helper handles the
 request,
>>>
>>>
>>> FWIW I have a strategy in mind to help us compartamentalize the user mode
>>> helper only to the dell-rbu driver, and as such phase out that code
>>> eventually
>>> completely. Its part of the goals I have with the extensible firmware
>>> API I've
>>> been proposing.
>>>
 that said even checking if the firmware exists in current path doesn't
 make sense for user mode request.

 So the patch should have used defer proble for direct load only
 during booting.
>>>
>>>
>>> What exact guarantees would we be giving to callers if they follow up
>>> on probe
>>> with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
>>> / probe
>>> (note that unless you're using async probe since we batch both so it
>>> doesn't really
>>> matter where you place your code) all together and then for the few
>>> remaining
>>> stragglers understand the requirements and provide an interface that
>>> lets them
>>> claim their requirements and try to meets them.
>>>
>>> A grammatical hunt for drivers who call fw API on init / probe can be
>>> completed, although I know the hunt needs a bit more fine tuning it
>>> surely can
>>> be completed. If we don't have many callers the compexity added for
>>> only a
>>> few callers with rather loose criteria seems rather unnecessary,
>>> specially if
>>> we can change the drivers and make these driver sthe exception rather
>>> than
>>> a norm.
>>>
>>> Then as for drivers *needing* the fw at probe why not have a proper
>>> interface
>>> that does guarantee they get the requirements they ask for first ? For
>>> instance
>>> a new probe type specified by the driver could enable the core to wait
>>> for say
>>> an event and then tirgger a probe, kind of how we ended up defining
>>> the async
>>> probe type preference:
>>>
>>> static struct some_bus_driver some_driver = {
>>>  .probe = some_probe,
>>>  .id_table = some_id,
>>>  .driver = {
>>>  .name = DEVICE_NAME,
>>>  .pm = _pm_ops,
>>>  .probe_type = PROBE_PREFER_POST_FOO,
>>>  },
>>> };
>>>
>>> Then we just don't try just hoping for completion but rather can do
>>> something
>>> about the criteria passed.
>
>
> So should the probe type indicate some event or should it just indicate what
> the driver needs, ie. .probe_type = PROBE_TYPE_NEED_FW.

What will kernel do when it sees such an option? It does not know when
firmware will become available (if ever). We can only try and fail and
then userspace may try re-probing devices once firmware is available.
I guess the question is how to let userspace know what devices failed
to probe because of the lack of firmware.

Thanks.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
> >On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
> >>On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
> >>>On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
> >>> wrote:
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive. The problem
> stated in
> this thread is when the firmware is not on initramfs but only on the
> rootfs.
> >>>
> >>>Yes, strictly speaking, user mode request can't be handled with defer
> >>>probe
> >>>during booting because we don't know how the user helper handles the
> >>>request,
> >>
> >>FWIW I have a strategy in mind to help us compartamentalize the user mode
> >>helper only to the dell-rbu driver, and as such phase out that code
> >>eventually
> >>completely. Its part of the goals I have with the extensible firmware
> >>API I've
> >>been proposing.
> >>
> >>>that said even checking if the firmware exists in current path doesn't
> >>>make sense for user mode request.
> >>>
> >>>So the patch should have used defer proble for direct load only
> >>>during booting.
> >>
> >>What exact guarantees would we be giving to callers if they follow up
> >>on probe
> >>with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
> >>/ probe
> >>(note that unless you're using async probe since we batch both so it
> >>doesn't really
> >>matter where you place your code) all together and then for the few
> >>remaining
> >>stragglers understand the requirements and provide an interface that
> >>lets them
> >>claim their requirements and try to meets them.
> >>
> >>A grammatical hunt for drivers who call fw API on init / probe can be
> >>completed, although I know the hunt needs a bit more fine tuning it
> >>surely can
> >>be completed. If we don't have many callers the compexity added for
> >>only a
> >>few callers with rather loose criteria seems rather unnecessary,
> >>specially if
> >>we can change the drivers and make these driver sthe exception rather
> >>than
> >>a norm.
> >>
> >>Then as for drivers *needing* the fw at probe why not have a proper
> >>interface
> >>that does guarantee they get the requirements they ask for first ? For
> >>instance
> >>a new probe type specified by the driver could enable the core to wait
> >>for say
> >>an event and then tirgger a probe, kind of how we ended up defining
> >>the async
> >>probe type preference:
> >>
> >>static struct some_bus_driver some_driver = {
> >> .probe = some_probe,
> >> .id_table = some_id,
> >> .driver = {
> >> .name = DEVICE_NAME,
> >> .pm = _pm_ops,
> >> .probe_type = PROBE_PREFER_POST_FOO,
> >> },
> >>};
> >>
> >>Then we just don't try just hoping for completion but rather can do
> >>something
> >>about the criteria passed.
> 
> So should the probe type indicate some event or should it just
> indicate what the driver needs, ie. .probe_type =
> PROBE_TYPE_NEED_FW.

Right so this is an open question. I suggested something like the above
since the deferred probe documentation on drivers/base/dd.c states:

 * Sometimes driver probe order matters, but the kernel doesn't always have 
 * dependency information 

I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
kernel where some work can be done to not only list the dependency
the information from the driver but also we know we can get it from
the kernel. In this case I do believe we could not only express the
requirement but also wait for it in the kernel. Before we do that
though I think it'd be good to do a grammar hunt to determine exactly
how popular all this fw on probe needed really is.

Greg, provided we find sufficient drivers that really do need firmware
on probe, any thoughts on this line of approach to address this?

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/02/2015 02:09 PM, Arend van Spriel wrote:

On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
 wrote:

Does this mean a built-in driver can not get firmware from initramfs or
built in the kernel early. Seems a bit too aggressive. The problem
stated in
this thread is when the firmware is not on initramfs but only on the
rootfs.


Yes, strictly speaking, user mode request can't be handled with defer
probe
during booting because we don't know how the user helper handles the
request,


FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code
eventually
completely. Its part of the goals I have with the extensible firmware
API I've
been proposing.


that said even checking if the firmware exists in current path doesn't
make sense for user mode request.

So the patch should have used defer proble for direct load only
during booting.


What exact guarantees would we be giving to callers if they follow up
on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
/ probe
(note that unless you're using async probe since we batch both so it
doesn't really
matter where you place your code) all together and then for the few
remaining
stragglers understand the requirements and provide an interface that
lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it
surely can
be completed. If we don't have many callers the compexity added for
only a
few callers with rather loose criteria seems rather unnecessary,
specially if
we can change the drivers and make these driver sthe exception rather
than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper
interface
that does guarantee they get the requirements they ask for first ? For
instance
a new probe type specified by the driver could enable the core to wait
for say
an event and then tirgger a probe, kind of how we ended up defining
the async
probe type preference:

static struct some_bus_driver some_driver = {
 .probe = some_probe,
 .id_table = some_id,
 .driver = {
 .name = DEVICE_NAME,
 .pm = _pm_ops,
 .probe_type = PROBE_PREFER_POST_FOO,
 },
};

Then we just don't try just hoping for completion but rather can do
something
about the criteria passed.


So should the probe type indicate some event or should it just indicate 
what the driver needs, ie. .probe_type = PROBE_TYPE_NEED_FW.


Regards,
Arend


That sounds good to me and learning about the async probe type. We do a
schedule work in our module_init to avoid the probe being done in init
context. Guess we can change that using the async probe type :-p

Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel  wrote:

Does this mean a built-in driver can not get firmware from initramfs or
built in the kernel early. Seems a bit too aggressive. The problem stated in
this thread is when the firmware is not on initramfs but only on the rootfs.


Yes, strictly speaking, user mode request can't be handled with defer probe
during booting because we don't know how the user helper handles the
request,


FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code eventually
completely. Its part of the goals I have with the extensible firmware API I've
been proposing.


that said even checking if the firmware exists in current path doesn't
make sense for user mode request.

So the patch should have used defer proble for direct load only
during booting.


What exact guarantees would we be giving to callers if they follow up on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init / probe
(note that unless you're using async probe since we batch both so it doesn't 
really
matter where you place your code) all together and then for the few remaining
stragglers understand the requirements and provide an interface that lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it surely can
be completed. If we don't have many callers the compexity added for only a
few callers with rather loose criteria seems rather unnecessary, specially if
we can change the drivers and make these driver sthe exception rather than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper interface
that does guarantee they get the requirements they ask for first ? For instance
a new probe type specified by the driver could enable the core to wait for say
an event and then tirgger a probe, kind of how we ended up defining the async
probe type preference:

static struct some_bus_driver some_driver = {
 .probe = some_probe,
 .id_table = some_id,
 .driver = {
 .name = DEVICE_NAME,
 .pm = _pm_ops,
 .probe_type = PROBE_PREFER_POST_FOO,
 },
};

Then we just don't try just hoping for completion but rather can do something
about the criteria passed.


That sounds good to me and learning about the async probe type. We do a 
schedule work in our module_init to avoid the probe being done in init 
context. Guess we can change that using the async probe type :-p


Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
> >On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
> >>On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
> >>>On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
> >>> wrote:
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive. The problem
> stated in
> this thread is when the firmware is not on initramfs but only on the
> rootfs.
> >>>
> >>>Yes, strictly speaking, user mode request can't be handled with defer
> >>>probe
> >>>during booting because we don't know how the user helper handles the
> >>>request,
> >>
> >>FWIW I have a strategy in mind to help us compartamentalize the user mode
> >>helper only to the dell-rbu driver, and as such phase out that code
> >>eventually
> >>completely. Its part of the goals I have with the extensible firmware
> >>API I've
> >>been proposing.
> >>
> >>>that said even checking if the firmware exists in current path doesn't
> >>>make sense for user mode request.
> >>>
> >>>So the patch should have used defer proble for direct load only
> >>>during booting.
> >>
> >>What exact guarantees would we be giving to callers if they follow up
> >>on probe
> >>with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
> >>/ probe
> >>(note that unless you're using async probe since we batch both so it
> >>doesn't really
> >>matter where you place your code) all together and then for the few
> >>remaining
> >>stragglers understand the requirements and provide an interface that
> >>lets them
> >>claim their requirements and try to meets them.
> >>
> >>A grammatical hunt for drivers who call fw API on init / probe can be
> >>completed, although I know the hunt needs a bit more fine tuning it
> >>surely can
> >>be completed. If we don't have many callers the compexity added for
> >>only a
> >>few callers with rather loose criteria seems rather unnecessary,
> >>specially if
> >>we can change the drivers and make these driver sthe exception rather
> >>than
> >>a norm.
> >>
> >>Then as for drivers *needing* the fw at probe why not have a proper
> >>interface
> >>that does guarantee they get the requirements they ask for first ? For
> >>instance
> >>a new probe type specified by the driver could enable the core to wait
> >>for say
> >>an event and then tirgger a probe, kind of how we ended up defining
> >>the async
> >>probe type preference:
> >>
> >>static struct some_bus_driver some_driver = {
> >> .probe = some_probe,
> >> .id_table = some_id,
> >> .driver = {
> >> .name = DEVICE_NAME,
> >> .pm = _pm_ops,
> >> .probe_type = PROBE_PREFER_POST_FOO,
> >> },
> >>};
> >>
> >>Then we just don't try just hoping for completion but rather can do
> >>something
> >>about the criteria passed.
> 
> So should the probe type indicate some event or should it just
> indicate what the driver needs, ie. .probe_type =
> PROBE_TYPE_NEED_FW.

Right so this is an open question. I suggested something like the above
since the deferred probe documentation on drivers/base/dd.c states:

 * Sometimes driver probe order matters, but the kernel doesn't always have 
 * dependency information 

I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
kernel where some work can be done to not only list the dependency
the information from the driver but also we know we can get it from
the kernel. In this case I do believe we could not only express the
requirement but also wait for it in the kernel. Before we do that
though I think it'd be good to do a grammar hunt to determine exactly
how popular all this fw on probe needed really is.

Greg, provided we find sufficient drivers that really do need firmware
on probe, any thoughts on this line of approach to address this?

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez  wrote:
> On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
>> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
>> > On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
>> >>
>> >> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
>> >>>
>> >>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
>> 
>>  On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>> >
>> > On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
>> >>
>> >> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
>> >>  wrote:
>> >>>
>> >>> Does this mean a built-in driver can not get firmware from initramfs
>> >>> or
>> >>> built in the kernel early. Seems a bit too aggressive. The problem
>> >>> stated in
>> >>> this thread is when the firmware is not on initramfs but only on the
>> >>> rootfs.
>> >>
>> >>
>> >> Yes, strictly speaking, user mode request can't be handled with defer
>> >> probe
>> >> during booting because we don't know how the user helper handles the
>> >> request,
>> >
>> >
>> > FWIW I have a strategy in mind to help us compartamentalize the user
>> > mode
>> > helper only to the dell-rbu driver, and as such phase out that code
>> > eventually
>> > completely. Its part of the goals I have with the extensible firmware
>> > API I've
>> > been proposing.
>> >
>> >> that said even checking if the firmware exists in current path doesn't
>> >> make sense for user mode request.
>> >>
>> >> So the patch should have used defer proble for direct load only
>> >> during booting.
>> >
>> >
>> > What exact guarantees would we be giving to callers if they follow up
>> > on probe
>> > with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
>> > / probe
>> > (note that unless you're using async probe since we batch both so it
>> > doesn't really
>> > matter where you place your code) all together and then for the few
>> > remaining
>> > stragglers understand the requirements and provide an interface that
>> > lets them
>> > claim their requirements and try to meets them.
>> >
>> > A grammatical hunt for drivers who call fw API on init / probe can be
>> > completed, although I know the hunt needs a bit more fine tuning it
>> > surely can
>> > be completed. If we don't have many callers the compexity added for
>> > only a
>> > few callers with rather loose criteria seems rather unnecessary,
>> > specially if
>> > we can change the drivers and make these driver sthe exception rather
>> > than
>> > a norm.
>> >
>> > Then as for drivers *needing* the fw at probe why not have a proper
>> > interface
>> > that does guarantee they get the requirements they ask for first ? For
>> > instance
>> > a new probe type specified by the driver could enable the core to wait
>> > for say
>> > an event and then tirgger a probe, kind of how we ended up defining
>> > the async
>> > probe type preference:
>> >
>> > static struct some_bus_driver some_driver = {
>> >  .probe = some_probe,
>> >  .id_table = some_id,
>> >  .driver = {
>> >  .name = DEVICE_NAME,
>> >  .pm = _pm_ops,
>> >  .probe_type = PROBE_PREFER_POST_FOO,
>> >  },
>> > };
>> >
>> > Then we just don't try just hoping for completion but rather can do
>> > something
>> > about the criteria passed.
>> >>>
>> >>>
>> >>> So should the probe type indicate some event or should it just
>> >>> indicate what the driver needs, ie. .probe_type =
>> >>> PROBE_TYPE_NEED_FW.
>> >>
>> >>
>> >> Right so this is an open question. I suggested something like the above
>> >> since the deferred probe documentation on drivers/base/dd.c states:
>> >>
>> >>   * Sometimes driver probe order matters, but the kernel doesn't always
>> >> have
>> >>   * dependency information
>> >>
>> >> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
>> >> kernel where some work can be done to not only list the dependency
>> >> the information from the driver but also we know we can get it from
>> >> the kernel. In this case I do believe we could not only express the
>> >> requirement but also wait for it in the kernel. Before we do that
>> >> though I think it'd be good to do a grammar hunt to determine exactly
>> >> how popular all this fw on probe needed really is.
>> >
>> >
>> > Ok. So some background why we need it in brcm80211 drivers. So as a 
>> > wireless
>> > network device driver the answer we got when asking for an event to load
>> > firware is upon IF_UP for a registered net device. Because we try to do
>> > things smart we query the firmware running 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Wed, Sep 2, 2015 at 4:29 PM, Dmitry Torokhov
 wrote:
> On Wed, Sep 2, 2015 at 4:22 PM, Luis R. Rodriguez  wrote:
>> On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
>>> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
>>> > Ok. So some background why we need it in brcm80211 drivers. So as a 
>>> > wireless
>>> > network device driver the answer we got when asking for an event to load
>>> > firware is upon IF_UP for a registered net device. Because we try to do
>>> > things smart we query the firmware running on the device for capabilities
>>> > before we can register the net device hence we request the firmware during
>>> > probe. This may be specific to wireless drivers (Intel has same approach 
>>> > if
>>> > not mistaken) but I suspect there may be more.
>>>
>>> We have the same issue with input devices: before we can register one
>>> we need to set their capabilities and to know their capabilities we
>>> quite often need to load their firmware/config and query the device.
>>
>> Should Arend's driver use async probe then?
>
> What has async probe have to do with anything? We will still be
> waiting for async probes to finish before we mount rootfs so it will
> not change absolutely anything.

:) Right, its what I was alluding to as well.

>> IMHO its just as hacky as using -EPROBE_DEFER too, but its at least
>> preemptively hacky. Sadly I can't think of clear and clever way for the 
>> kernel
>> to know when firmware will be ready either...  Would userspace know? Should 
>> the
>> kernel learn this from userspace ?
>
> Yes. Given only userspace knows when firmware is available (I could
> have it on a separate device and mount it at some time). So maybe
> userpsace should simply try and scan busses for unbound devices and
> tell them to re-probe when it decides that firmware is finally
> available.

OK, the folks wanting this mechanism can implement it then. Short of
that we only have hacks.

 Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 5:13 AM, Arend van Spriel  wrote:
> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
>>
>> On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>>>
>>> On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

 On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
  wrote:
>
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive. The problem
> stated in
> this thread is when the firmware is not on initramfs but only on the
> rootfs.


 Yes, strictly speaking, user mode request can't be handled with defer
 probe
 during booting because we don't know how the user helper handles the
 request,
>>>
>>>
>>> FWIW I have a strategy in mind to help us compartamentalize the user mode
>>> helper only to the dell-rbu driver, and as such phase out that code
>>> eventually
>>> completely. Its part of the goals I have with the extensible firmware
>>> API I've
>>> been proposing.
>>>
 that said even checking if the firmware exists in current path doesn't
 make sense for user mode request.

 So the patch should have used defer proble for direct load only
 during booting.
>>>
>>>
>>> What exact guarantees would we be giving to callers if they follow up
>>> on probe
>>> with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
>>> / probe
>>> (note that unless you're using async probe since we batch both so it
>>> doesn't really
>>> matter where you place your code) all together and then for the few
>>> remaining
>>> stragglers understand the requirements and provide an interface that
>>> lets them
>>> claim their requirements and try to meets them.
>>>
>>> A grammatical hunt for drivers who call fw API on init / probe can be
>>> completed, although I know the hunt needs a bit more fine tuning it
>>> surely can
>>> be completed. If we don't have many callers the compexity added for
>>> only a
>>> few callers with rather loose criteria seems rather unnecessary,
>>> specially if
>>> we can change the drivers and make these driver sthe exception rather
>>> than
>>> a norm.
>>>
>>> Then as for drivers *needing* the fw at probe why not have a proper
>>> interface
>>> that does guarantee they get the requirements they ask for first ? For
>>> instance
>>> a new probe type specified by the driver could enable the core to wait
>>> for say
>>> an event and then tirgger a probe, kind of how we ended up defining
>>> the async
>>> probe type preference:
>>>
>>> static struct some_bus_driver some_driver = {
>>>  .probe = some_probe,
>>>  .id_table = some_id,
>>>  .driver = {
>>>  .name = DEVICE_NAME,
>>>  .pm = _pm_ops,
>>>  .probe_type = PROBE_PREFER_POST_FOO,
>>>  },
>>> };
>>>
>>> Then we just don't try just hoping for completion but rather can do
>>> something
>>> about the criteria passed.
>
>
> So should the probe type indicate some event or should it just indicate what
> the driver needs, ie. .probe_type = PROBE_TYPE_NEED_FW.

What will kernel do when it sees such an option? It does not know when
firmware will become available (if ever). We can only try and fail and
then userspace may try re-probing devices once firmware is available.
I guess the question is how to let userspace know what devices failed
to probe because of the lack of firmware.

Thanks.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:

On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:

On 09/02/2015 02:09 PM, Arend van Spriel wrote:

On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
 wrote:

Does this mean a built-in driver can not get firmware from initramfs or
built in the kernel early. Seems a bit too aggressive. The problem
stated in
this thread is when the firmware is not on initramfs but only on the
rootfs.


Yes, strictly speaking, user mode request can't be handled with defer
probe
during booting because we don't know how the user helper handles the
request,


FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code
eventually
completely. Its part of the goals I have with the extensible firmware
API I've
been proposing.


that said even checking if the firmware exists in current path doesn't
make sense for user mode request.

So the patch should have used defer proble for direct load only
during booting.


What exact guarantees would we be giving to callers if they follow up
on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
/ probe
(note that unless you're using async probe since we batch both so it
doesn't really
matter where you place your code) all together and then for the few
remaining
stragglers understand the requirements and provide an interface that
lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it
surely can
be completed. If we don't have many callers the compexity added for
only a
few callers with rather loose criteria seems rather unnecessary,
specially if
we can change the drivers and make these driver sthe exception rather
than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper
interface
that does guarantee they get the requirements they ask for first ? For
instance
a new probe type specified by the driver could enable the core to wait
for say
an event and then tirgger a probe, kind of how we ended up defining
the async
probe type preference:

static struct some_bus_driver some_driver = {
 .probe = some_probe,
 .id_table = some_id,
 .driver = {
 .name = DEVICE_NAME,
 .pm = _pm_ops,
 .probe_type = PROBE_PREFER_POST_FOO,
 },
};

Then we just don't try just hoping for completion but rather can do
something
about the criteria passed.


So should the probe type indicate some event or should it just
indicate what the driver needs, ie. .probe_type =
PROBE_TYPE_NEED_FW.


Right so this is an open question. I suggested something like the above
since the deferred probe documentation on drivers/base/dd.c states:

  * Sometimes driver probe order matters, but the kernel doesn't always have
  * dependency information

I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
kernel where some work can be done to not only list the dependency
the information from the driver but also we know we can get it from
the kernel. In this case I do believe we could not only express the
requirement but also wait for it in the kernel. Before we do that
though I think it'd be good to do a grammar hunt to determine exactly
how popular all this fw on probe needed really is.


Ok. So some background why we need it in brcm80211 drivers. So as a 
wireless network device driver the answer we got when asking for an 
event to load firware is upon IF_UP for a registered net device. Because 
we try to do things smart we query the firmware running on the device 
for capabilities before we can register the net device hence we request 
the firmware during probe. This may be specific to wireless drivers 
(Intel has same approach if not mistaken) but I suspect there may be more.


Regards,
Arend



Greg, provided we find sufficient drivers that really do need firmware
on probe, any thoughts on this line of approach to address this?

   Luis



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
> On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
>>
>> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
>>>
>>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:

 On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
>
> On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
>>
>> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
>>  wrote:
>>>
>>> Does this mean a built-in driver can not get firmware from initramfs
>>> or
>>> built in the kernel early. Seems a bit too aggressive. The problem
>>> stated in
>>> this thread is when the firmware is not on initramfs but only on the
>>> rootfs.
>>
>>
>> Yes, strictly speaking, user mode request can't be handled with defer
>> probe
>> during booting because we don't know how the user helper handles the
>> request,
>
>
> FWIW I have a strategy in mind to help us compartamentalize the user
> mode
> helper only to the dell-rbu driver, and as such phase out that code
> eventually
> completely. Its part of the goals I have with the extensible firmware
> API I've
> been proposing.
>
>> that said even checking if the firmware exists in current path doesn't
>> make sense for user mode request.
>>
>> So the patch should have used defer proble for direct load only
>> during booting.
>
>
> What exact guarantees would we be giving to callers if they follow up
> on probe
> with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
> / probe
> (note that unless you're using async probe since we batch both so it
> doesn't really
> matter where you place your code) all together and then for the few
> remaining
> stragglers understand the requirements and provide an interface that
> lets them
> claim their requirements and try to meets them.
>
> A grammatical hunt for drivers who call fw API on init / probe can be
> completed, although I know the hunt needs a bit more fine tuning it
> surely can
> be completed. If we don't have many callers the compexity added for
> only a
> few callers with rather loose criteria seems rather unnecessary,
> specially if
> we can change the drivers and make these driver sthe exception rather
> than
> a norm.
>
> Then as for drivers *needing* the fw at probe why not have a proper
> interface
> that does guarantee they get the requirements they ask for first ? For
> instance
> a new probe type specified by the driver could enable the core to wait
> for say
> an event and then tirgger a probe, kind of how we ended up defining
> the async
> probe type preference:
>
> static struct some_bus_driver some_driver = {
>  .probe = some_probe,
>  .id_table = some_id,
>  .driver = {
>  .name = DEVICE_NAME,
>  .pm = _pm_ops,
>  .probe_type = PROBE_PREFER_POST_FOO,
>  },
> };
>
> Then we just don't try just hoping for completion but rather can do
> something
> about the criteria passed.
>>>
>>>
>>> So should the probe type indicate some event or should it just
>>> indicate what the driver needs, ie. .probe_type =
>>> PROBE_TYPE_NEED_FW.
>>
>>
>> Right so this is an open question. I suggested something like the above
>> since the deferred probe documentation on drivers/base/dd.c states:
>>
>>   * Sometimes driver probe order matters, but the kernel doesn't always
>> have
>>   * dependency information
>>
>> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
>> kernel where some work can be done to not only list the dependency
>> the information from the driver but also we know we can get it from
>> the kernel. In this case I do believe we could not only express the
>> requirement but also wait for it in the kernel. Before we do that
>> though I think it'd be good to do a grammar hunt to determine exactly
>> how popular all this fw on probe needed really is.
>
>
> Ok. So some background why we need it in brcm80211 drivers. So as a wireless
> network device driver the answer we got when asking for an event to load
> firware is upon IF_UP for a registered net device. Because we try to do
> things smart we query the firmware running on the device for capabilities
> before we can register the net device hence we request the firmware during
> probe. This may be specific to wireless drivers (Intel has same approach if
> not mistaken) but I suspect there may be more.

We have the same issue with input devices: before we can register one
we need to set their capabilities and to know their capabilities we
quite often need to load their firmware/config and query the device.

Thanks.

-- 
Dmitry
--
To unsubscribe from this 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Wed, Sep 02, 2015 at 04:13:51PM -0700, Dmitry Torokhov wrote:
> On Wed, Sep 2, 2015 at 2:03 PM, Arend van Spriel  wrote:
> > On 09/02/2015 08:58 PM, Luis R. Rodriguez wrote:
> >>
> >> On Wed, Sep 02, 2015 at 02:13:49PM +0200, Arend van Spriel wrote:
> >>>
> >>> On 09/02/2015 02:09 PM, Arend van Spriel wrote:
> 
>  On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:
> >
> > On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
> >>
> >> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
> >>  wrote:
> >>>
> >>> Does this mean a built-in driver can not get firmware from initramfs
> >>> or
> >>> built in the kernel early. Seems a bit too aggressive. The problem
> >>> stated in
> >>> this thread is when the firmware is not on initramfs but only on the
> >>> rootfs.
> >>
> >>
> >> Yes, strictly speaking, user mode request can't be handled with defer
> >> probe
> >> during booting because we don't know how the user helper handles the
> >> request,
> >
> >
> > FWIW I have a strategy in mind to help us compartamentalize the user
> > mode
> > helper only to the dell-rbu driver, and as such phase out that code
> > eventually
> > completely. Its part of the goals I have with the extensible firmware
> > API I've
> > been proposing.
> >
> >> that said even checking if the firmware exists in current path doesn't
> >> make sense for user mode request.
> >>
> >> So the patch should have used defer proble for direct load only
> >> during booting.
> >
> >
> > What exact guarantees would we be giving to callers if they follow up
> > on probe
> > with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
> > / probe
> > (note that unless you're using async probe since we batch both so it
> > doesn't really
> > matter where you place your code) all together and then for the few
> > remaining
> > stragglers understand the requirements and provide an interface that
> > lets them
> > claim their requirements and try to meets them.
> >
> > A grammatical hunt for drivers who call fw API on init / probe can be
> > completed, although I know the hunt needs a bit more fine tuning it
> > surely can
> > be completed. If we don't have many callers the compexity added for
> > only a
> > few callers with rather loose criteria seems rather unnecessary,
> > specially if
> > we can change the drivers and make these driver sthe exception rather
> > than
> > a norm.
> >
> > Then as for drivers *needing* the fw at probe why not have a proper
> > interface
> > that does guarantee they get the requirements they ask for first ? For
> > instance
> > a new probe type specified by the driver could enable the core to wait
> > for say
> > an event and then tirgger a probe, kind of how we ended up defining
> > the async
> > probe type preference:
> >
> > static struct some_bus_driver some_driver = {
> >  .probe = some_probe,
> >  .id_table = some_id,
> >  .driver = {
> >  .name = DEVICE_NAME,
> >  .pm = _pm_ops,
> >  .probe_type = PROBE_PREFER_POST_FOO,
> >  },
> > };
> >
> > Then we just don't try just hoping for completion but rather can do
> > something
> > about the criteria passed.
> >>>
> >>>
> >>> So should the probe type indicate some event or should it just
> >>> indicate what the driver needs, ie. .probe_type =
> >>> PROBE_TYPE_NEED_FW.
> >>
> >>
> >> Right so this is an open question. I suggested something like the above
> >> since the deferred probe documentation on drivers/base/dd.c states:
> >>
> >>   * Sometimes driver probe order matters, but the kernel doesn't always
> >> have
> >>   * dependency information
> >>
> >> I'm alluding that we consider *avoiding* -EPROBE_DEFER for areas of the
> >> kernel where some work can be done to not only list the dependency
> >> the information from the driver but also we know we can get it from
> >> the kernel. In this case I do believe we could not only express the
> >> requirement but also wait for it in the kernel. Before we do that
> >> though I think it'd be good to do a grammar hunt to determine exactly
> >> how popular all this fw on probe needed really is.
> >
> >
> > Ok. So some background why we need it in brcm80211 drivers. So as a wireless
> > network device driver the answer we got when asking for an event to load
> > firware is upon IF_UP for a registered net device. Because we try to do
> > things smart we query the firmware running on the device for capabilities
> > before we can register the net device hence we request the firmware during
> > probe. This may be specific to wireless drivers (Intel has same approach if
> > not 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel  wrote:

Does this mean a built-in driver can not get firmware from initramfs or
built in the kernel early. Seems a bit too aggressive. The problem stated in
this thread is when the firmware is not on initramfs but only on the rootfs.


Yes, strictly speaking, user mode request can't be handled with defer probe
during booting because we don't know how the user helper handles the
request,


FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code eventually
completely. Its part of the goals I have with the extensible firmware API I've
been proposing.


that said even checking if the firmware exists in current path doesn't
make sense for user mode request.

So the patch should have used defer proble for direct load only
during booting.


What exact guarantees would we be giving to callers if they follow up on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init / probe
(note that unless you're using async probe since we batch both so it doesn't 
really
matter where you place your code) all together and then for the few remaining
stragglers understand the requirements and provide an interface that lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it surely can
be completed. If we don't have many callers the compexity added for only a
few callers with rather loose criteria seems rather unnecessary, specially if
we can change the drivers and make these driver sthe exception rather than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper interface
that does guarantee they get the requirements they ask for first ? For instance
a new probe type specified by the driver could enable the core to wait for say
an event and then tirgger a probe, kind of how we ended up defining the async
probe type preference:

static struct some_bus_driver some_driver = {
 .probe = some_probe,
 .id_table = some_id,
 .driver = {
 .name = DEVICE_NAME,
 .pm = _pm_ops,
 .probe_type = PROBE_PREFER_POST_FOO,
 },
};

Then we just don't try just hoping for completion but rather can do something
about the criteria passed.


That sounds good to me and learning about the async probe type. We do a 
schedule work in our module_init to avoid the probe being done in init 
context. Guess we can change that using the async probe type :-p


Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 09/02/2015 02:09 PM, Arend van Spriel wrote:

On 09/02/2015 03:19 AM, Luis R. Rodriguez wrote:

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel
 wrote:

Does this mean a built-in driver can not get firmware from initramfs or
built in the kernel early. Seems a bit too aggressive. The problem
stated in
this thread is when the firmware is not on initramfs but only on the
rootfs.


Yes, strictly speaking, user mode request can't be handled with defer
probe
during booting because we don't know how the user helper handles the
request,


FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code
eventually
completely. Its part of the goals I have with the extensible firmware
API I've
been proposing.


that said even checking if the firmware exists in current path doesn't
make sense for user mode request.

So the patch should have used defer proble for direct load only
during booting.


What exact guarantees would we be giving to callers if they follow up
on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init
/ probe
(note that unless you're using async probe since we batch both so it
doesn't really
matter where you place your code) all together and then for the few
remaining
stragglers understand the requirements and provide an interface that
lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it
surely can
be completed. If we don't have many callers the compexity added for
only a
few callers with rather loose criteria seems rather unnecessary,
specially if
we can change the drivers and make these driver sthe exception rather
than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper
interface
that does guarantee they get the requirements they ask for first ? For
instance
a new probe type specified by the driver could enable the core to wait
for say
an event and then tirgger a probe, kind of how we ended up defining
the async
probe type preference:

static struct some_bus_driver some_driver = {
 .probe = some_probe,
 .id_table = some_id,
 .driver = {
 .name = DEVICE_NAME,
 .pm = _pm_ops,
 .probe_type = PROBE_PREFER_POST_FOO,
 },
};

Then we just don't try just hoping for completion but rather can do
something
about the criteria passed.


So should the probe type indicate some event or should it just indicate 
what the driver needs, ie. .probe_type = PROBE_TYPE_NEED_FW.


Regards,
Arend


That sounds good to me and learning about the async probe type. We do a
schedule work in our module_init to avoid the probe being done in init
context. Guess we can change that using the async probe type :-p

Regards,
Arend
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel  wrote:
> > Does this mean a built-in driver can not get firmware from initramfs or
> > built in the kernel early. Seems a bit too aggressive. The problem stated in
> > this thread is when the firmware is not on initramfs but only on the rootfs.
> 
> Yes, strictly speaking, user mode request can't be handled with defer probe
> during booting because we don't know how the user helper handles the
> request,

FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code eventually
completely. Its part of the goals I have with the extensible firmware API I've
been proposing.

> that said even checking if the firmware exists in current path doesn't
> make sense for user mode request.
> 
> So the patch should have used defer proble for direct load only
> during booting.

What exact guarantees would we be giving to callers if they follow up on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init / probe
(note that unless you're using async probe since we batch both so it doesn't 
really
matter where you place your code) all together and then for the few remaining
stragglers understand the requirements and provide an interface that lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it surely can
be completed. If we don't have many callers the compexity added for only a
few callers with rather loose criteria seems rather unnecessary, specially if
we can change the drivers and make these driver sthe exception rather than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper interface
that does guarantee they get the requirements they ask for first ? For instance
a new probe type specified by the driver could enable the core to wait for say
an event and then tirgger a probe, kind of how we ended up defining the async
probe type preference:

static struct some_bus_driver some_driver = {  
.probe = some_probe,  
.id_table = some_id,  
.driver = { 
.name = DEVICE_NAME,
.pm = _pm_ops,   
.probe_type = PROBE_PREFER_POST_FOO,
},  
}; 

Then we just don't try just hoping for completion but rather can do something
about the criteria passed.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Sat, Aug 29, 2015 at 12:09:01PM +0800, Ming Lei wrote:
> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:
> > On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
> >> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
> >>  wrote:
> >> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
> >> >  wrote:
> >> >>
> >> >> I think the options are to either :-
> >> >>
> >> >> 1) Don not support audio DSP drivers using topology data as built-in
> >> >> drivers. Audio is not really a critical system required for booting
> >> >> anyway.
> >> >
> >> > Yes, forcing it to be a module and not letting people compile it in by
> >> > mistake (and then not have it work) is an option.
> >> >
> >> > That said, there are situations where people don't want to use
> >> > modules. I used to eschew them for security reasons, for example - now
> >> > I instead just do a one-time temporary key. But others may have other
> >> > reasons to try to avoid modules.
> >> >
> >> >> 2) Create a default PCM for every driver that has topology data on the
> >> >> assumption that every sound card will at least 1 PCM. This PCM can then
> >> >> be re-configured when the FW is loaded.
> >> >
> >> > That would seem to be the better option if it is reasonably 
> >> > implementable.
> >> >
> >> > Of course, some kind of timer-based retry (limited *somehow*) of the
> >> > fw loading could work too, but smells really really hacky.
> >>
> >> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
> >> which should be one kind of fix, but looks there were objections at that 
> >> time.
> >
> > That would still be a hack. I'll note there is also asynchronous probe 
> > support
> > now but to use that would also be a hack for this issue. We don't want to
> 
> If we think firmware as one kind of resources like regulators, gpio and 
> others,
> PROBE_DEFER is one good match for firmware loading case, and
> it has been used by lots of drivers, so why can't it be used for
> firmware loading?

I'm glad you asked, it begs the question if we could have done something better
for these other components. In short its a matter of if we have an interface
that would let devices coming up ask: are my requirements available yet? Reason
we kick -EPROBE_DEFER is we can't answer this as we have no way to map some
of these requirements pricely so -EPROBE_DEFER is the best we can do at times.

It doesn't mean we shouldn't think harder, and for firmware I think we can and
should try harder to answer these questions.

I'm arguing that its a viable solution to use -EPROBE_DEFER but I don't think
its the best we can do but also I worry about the lack of semantics that would
be implied by user if they start doing this all over. In terms of semantics
I'd want at least some undestanding by the caller over certain guarantees of
what we are going to try to do for them by using -EPROBE_DEFER.

> One problem is that we need to convert drivers into returning -EPROBE_DEFER
> in case of request failure, and that may involve some work, but which
> should be mechanical.

And there may be cases where the fs might already be available, so it would
be pointless to retry if the error was true. To me that's a bit sloppy, and
part of the sloppiness comes from the lack of clear semantics. Its why we are
having this discussion. Its also not the first of its case and its why I'm kind
of trying to be a bit pedantic. I would prefer to avoid just a bandaid.

> > encourage folks to go down that road.  They'd be hacks for this issue as you
> > are simply delaying the driver probe for a later time and there is no 
> > guarantee
> > that any pivot_root() might have already been completed later to ensure your
> > driver's fw file is present. So it may work or it may not.
> 
> We can trigger defer probe explicitly once root fs is setup or other condition
> is met.

Now we're talking, that's the sort of line of solution I'd much prefer, but 
again
that's building on top of a use case that I think we should try to avoid. I 
think
the strategy is sound but not the way we're deferring probe. I think that's 
prone
to error and the solution lacks clarity.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Sat, Aug 29, 2015 at 12:09:01PM +0800, Ming Lei wrote:
> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:
> > On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
> >> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
> >>  wrote:
> >> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
> >> >  wrote:
> >> >>
> >> >> I think the options are to either :-
> >> >>
> >> >> 1) Don not support audio DSP drivers using topology data as built-in
> >> >> drivers. Audio is not really a critical system required for booting
> >> >> anyway.
> >> >
> >> > Yes, forcing it to be a module and not letting people compile it in by
> >> > mistake (and then not have it work) is an option.
> >> >
> >> > That said, there are situations where people don't want to use
> >> > modules. I used to eschew them for security reasons, for example - now
> >> > I instead just do a one-time temporary key. But others may have other
> >> > reasons to try to avoid modules.
> >> >
> >> >> 2) Create a default PCM for every driver that has topology data on the
> >> >> assumption that every sound card will at least 1 PCM. This PCM can then
> >> >> be re-configured when the FW is loaded.
> >> >
> >> > That would seem to be the better option if it is reasonably 
> >> > implementable.
> >> >
> >> > Of course, some kind of timer-based retry (limited *somehow*) of the
> >> > fw loading could work too, but smells really really hacky.
> >>
> >> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
> >> which should be one kind of fix, but looks there were objections at that 
> >> time.
> >
> > That would still be a hack. I'll note there is also asynchronous probe 
> > support
> > now but to use that would also be a hack for this issue. We don't want to
> 
> If we think firmware as one kind of resources like regulators, gpio and 
> others,
> PROBE_DEFER is one good match for firmware loading case, and
> it has been used by lots of drivers, so why can't it be used for
> firmware loading?

I'm glad you asked, it begs the question if we could have done something better
for these other components. In short its a matter of if we have an interface
that would let devices coming up ask: are my requirements available yet? Reason
we kick -EPROBE_DEFER is we can't answer this as we have no way to map some
of these requirements pricely so -EPROBE_DEFER is the best we can do at times.

It doesn't mean we shouldn't think harder, and for firmware I think we can and
should try harder to answer these questions.

I'm arguing that its a viable solution to use -EPROBE_DEFER but I don't think
its the best we can do but also I worry about the lack of semantics that would
be implied by user if they start doing this all over. In terms of semantics
I'd want at least some undestanding by the caller over certain guarantees of
what we are going to try to do for them by using -EPROBE_DEFER.

> One problem is that we need to convert drivers into returning -EPROBE_DEFER
> in case of request failure, and that may involve some work, but which
> should be mechanical.

And there may be cases where the fs might already be available, so it would
be pointless to retry if the error was true. To me that's a bit sloppy, and
part of the sloppiness comes from the lack of clear semantics. Its why we are
having this discussion. Its also not the first of its case and its why I'm kind
of trying to be a bit pedantic. I would prefer to avoid just a bandaid.

> > encourage folks to go down that road.  They'd be hacks for this issue as you
> > are simply delaying the driver probe for a later time and there is no 
> > guarantee
> > that any pivot_root() might have already been completed later to ensure your
> > driver's fw file is present. So it may work or it may not.
> 
> We can trigger defer probe explicitly once root fs is setup or other condition
> is met.

Now we're talking, that's the sort of line of solution I'd much prefer, but 
again
that's building on top of a use case that I think we should try to avoid. I 
think
the strategy is sound but not the way we're deferring probe. I think that's 
prone
to error and the solution lacks clarity.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Mon, Aug 31, 2015 at 10:21:34PM +0800, Ming Lei wrote:
> On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel  wrote:
> > Does this mean a built-in driver can not get firmware from initramfs or
> > built in the kernel early. Seems a bit too aggressive. The problem stated in
> > this thread is when the firmware is not on initramfs but only on the rootfs.
> 
> Yes, strictly speaking, user mode request can't be handled with defer probe
> during booting because we don't know how the user helper handles the
> request,

FWIW I have a strategy in mind to help us compartamentalize the user mode
helper only to the dell-rbu driver, and as such phase out that code eventually
completely. Its part of the goals I have with the extensible firmware API I've
been proposing.

> that said even checking if the firmware exists in current path doesn't
> make sense for user mode request.
> 
> So the patch should have used defer proble for direct load only
> during booting.

What exact guarantees would we be giving to callers if they follow up on probe
with -EDEFER_PROBE ? I'd much prefer to try to avoid such uses in init / probe
(note that unless you're using async probe since we batch both so it doesn't 
really
matter where you place your code) all together and then for the few remaining
stragglers understand the requirements and provide an interface that lets them
claim their requirements and try to meets them.

A grammatical hunt for drivers who call fw API on init / probe can be
completed, although I know the hunt needs a bit more fine tuning it surely can
be completed. If we don't have many callers the compexity added for only a
few callers with rather loose criteria seems rather unnecessary, specially if
we can change the drivers and make these driver sthe exception rather than
a norm.

Then as for drivers *needing* the fw at probe why not have a proper interface
that does guarantee they get the requirements they ask for first ? For instance
a new probe type specified by the driver could enable the core to wait for say
an event and then tirgger a probe, kind of how we ended up defining the async
probe type preference:

static struct some_bus_driver some_driver = {  
.probe = some_probe,  
.id_table = some_id,  
.driver = { 
.name = DEVICE_NAME,
.pm = _pm_ops,   
.probe_type = PROBE_PREFER_POST_FOO,
},  
}; 

Then we just don't try just hoping for completion but rather can do something
about the criteria passed.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel  wrote:
> On 08/29/2015 12:38 PM, Ming Lei wrote:
>>
>> On Sat, 29 Aug 2015 10:50:22 +0200
>> Arend van Spriel  wrote:
>>
>>> On 08/29/2015 09:11 AM, Takashi Iwai wrote:

 On Sat, 29 Aug 2015 06:09:01 +0200,
 Ming Lei wrote:
>
>
> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez 
> wrote:
>>
>> On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
>>>
>>> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
>>>  wrote:

 On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
  wrote:
>
>
> I think the options are to either :-
>
> 1) Don not support audio DSP drivers using topology data as
> built-in
> drivers. Audio is not really a critical system required for booting
> anyway.


 Yes, forcing it to be a module and not letting people compile it in
 by
 mistake (and then not have it work) is an option.

 That said, there are situations where people don't want to use
 modules. I used to eschew them for security reasons, for example -
 now
 I instead just do a one-time temporary key. But others may have
 other
 reasons to try to avoid modules.

> 2) Create a default PCM for every driver that has topology data on
> the
> assumption that every sound card will at least 1 PCM. This PCM can
> then
> be re-configured when the FW is loaded.


 That would seem to be the better option if it is reasonably
 implementable.

 Of course, some kind of timer-based retry (limited *somehow*) of the
 fw loading could work too, but smells really really hacky.
>>>
>>>
>>> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
>>> which should be one kind of fix, but looks there were objections at
>>> that time.
>>
>>
>> That would still be a hack. I'll note there is also asynchronous probe
>> support
>> now but to use that would also be a hack for this issue. We don't want
>> to
>
>
> If we think firmware as one kind of resources like regulators, gpio and
> others,
> PROBE_DEFER is one good match for firmware loading case, and
> it has been used by lots of drivers, so why can't it be used for
> firmware loading?
>
> One problem is that we need to convert drivers into returning
> -EPROBE_DEFER
> in case of request failure, and that may involve some work, but which
> should be mechanical.


 I find such a delaying mechanism not so bad, too.  It's very
 straightforward, at least, no big pain in the transition in the driver
 side.
>>>
>>>
>>> Not sure how this is going to work with request_firmware_nowait(). We
>>> use that in our drivers to get rid of ~60 sec. delay in probe and
>>> consequently boot time when built-in. So basically we return 0 on probe
>>> lacking better knowledge. Guess we can always move back to
>>> request_firmware calls when defer_probe support is available.
>>
>>
>> How about the following untested draft patch?
>>
>> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
>> index be0eb46..f66912f 100644
>> --- a/drivers/base/dd.c
>> +++ b/drivers/base/dd.c
>> @@ -171,6 +171,12 @@ static void driver_deferred_probe_trigger(void)
>> queue_work(deferred_wq, _probe_work);
>>   }
>>
>> +void driver_trigger_fw_load()
>> +{
>> +   driver_deferred_probe_trigger();
>> +}
>> +EXPORT_SYMBOL_GPL(driver_trigger_fw_load);
>> +
>>   /**
>>* deferred_probe_initcall() - Enable probing of deferred devices
>>*
>> diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
>> index 8524450..f879a07 100644
>> --- a/drivers/base/firmware_class.c
>> +++ b/drivers/base/firmware_class.c
>> @@ -1132,6 +1132,11 @@ _request_firmware(const struct firmware
>> **firmware_p, const char *name,
>> if (ret <= 0) /* error or already assigned */
>> goto out;
>>
>> +   if (system_state == SYSTEM_BOOTING) {
>> +   ret = -EPROBE_DEFER;
>> +   goto out;
>> +   }
>> +
>> ret = 0;
>> timeout = firmware_loading_timeout();
>> if (opt_flags & FW_OPT_NOWAIT) {
>> @@ -1311,6 +1316,9 @@ request_firmware_nowait(
>>   {
>> struct firmware_work *fw_work;
>>
>> +   if (system_state == SYSTEM_BOOTING)
>> +   return -EPROBE_DEFER;
>> +
>
>
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive. The problem stated in
> this thread is when the firmware is not on initramfs but only on the rootfs.

Yes, strictly speaking, user mode request can't be handled with defer probe
during booting because we don't know how the user helper handles the
request, that said even 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Sun, Aug 30, 2015 at 4:25 PM, Arend van Spriel  wrote:
> On 08/29/2015 12:38 PM, Ming Lei wrote:
>>
>> On Sat, 29 Aug 2015 10:50:22 +0200
>> Arend van Spriel  wrote:
>>
>>> On 08/29/2015 09:11 AM, Takashi Iwai wrote:

 On Sat, 29 Aug 2015 06:09:01 +0200,
 Ming Lei wrote:
>
>
> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez 
> wrote:
>>
>> On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
>>>
>>> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
>>>  wrote:

 On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
  wrote:
>
>
> I think the options are to either :-
>
> 1) Don not support audio DSP drivers using topology data as
> built-in
> drivers. Audio is not really a critical system required for booting
> anyway.


 Yes, forcing it to be a module and not letting people compile it in
 by
 mistake (and then not have it work) is an option.

 That said, there are situations where people don't want to use
 modules. I used to eschew them for security reasons, for example -
 now
 I instead just do a one-time temporary key. But others may have
 other
 reasons to try to avoid modules.

> 2) Create a default PCM for every driver that has topology data on
> the
> assumption that every sound card will at least 1 PCM. This PCM can
> then
> be re-configured when the FW is loaded.


 That would seem to be the better option if it is reasonably
 implementable.

 Of course, some kind of timer-based retry (limited *somehow*) of the
 fw loading could work too, but smells really really hacky.
>>>
>>>
>>> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
>>> which should be one kind of fix, but looks there were objections at
>>> that time.
>>
>>
>> That would still be a hack. I'll note there is also asynchronous probe
>> support
>> now but to use that would also be a hack for this issue. We don't want
>> to
>
>
> If we think firmware as one kind of resources like regulators, gpio and
> others,
> PROBE_DEFER is one good match for firmware loading case, and
> it has been used by lots of drivers, so why can't it be used for
> firmware loading?
>
> One problem is that we need to convert drivers into returning
> -EPROBE_DEFER
> in case of request failure, and that may involve some work, but which
> should be mechanical.


 I find such a delaying mechanism not so bad, too.  It's very
 straightforward, at least, no big pain in the transition in the driver
 side.
>>>
>>>
>>> Not sure how this is going to work with request_firmware_nowait(). We
>>> use that in our drivers to get rid of ~60 sec. delay in probe and
>>> consequently boot time when built-in. So basically we return 0 on probe
>>> lacking better knowledge. Guess we can always move back to
>>> request_firmware calls when defer_probe support is available.
>>
>>
>> How about the following untested draft patch?
>>
>> diff --git a/drivers/base/dd.c b/drivers/base/dd.c
>> index be0eb46..f66912f 100644
>> --- a/drivers/base/dd.c
>> +++ b/drivers/base/dd.c
>> @@ -171,6 +171,12 @@ static void driver_deferred_probe_trigger(void)
>> queue_work(deferred_wq, _probe_work);
>>   }
>>
>> +void driver_trigger_fw_load()
>> +{
>> +   driver_deferred_probe_trigger();
>> +}
>> +EXPORT_SYMBOL_GPL(driver_trigger_fw_load);
>> +
>>   /**
>>* deferred_probe_initcall() - Enable probing of deferred devices
>>*
>> diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
>> index 8524450..f879a07 100644
>> --- a/drivers/base/firmware_class.c
>> +++ b/drivers/base/firmware_class.c
>> @@ -1132,6 +1132,11 @@ _request_firmware(const struct firmware
>> **firmware_p, const char *name,
>> if (ret <= 0) /* error or already assigned */
>> goto out;
>>
>> +   if (system_state == SYSTEM_BOOTING) {
>> +   ret = -EPROBE_DEFER;
>> +   goto out;
>> +   }
>> +
>> ret = 0;
>> timeout = firmware_loading_timeout();
>> if (opt_flags & FW_OPT_NOWAIT) {
>> @@ -1311,6 +1316,9 @@ request_firmware_nowait(
>>   {
>> struct firmware_work *fw_work;
>>
>> +   if (system_state == SYSTEM_BOOTING)
>> +   return -EPROBE_DEFER;
>> +
>
>
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive. The problem stated in
> this thread is when the firmware is not on initramfs but only on the rootfs.

Yes, strictly speaking, user mode request can't 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Linus Torvalds]

On Sun, Aug 30, 2015 at 1:25 AM, Arend van Spriel  wrote:
> On 08/29/2015 12:38 PM, Ming Lei wrote:
>
> Does this mean a built-in driver can not get firmware from initramfs or
> built in the kernel early. Seems a bit too aggressive.

Yeah, that seems wrong. Loading firmware from initramfs is required
for some things, like disk drivers. Of course, depending on how it's
done, it's all after the SYSTEM_BOOTING phase, but ..

What we *might* do is to not allow it for the user-mode helper
fallback, but I think it's more likely that we'll just deprecate the
usermode helper fw loader entirely, so adding new error cases for it
seems pointless.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 08/29/2015 12:38 PM, Ming Lei wrote:

On Sat, 29 Aug 2015 10:50:22 +0200
Arend van Spriel  wrote:


On 08/29/2015 09:11 AM, Takashi Iwai wrote:

On Sat, 29 Aug 2015 06:09:01 +0200,
Ming Lei wrote:


On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:

On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:

On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
 wrote:

On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
 wrote:


I think the options are to either :-

1) Don not support audio DSP drivers using topology data as built-in
drivers. Audio is not really a critical system required for booting
anyway.


Yes, forcing it to be a module and not letting people compile it in by
mistake (and then not have it work) is an option.

That said, there are situations where people don't want to use
modules. I used to eschew them for security reasons, for example - now
I instead just do a one-time temporary key. But others may have other
reasons to try to avoid modules.


2) Create a default PCM for every driver that has topology data on the
assumption that every sound card will at least 1 PCM. This PCM can then
be re-configured when the FW is loaded.


That would seem to be the better option if it is reasonably implementable.

Of course, some kind of timer-based retry (limited *somehow*) of the
fw loading could work too, but smells really really hacky.


Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
which should be one kind of fix, but looks there were objections at that time.


That would still be a hack. I'll note there is also asynchronous probe support
now but to use that would also be a hack for this issue. We don't want to


If we think firmware as one kind of resources like regulators, gpio and others,
PROBE_DEFER is one good match for firmware loading case, and
it has been used by lots of drivers, so why can't it be used for
firmware loading?

One problem is that we need to convert drivers into returning -EPROBE_DEFER
in case of request failure, and that may involve some work, but which
should be mechanical.


I find such a delaying mechanism not so bad, too.  It's very
straightforward, at least, no big pain in the transition in the driver
side.


Not sure how this is going to work with request_firmware_nowait(). We
use that in our drivers to get rid of ~60 sec. delay in probe and
consequently boot time when built-in. So basically we return 0 on probe
lacking better knowledge. Guess we can always move back to
request_firmware calls when defer_probe support is available.


How about the following untested draft patch?

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index be0eb46..f66912f 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -171,6 +171,12 @@ static void driver_deferred_probe_trigger(void)
queue_work(deferred_wq, _probe_work);
  }

+void driver_trigger_fw_load()
+{
+   driver_deferred_probe_trigger();
+}
+EXPORT_SYMBOL_GPL(driver_trigger_fw_load);
+
  /**
   * deferred_probe_initcall() - Enable probing of deferred devices
   *
diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
index 8524450..f879a07 100644
--- a/drivers/base/firmware_class.c
+++ b/drivers/base/firmware_class.c
@@ -1132,6 +1132,11 @@ _request_firmware(const struct firmware **firmware_p, 
const char *name,
if (ret <= 0) /* error or already assigned */
goto out;

+   if (system_state == SYSTEM_BOOTING) {
+   ret = -EPROBE_DEFER;
+   goto out;
+   }
+
ret = 0;
timeout = firmware_loading_timeout();
if (opt_flags & FW_OPT_NOWAIT) {
@@ -1311,6 +1316,9 @@ request_firmware_nowait(
  {
struct firmware_work *fw_work;

+   if (system_state == SYSTEM_BOOTING)
+   return -EPROBE_DEFER;
+


Does this mean a built-in driver can not get firmware from initramfs or 
built in the kernel early. Seems a bit too aggressive. The problem 
stated in this thread is when the firmware is not on initramfs but only 
on the rootfs.


Regards,
Arend


fw_work = kzalloc(sizeof(struct firmware_work), gfp);
if (!fw_work)
return -ENOMEM;
diff --git a/include/linux/device.h b/include/linux/device.h
index 5d7bc63..1c189fe 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -289,6 +289,7 @@ extern struct device_driver *driver_find(const char *name,
 struct bus_type *bus);
  extern int driver_probe_done(void);
  extern void wait_for_device_probe(void);
+extern void driver_trigger_fw_load(void);


  /* sysfs interface for exporting driver attributes */
diff --git a/init/main.c b/init/main.c
index 9e64d70..be8411b 100644
--- a/init/main.c
+++ b/init/main.c
@@ -943,6 +943,9 @@ static int __ref kernel_init(void *unused)

flush_delayed_fput();

+   /* trigger probe for request_firmware and its no_wait pair */
+   driver_trigger_fw_load();
+
if (ramdisk_execute_command) {

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 08/29/2015 12:38 PM, Ming Lei wrote:

On Sat, 29 Aug 2015 10:50:22 +0200
Arend van Spriel ar...@broadcom.com wrote:


On 08/29/2015 09:11 AM, Takashi Iwai wrote:

On Sat, 29 Aug 2015 06:09:01 +0200,
Ming Lei wrote:


On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez mcg...@suse.com wrote:

On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:

On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
torva...@linux-foundation.org wrote:

On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
liam.r.girdw...@linux.intel.com wrote:


I think the options are to either :-

1) Don not support audio DSP drivers using topology data as built-in
drivers. Audio is not really a critical system required for booting
anyway.


Yes, forcing it to be a module and not letting people compile it in by
mistake (and then not have it work) is an option.

That said, there are situations where people don't want to use
modules. I used to eschew them for security reasons, for example - now
I instead just do a one-time temporary key. But others may have other
reasons to try to avoid modules.


2) Create a default PCM for every driver that has topology data on the
assumption that every sound card will at least 1 PCM. This PCM can then
be re-configured when the FW is loaded.


That would seem to be the better option if it is reasonably implementable.

Of course, some kind of timer-based retry (limited *somehow*) of the
fw loading could work too, but smells really really hacky.


Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
which should be one kind of fix, but looks there were objections at that time.


That would still be a hack. I'll note there is also asynchronous probe support
now but to use that would also be a hack for this issue. We don't want to


If we think firmware as one kind of resources like regulators, gpio and others,
PROBE_DEFER is one good match for firmware loading case, and
it has been used by lots of drivers, so why can't it be used for
firmware loading?

One problem is that we need to convert drivers into returning -EPROBE_DEFER
in case of request failure, and that may involve some work, but which
should be mechanical.


I find such a delaying mechanism not so bad, too.  It's very
straightforward, at least, no big pain in the transition in the driver
side.


Not sure how this is going to work with request_firmware_nowait(). We
use that in our drivers to get rid of ~60 sec. delay in probe and
consequently boot time when built-in. So basically we return 0 on probe
lacking better knowledge. Guess we can always move back to
request_firmware calls when defer_probe support is available.


How about the following untested draft patch?

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index be0eb46..f66912f 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -171,6 +171,12 @@ static void driver_deferred_probe_trigger(void)
queue_work(deferred_wq, deferred_probe_work);
  }

+void driver_trigger_fw_load()
+{
+   driver_deferred_probe_trigger();
+}
+EXPORT_SYMBOL_GPL(driver_trigger_fw_load);
+
  /**
   * deferred_probe_initcall() - Enable probing of deferred devices
   *
diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
index 8524450..f879a07 100644
--- a/drivers/base/firmware_class.c
+++ b/drivers/base/firmware_class.c
@@ -1132,6 +1132,11 @@ _request_firmware(const struct firmware **firmware_p, 
const char *name,
if (ret = 0) /* error or already assigned */
goto out;

+   if (system_state == SYSTEM_BOOTING) {
+   ret = -EPROBE_DEFER;
+   goto out;
+   }
+
ret = 0;
timeout = firmware_loading_timeout();
if (opt_flags  FW_OPT_NOWAIT) {
@@ -1311,6 +1316,9 @@ request_firmware_nowait(
  {
struct firmware_work *fw_work;

+   if (system_state == SYSTEM_BOOTING)
+   return -EPROBE_DEFER;
+


Does this mean a built-in driver can not get firmware from initramfs or 
built in the kernel early. Seems a bit too aggressive. The problem 
stated in this thread is when the firmware is not on initramfs but only 
on the rootfs.


Regards,
Arend


fw_work = kzalloc(sizeof(struct firmware_work), gfp);
if (!fw_work)
return -ENOMEM;
diff --git a/include/linux/device.h b/include/linux/device.h
index 5d7bc63..1c189fe 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -289,6 +289,7 @@ extern struct device_driver *driver_find(const char *name,
 struct bus_type *bus);
  extern int driver_probe_done(void);
  extern void wait_for_device_probe(void);
+extern void driver_trigger_fw_load(void);


  /* sysfs interface for exporting driver attributes */
diff --git a/init/main.c b/init/main.c
index 9e64d70..be8411b 100644
--- a/init/main.c
+++ b/init/main.c
@@ -943,6 +943,9 @@ static int __ref kernel_init(void *unused)

flush_delayed_fput();

+   /* trigger probe for request_firmware and its no_wait 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Linus Torvalds]

On Sun, Aug 30, 2015 at 1:25 AM, Arend van Spriel ar...@broadcom.com wrote:
 On 08/29/2015 12:38 PM, Ming Lei wrote:

 Does this mean a built-in driver can not get firmware from initramfs or
 built in the kernel early. Seems a bit too aggressive.

Yeah, that seems wrong. Loading firmware from initramfs is required
for some things, like disk drivers. Of course, depending on how it's
done, it's all after the SYSTEM_BOOTING phase, but ..

What we *might* do is to not allow it for the user-mode helper
fallback, but I think it's more likely that we'll just deprecate the
usermode helper fw loader entirely, so adding new error cases for it
seems pointless.

Linus
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Sat, 29 Aug 2015 10:50:22 +0200
Arend van Spriel  wrote:

> On 08/29/2015 09:11 AM, Takashi Iwai wrote:
> > On Sat, 29 Aug 2015 06:09:01 +0200,
> > Ming Lei wrote:
> >>
> >> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:
> >>> On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
>  On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
>   wrote:
> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
> >  wrote:
> >>
> >> I think the options are to either :-
> >>
> >> 1) Don not support audio DSP drivers using topology data as built-in
> >> drivers. Audio is not really a critical system required for booting
> >> anyway.
> >
> > Yes, forcing it to be a module and not letting people compile it in by
> > mistake (and then not have it work) is an option.
> >
> > That said, there are situations where people don't want to use
> > modules. I used to eschew them for security reasons, for example - now
> > I instead just do a one-time temporary key. But others may have other
> > reasons to try to avoid modules.
> >
> >> 2) Create a default PCM for every driver that has topology data on the
> >> assumption that every sound card will at least 1 PCM. This PCM can then
> >> be re-configured when the FW is loaded.
> >
> > That would seem to be the better option if it is reasonably 
> > implementable.
> >
> > Of course, some kind of timer-based retry (limited *somehow*) of the
> > fw loading could work too, but smells really really hacky.
> 
>  Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
>  which should be one kind of fix, but looks there were objections at that 
>  time.
> >>>
> >>> That would still be a hack. I'll note there is also asynchronous probe 
> >>> support
> >>> now but to use that would also be a hack for this issue. We don't want to
> >>
> >> If we think firmware as one kind of resources like regulators, gpio and 
> >> others,
> >> PROBE_DEFER is one good match for firmware loading case, and
> >> it has been used by lots of drivers, so why can't it be used for
> >> firmware loading?
> >>
> >> One problem is that we need to convert drivers into returning -EPROBE_DEFER
> >> in case of request failure, and that may involve some work, but which
> >> should be mechanical.
> >
> > I find such a delaying mechanism not so bad, too.  It's very
> > straightforward, at least, no big pain in the transition in the driver
> > side.
> 
> Not sure how this is going to work with request_firmware_nowait(). We 
> use that in our drivers to get rid of ~60 sec. delay in probe and 
> consequently boot time when built-in. So basically we return 0 on probe 
> lacking better knowledge. Guess we can always move back to 
> request_firmware calls when defer_probe support is available.

How about the following untested draft patch?

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index be0eb46..f66912f 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -171,6 +171,12 @@ static void driver_deferred_probe_trigger(void)
queue_work(deferred_wq, _probe_work);
 }
 
+void driver_trigger_fw_load()
+{
+   driver_deferred_probe_trigger();
+}
+EXPORT_SYMBOL_GPL(driver_trigger_fw_load);
+
 /**
  * deferred_probe_initcall() - Enable probing of deferred devices
  *
diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
index 8524450..f879a07 100644
--- a/drivers/base/firmware_class.c
+++ b/drivers/base/firmware_class.c
@@ -1132,6 +1132,11 @@ _request_firmware(const struct firmware **firmware_p, 
const char *name,
if (ret <= 0) /* error or already assigned */
goto out;
 
+   if (system_state == SYSTEM_BOOTING) {
+   ret = -EPROBE_DEFER;
+   goto out;
+   }
+
ret = 0;
timeout = firmware_loading_timeout();
if (opt_flags & FW_OPT_NOWAIT) {
@@ -1311,6 +1316,9 @@ request_firmware_nowait(
 {
struct firmware_work *fw_work;
 
+   if (system_state == SYSTEM_BOOTING)
+   return -EPROBE_DEFER;
+
fw_work = kzalloc(sizeof(struct firmware_work), gfp);
if (!fw_work)
return -ENOMEM;
diff --git a/include/linux/device.h b/include/linux/device.h
index 5d7bc63..1c189fe 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -289,6 +289,7 @@ extern struct device_driver *driver_find(const char *name,
 struct bus_type *bus);
 extern int driver_probe_done(void);
 extern void wait_for_device_probe(void);
+extern void driver_trigger_fw_load(void);
 
 
 /* sysfs interface for exporting driver attributes */
diff --git a/init/main.c b/init/main.c
index 9e64d70..be8411b 100644
--- a/init/main.c
+++ b/init/main.c
@@ -943,6 +943,9 @@ static int __ref kernel_init(void *unused)
 
flush_delayed_fput();
 
+   /* trigger probe for request_firmware and its no_wait pair */
+   

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 08/29/2015 09:11 AM, Takashi Iwai wrote:

On Sat, 29 Aug 2015 06:09:01 +0200,
Ming Lei wrote:


On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:

On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:

On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
 wrote:

On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
 wrote:


I think the options are to either :-

1) Don not support audio DSP drivers using topology data as built-in
drivers. Audio is not really a critical system required for booting
anyway.


Yes, forcing it to be a module and not letting people compile it in by
mistake (and then not have it work) is an option.

That said, there are situations where people don't want to use
modules. I used to eschew them for security reasons, for example - now
I instead just do a one-time temporary key. But others may have other
reasons to try to avoid modules.


2) Create a default PCM for every driver that has topology data on the
assumption that every sound card will at least 1 PCM. This PCM can then
be re-configured when the FW is loaded.


That would seem to be the better option if it is reasonably implementable.

Of course, some kind of timer-based retry (limited *somehow*) of the
fw loading could work too, but smells really really hacky.


Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
which should be one kind of fix, but looks there were objections at that time.


That would still be a hack. I'll note there is also asynchronous probe support
now but to use that would also be a hack for this issue. We don't want to


If we think firmware as one kind of resources like regulators, gpio and others,
PROBE_DEFER is one good match for firmware loading case, and
it has been used by lots of drivers, so why can't it be used for
firmware loading?

One problem is that we need to convert drivers into returning -EPROBE_DEFER
in case of request failure, and that may involve some work, but which
should be mechanical.


I find such a delaying mechanism not so bad, too.  It's very
straightforward, at least, no big pain in the transition in the driver
side.


Not sure how this is going to work with request_firmware_nowait(). We 
use that in our drivers to get rid of ~60 sec. delay in probe and 
consequently boot time when built-in. So basically we return 0 on probe 
lacking better knowledge. Guess we can always move back to 
request_firmware calls when defer_probe support is available.


Regards,
Arend


encourage folks to go down that road.  They'd be hacks for this issue as you
are simply delaying the driver probe for a later time and there is no guarantee
that any pivot_root() might have already been completed later to ensure your
driver's fw file is present. So it may work or it may not.


We can trigger defer probe explicitly once root fs is setup or other condition
is met.


Right, how to trigger the reprobe (and relevant optimization) needs to
be considered on top of the current mechanism.


Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Takashi Iwai]

On Sat, 29 Aug 2015 06:09:01 +0200,
Ming Lei wrote:
> 
> On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:
> > On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
> >> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
> >>  wrote:
> >> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
> >> >  wrote:
> >> >>
> >> >> I think the options are to either :-
> >> >>
> >> >> 1) Don not support audio DSP drivers using topology data as built-in
> >> >> drivers. Audio is not really a critical system required for booting
> >> >> anyway.
> >> >
> >> > Yes, forcing it to be a module and not letting people compile it in by
> >> > mistake (and then not have it work) is an option.
> >> >
> >> > That said, there are situations where people don't want to use
> >> > modules. I used to eschew them for security reasons, for example - now
> >> > I instead just do a one-time temporary key. But others may have other
> >> > reasons to try to avoid modules.
> >> >
> >> >> 2) Create a default PCM for every driver that has topology data on the
> >> >> assumption that every sound card will at least 1 PCM. This PCM can then
> >> >> be re-configured when the FW is loaded.
> >> >
> >> > That would seem to be the better option if it is reasonably 
> >> > implementable.
> >> >
> >> > Of course, some kind of timer-based retry (limited *somehow*) of the
> >> > fw loading could work too, but smells really really hacky.
> >>
> >> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
> >> which should be one kind of fix, but looks there were objections at that 
> >> time.
> >
> > That would still be a hack. I'll note there is also asynchronous probe 
> > support
> > now but to use that would also be a hack for this issue. We don't want to
> 
> If we think firmware as one kind of resources like regulators, gpio and 
> others,
> PROBE_DEFER is one good match for firmware loading case, and
> it has been used by lots of drivers, so why can't it be used for
> firmware loading?
> 
> One problem is that we need to convert drivers into returning -EPROBE_DEFER
> in case of request failure, and that may involve some work, but which
> should be mechanical.

I find such a delaying mechanism not so bad, too.  It's very
straightforward, at least, no big pain in the transition in the driver
side.

> > encourage folks to go down that road.  They'd be hacks for this issue as you
> > are simply delaying the driver probe for a later time and there is no 
> > guarantee
> > that any pivot_root() might have already been completed later to ensure your
> > driver's fw file is present. So it may work or it may not.
> 
> We can trigger defer probe explicitly once root fs is setup or other condition
> is met.

Right, how to trigger the reprobe (and relevant optimization) needs to
be considered on top of the current mechanism.


Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Arend van Spriel]

On 08/29/2015 09:11 AM, Takashi Iwai wrote:

On Sat, 29 Aug 2015 06:09:01 +0200,
Ming Lei wrote:


On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez mcg...@suse.com wrote:

On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:

On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
torva...@linux-foundation.org wrote:

On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
liam.r.girdw...@linux.intel.com wrote:


I think the options are to either :-

1) Don not support audio DSP drivers using topology data as built-in
drivers. Audio is not really a critical system required for booting
anyway.


Yes, forcing it to be a module and not letting people compile it in by
mistake (and then not have it work) is an option.

That said, there are situations where people don't want to use
modules. I used to eschew them for security reasons, for example - now
I instead just do a one-time temporary key. But others may have other
reasons to try to avoid modules.


2) Create a default PCM for every driver that has topology data on the
assumption that every sound card will at least 1 PCM. This PCM can then
be re-configured when the FW is loaded.


That would seem to be the better option if it is reasonably implementable.

Of course, some kind of timer-based retry (limited *somehow*) of the
fw loading could work too, but smells really really hacky.


Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
which should be one kind of fix, but looks there were objections at that time.


That would still be a hack. I'll note there is also asynchronous probe support
now but to use that would also be a hack for this issue. We don't want to


If we think firmware as one kind of resources like regulators, gpio and others,
PROBE_DEFER is one good match for firmware loading case, and
it has been used by lots of drivers, so why can't it be used for
firmware loading?

One problem is that we need to convert drivers into returning -EPROBE_DEFER
in case of request failure, and that may involve some work, but which
should be mechanical.


I find such a delaying mechanism not so bad, too.  It's very
straightforward, at least, no big pain in the transition in the driver
side.


Not sure how this is going to work with request_firmware_nowait(). We 
use that in our drivers to get rid of ~60 sec. delay in probe and 
consequently boot time when built-in. So basically we return 0 on probe 
lacking better knowledge. Guess we can always move back to 
request_firmware calls when defer_probe support is available.


Regards,
Arend


encourage folks to go down that road.  They'd be hacks for this issue as you
are simply delaying the driver probe for a later time and there is no guarantee
that any pivot_root() might have already been completed later to ensure your
driver's fw file is present. So it may work or it may not.


We can trigger defer probe explicitly once root fs is setup or other condition
is met.


Right, how to trigger the reprobe (and relevant optimization) needs to
be considered on top of the current mechanism.


Takashi
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/



--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Takashi Iwai]

On Sat, 29 Aug 2015 06:09:01 +0200,
Ming Lei wrote:
 
 On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez mcg...@suse.com wrote:
  On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
  On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
  torva...@linux-foundation.org wrote:
   On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
   liam.r.girdw...@linux.intel.com wrote:
  
   I think the options are to either :-
  
   1) Don not support audio DSP drivers using topology data as built-in
   drivers. Audio is not really a critical system required for booting
   anyway.
  
   Yes, forcing it to be a module and not letting people compile it in by
   mistake (and then not have it work) is an option.
  
   That said, there are situations where people don't want to use
   modules. I used to eschew them for security reasons, for example - now
   I instead just do a one-time temporary key. But others may have other
   reasons to try to avoid modules.
  
   2) Create a default PCM for every driver that has topology data on the
   assumption that every sound card will at least 1 PCM. This PCM can then
   be re-configured when the FW is loaded.
  
   That would seem to be the better option if it is reasonably 
   implementable.
  
   Of course, some kind of timer-based retry (limited *somehow*) of the
   fw loading could work too, but smells really really hacky.
 
  Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
  which should be one kind of fix, but looks there were objections at that 
  time.
 
  That would still be a hack. I'll note there is also asynchronous probe 
  support
  now but to use that would also be a hack for this issue. We don't want to
 
 If we think firmware as one kind of resources like regulators, gpio and 
 others,
 PROBE_DEFER is one good match for firmware loading case, and
 it has been used by lots of drivers, so why can't it be used for
 firmware loading?
 
 One problem is that we need to convert drivers into returning -EPROBE_DEFER
 in case of request failure, and that may involve some work, but which
 should be mechanical.

I find such a delaying mechanism not so bad, too.  It's very
straightforward, at least, no big pain in the transition in the driver
side.

  encourage folks to go down that road.  They'd be hacks for this issue as you
  are simply delaying the driver probe for a later time and there is no 
  guarantee
  that any pivot_root() might have already been completed later to ensure your
  driver's fw file is present. So it may work or it may not.
 
 We can trigger defer probe explicitly once root fs is setup or other condition
 is met.

Right, how to trigger the reprobe (and relevant optimization) needs to
be considered on top of the current mechanism.


Takashi
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Sat, 29 Aug 2015 10:50:22 +0200
Arend van Spriel ar...@broadcom.com wrote:

 On 08/29/2015 09:11 AM, Takashi Iwai wrote:
  On Sat, 29 Aug 2015 06:09:01 +0200,
  Ming Lei wrote:
 
  On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez mcg...@suse.com wrote:
  On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
  On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
  torva...@linux-foundation.org wrote:
  On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
  liam.r.girdw...@linux.intel.com wrote:
 
  I think the options are to either :-
 
  1) Don not support audio DSP drivers using topology data as built-in
  drivers. Audio is not really a critical system required for booting
  anyway.
 
  Yes, forcing it to be a module and not letting people compile it in by
  mistake (and then not have it work) is an option.
 
  That said, there are situations where people don't want to use
  modules. I used to eschew them for security reasons, for example - now
  I instead just do a one-time temporary key. But others may have other
  reasons to try to avoid modules.
 
  2) Create a default PCM for every driver that has topology data on the
  assumption that every sound card will at least 1 PCM. This PCM can then
  be re-configured when the FW is loaded.
 
  That would seem to be the better option if it is reasonably 
  implementable.
 
  Of course, some kind of timer-based retry (limited *somehow*) of the
  fw loading could work too, but smells really really hacky.
 
  Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
  which should be one kind of fix, but looks there were objections at that 
  time.
 
  That would still be a hack. I'll note there is also asynchronous probe 
  support
  now but to use that would also be a hack for this issue. We don't want to
 
  If we think firmware as one kind of resources like regulators, gpio and 
  others,
  PROBE_DEFER is one good match for firmware loading case, and
  it has been used by lots of drivers, so why can't it be used for
  firmware loading?
 
  One problem is that we need to convert drivers into returning -EPROBE_DEFER
  in case of request failure, and that may involve some work, but which
  should be mechanical.
 
  I find such a delaying mechanism not so bad, too.  It's very
  straightforward, at least, no big pain in the transition in the driver
  side.
 
 Not sure how this is going to work with request_firmware_nowait(). We 
 use that in our drivers to get rid of ~60 sec. delay in probe and 
 consequently boot time when built-in. So basically we return 0 on probe 
 lacking better knowledge. Guess we can always move back to 
 request_firmware calls when defer_probe support is available.

How about the following untested draft patch?

diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index be0eb46..f66912f 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -171,6 +171,12 @@ static void driver_deferred_probe_trigger(void)
queue_work(deferred_wq, deferred_probe_work);
 }
 
+void driver_trigger_fw_load()
+{
+   driver_deferred_probe_trigger();
+}
+EXPORT_SYMBOL_GPL(driver_trigger_fw_load);
+
 /**
  * deferred_probe_initcall() - Enable probing of deferred devices
  *
diff --git a/drivers/base/firmware_class.c b/drivers/base/firmware_class.c
index 8524450..f879a07 100644
--- a/drivers/base/firmware_class.c
+++ b/drivers/base/firmware_class.c
@@ -1132,6 +1132,11 @@ _request_firmware(const struct firmware **firmware_p, 
const char *name,
if (ret = 0) /* error or already assigned */
goto out;
 
+   if (system_state == SYSTEM_BOOTING) {
+   ret = -EPROBE_DEFER;
+   goto out;
+   }
+
ret = 0;
timeout = firmware_loading_timeout();
if (opt_flags  FW_OPT_NOWAIT) {
@@ -1311,6 +1316,9 @@ request_firmware_nowait(
 {
struct firmware_work *fw_work;
 
+   if (system_state == SYSTEM_BOOTING)
+   return -EPROBE_DEFER;
+
fw_work = kzalloc(sizeof(struct firmware_work), gfp);
if (!fw_work)
return -ENOMEM;
diff --git a/include/linux/device.h b/include/linux/device.h
index 5d7bc63..1c189fe 100644
--- a/include/linux/device.h
+++ b/include/linux/device.h
@@ -289,6 +289,7 @@ extern struct device_driver *driver_find(const char *name,
 struct bus_type *bus);
 extern int driver_probe_done(void);
 extern void wait_for_device_probe(void);
+extern void driver_trigger_fw_load(void);
 
 
 /* sysfs interface for exporting driver attributes */
diff --git a/init/main.c b/init/main.c
index 9e64d70..be8411b 100644
--- a/init/main.c
+++ b/init/main.c
@@ -943,6 +943,9 @@ static int __ref kernel_init(void *unused)
 
flush_delayed_fput();
 
+   /* trigger probe for request_firmware and its no_wait pair */
+   driver_trigger_fw_load();
+
if (ramdisk_execute_command) {
ret = run_init_process(ramdisk_execute_command);
if (!ret)



Thanks,
--
To 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez  wrote:
> On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
>> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
>>  wrote:
>> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
>> >  wrote:
>> >>
>> >> I think the options are to either :-
>> >>
>> >> 1) Don not support audio DSP drivers using topology data as built-in
>> >> drivers. Audio is not really a critical system required for booting
>> >> anyway.
>> >
>> > Yes, forcing it to be a module and not letting people compile it in by
>> > mistake (and then not have it work) is an option.
>> >
>> > That said, there are situations where people don't want to use
>> > modules. I used to eschew them for security reasons, for example - now
>> > I instead just do a one-time temporary key. But others may have other
>> > reasons to try to avoid modules.
>> >
>> >> 2) Create a default PCM for every driver that has topology data on the
>> >> assumption that every sound card will at least 1 PCM. This PCM can then
>> >> be re-configured when the FW is loaded.
>> >
>> > That would seem to be the better option if it is reasonably implementable.
>> >
>> > Of course, some kind of timer-based retry (limited *somehow*) of the
>> > fw loading could work too, but smells really really hacky.
>>
>> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
>> which should be one kind of fix, but looks there were objections at that 
>> time.
>
> That would still be a hack. I'll note there is also asynchronous probe support
> now but to use that would also be a hack for this issue. We don't want to

If we think firmware as one kind of resources like regulators, gpio and others,
PROBE_DEFER is one good match for firmware loading case, and
it has been used by lots of drivers, so why can't it be used for
firmware loading?

One problem is that we need to convert drivers into returning -EPROBE_DEFER
in case of request failure, and that may involve some work, but which
should be mechanical.

> encourage folks to go down that road.  They'd be hacks for this issue as you
> are simply delaying the driver probe for a later time and there is no 
> guarantee
> that any pivot_root() might have already been completed later to ensure your
> driver's fw file is present. So it may work or it may not.

We can trigger defer probe explicitly once root fs is setup or other condition
is met.

>
> We should instead strive to be clear about expectations and requirements both
> through documentation and when possible through APIs. I'll send out an RFC
> which adds some grammar rules which can help us police this. I currently only
> spot two drivers that require fixing.
>
>   Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
> On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
>  wrote:
> > On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
> >  wrote:
> >>
> >> I think the options are to either :-
> >>
> >> 1) Don not support audio DSP drivers using topology data as built-in
> >> drivers. Audio is not really a critical system required for booting
> >> anyway.
> >
> > Yes, forcing it to be a module and not letting people compile it in by
> > mistake (and then not have it work) is an option.
> >
> > That said, there are situations where people don't want to use
> > modules. I used to eschew them for security reasons, for example - now
> > I instead just do a one-time temporary key. But others may have other
> > reasons to try to avoid modules.
> >
> >> 2) Create a default PCM for every driver that has topology data on the
> >> assumption that every sound card will at least 1 PCM. This PCM can then
> >> be re-configured when the FW is loaded.
> >
> > That would seem to be the better option if it is reasonably implementable.
> >
> > Of course, some kind of timer-based retry (limited *somehow*) of the
> > fw loading could work too, but smells really really hacky.
> 
> Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
> which should be one kind of fix, but looks there were objections at that time.

That would still be a hack. I'll note there is also asynchronous probe support
now but to use that would also be a hack for this issue. We don't want to
encourage folks to go down that road.  They'd be hacks for this issue as you
are simply delaying the driver probe for a later time and there is no guarantee
that any pivot_root() might have already been completed later to ensure your
driver's fw file is present. So it may work or it may not.

We should instead strive to be clear about expectations and requirements both
through documentation and when possible through APIs. I'll send out an RFC
which adds some grammar rules which can help us police this. I currently only
spot two drivers that require fixing.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Sat, Aug 29, 2015 at 9:11 AM, Luis R. Rodriguez mcg...@suse.com wrote:
 On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
 On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
 torva...@linux-foundation.org wrote:
  On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
  liam.r.girdw...@linux.intel.com wrote:
 
  I think the options are to either :-
 
  1) Don not support audio DSP drivers using topology data as built-in
  drivers. Audio is not really a critical system required for booting
  anyway.
 
  Yes, forcing it to be a module and not letting people compile it in by
  mistake (and then not have it work) is an option.
 
  That said, there are situations where people don't want to use
  modules. I used to eschew them for security reasons, for example - now
  I instead just do a one-time temporary key. But others may have other
  reasons to try to avoid modules.
 
  2) Create a default PCM for every driver that has topology data on the
  assumption that every sound card will at least 1 PCM. This PCM can then
  be re-configured when the FW is loaded.
 
  That would seem to be the better option if it is reasonably implementable.
 
  Of course, some kind of timer-based retry (limited *somehow*) of the
  fw loading could work too, but smells really really hacky.

 Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
 which should be one kind of fix, but looks there were objections at that 
 time.

 That would still be a hack. I'll note there is also asynchronous probe support
 now but to use that would also be a hack for this issue. We don't want to

If we think firmware as one kind of resources like regulators, gpio and others,
PROBE_DEFER is one good match for firmware loading case, and
it has been used by lots of drivers, so why can't it be used for
firmware loading?

One problem is that we need to convert drivers into returning -EPROBE_DEFER
in case of request failure, and that may involve some work, but which
should be mechanical.

 encourage folks to go down that road.  They'd be hacks for this issue as you
 are simply delaying the driver probe for a later time and there is no 
 guarantee
 that any pivot_root() might have already been completed later to ensure your
 driver's fw file is present. So it may work or it may not.

We can trigger defer probe explicitly once root fs is setup or other condition
is met.


 We should instead strive to be clear about expectations and requirements both
 through documentation and when possible through APIs. I'll send out an RFC
 which adds some grammar rules which can help us police this. I currently only
 spot two drivers that require fixing.

   Luis
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Thu, Aug 27, 2015 at 08:55:13AM +0800, Ming Lei wrote:
 On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
 torva...@linux-foundation.org wrote:
  On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
  liam.r.girdw...@linux.intel.com wrote:
 
  I think the options are to either :-
 
  1) Don not support audio DSP drivers using topology data as built-in
  drivers. Audio is not really a critical system required for booting
  anyway.
 
  Yes, forcing it to be a module and not letting people compile it in by
  mistake (and then not have it work) is an option.
 
  That said, there are situations where people don't want to use
  modules. I used to eschew them for security reasons, for example - now
  I instead just do a one-time temporary key. But others may have other
  reasons to try to avoid modules.
 
  2) Create a default PCM for every driver that has topology data on the
  assumption that every sound card will at least 1 PCM. This PCM can then
  be re-configured when the FW is loaded.
 
  That would seem to be the better option if it is reasonably implementable.
 
  Of course, some kind of timer-based retry (limited *somehow*) of the
  fw loading could work too, but smells really really hacky.
 
 Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
 which should be one kind of fix, but looks there were objections at that time.

That would still be a hack. I'll note there is also asynchronous probe support
now but to use that would also be a hack for this issue. We don't want to
encourage folks to go down that road.  They'd be hacks for this issue as you
are simply delaying the driver probe for a later time and there is no guarantee
that any pivot_root() might have already been completed later to ensure your
driver's fw file is present. So it may work or it may not.

We should instead strive to be clear about expectations and requirements both
through documentation and when possible through APIs. I'll send out an RFC
which adds some grammar rules which can help us police this. I currently only
spot two drivers that require fixing.

  Luis
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Liam Girdwood]

On Thu, 2015-08-27 at 01:50 +, Lin, Mengdong wrote:
> > -Original Message-
> > From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
> > Sent: Wednesday, August 26, 2015 5:01 PM
> > To: Jie, Yang
> > Cc: Takashi Iwai; Dmitry Torokhov; Luis R. Rodriguez;
> > joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> > Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis Rodriguez;
> > lkml; yalin wang; Lin, Mengdong
> > Subject: Re: Problems loading firmware using built-in drivers with kernels 
> > that
> > use initramfs.
> > 
> > On Wed, 2015-08-26 at 08:29 +, Jie, Yang wrote:
> > > > -Original Message-
> > > > From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
> > 
> > > > I think the options are to either :-
> > > >
> > > > 1) Don not support audio DSP drivers using topology data as built-in
> > drivers.
> > > > Audio is not really a critical system required for booting anyway.
> > > >
> > > > 2) Create a default PCM for every driver that has topology data on
> > > > the assumption that every sound card will at least 1 PCM. This PCM
> > > > can then be re-configured when the FW is loaded.
> > >
> > > Yep, this case is quite similar with what Linus described.
> > >
> > > Is it possible that we can probe pcm device after firmware is loaded
> > > for this case?
> > >
> > 
> > The PCM devices are defined in the topology data so it is only possible to
> > create the PCM device *after* the firmware is loaded in these drivers.
> > 
> > Liam
> 
> It seems this can prevent audio device driver in built-in mode to use 
> topology drivers.
> 
> If topology is present, the vendor driver also uses request_firmware() to 
> load the topology data file in the probe phase. Then ASoC core will create 
> the sound card and its PCM devices. 
> 
> Shall we force the device drivers that depends on topology to be configured 
> as modules?

Yes, this is probably the best/quickest solution for the short term
until we can implement and test option 2.

Liam

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Liam Girdwood]

On Thu, 2015-08-27 at 01:50 +, Lin, Mengdong wrote:
  -Original Message-
  From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
  Sent: Wednesday, August 26, 2015 5:01 PM
  To: Jie, Yang
  Cc: Takashi Iwai; Dmitry Torokhov; Luis R. Rodriguez;
  joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
  Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis Rodriguez;
  lkml; yalin wang; Lin, Mengdong
  Subject: Re: Problems loading firmware using built-in drivers with kernels 
  that
  use initramfs.
  
  On Wed, 2015-08-26 at 08:29 +, Jie, Yang wrote:
-Original Message-
From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
  
I think the options are to either :-
   
1) Don not support audio DSP drivers using topology data as built-in
  drivers.
Audio is not really a critical system required for booting anyway.
   
2) Create a default PCM for every driver that has topology data on
the assumption that every sound card will at least 1 PCM. This PCM
can then be re-configured when the FW is loaded.
  
   Yep, this case is quite similar with what Linus described.
  
   Is it possible that we can probe pcm device after firmware is loaded
   for this case?
  
  
  The PCM devices are defined in the topology data so it is only possible to
  create the PCM device *after* the firmware is loaded in these drivers.
  
  Liam
 
 It seems this can prevent audio device driver in built-in mode to use 
 topology drivers.
 
 If topology is present, the vendor driver also uses request_firmware() to 
 load the topology data file in the probe phase. Then ASoC core will create 
 the sound card and its PCM devices. 
 
 Shall we force the device drivers that depends on topology to be configured 
 as modules?

Yes, this is probably the best/quickest solution for the short term
until we can implement and test option 2.

Liam

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Lin, Mengdong]

> -Original Message-
> From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
> Sent: Wednesday, August 26, 2015 5:01 PM
> To: Jie, Yang
> Cc: Takashi Iwai; Dmitry Torokhov; Luis R. Rodriguez;
> joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis Rodriguez;
> lkml; yalin wang; Lin, Mengdong
> Subject: Re: Problems loading firmware using built-in drivers with kernels 
> that
> use initramfs.
> 
> On Wed, 2015-08-26 at 08:29 +, Jie, Yang wrote:
> > > -Original Message-
> > > From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
> 
> > > I think the options are to either :-
> > >
> > > 1) Don not support audio DSP drivers using topology data as built-in
> drivers.
> > > Audio is not really a critical system required for booting anyway.
> > >
> > > 2) Create a default PCM for every driver that has topology data on
> > > the assumption that every sound card will at least 1 PCM. This PCM
> > > can then be re-configured when the FW is loaded.
> >
> > Yep, this case is quite similar with what Linus described.
> >
> > Is it possible that we can probe pcm device after firmware is loaded
> > for this case?
> >
> 
> The PCM devices are defined in the topology data so it is only possible to
> create the PCM device *after* the firmware is loaded in these drivers.
> 
> Liam

It seems this can prevent audio device driver in built-in mode to use topology 
drivers.

If topology is present, the vendor driver also uses request_firmware() to load 
the topology data file in the probe phase. Then ASoC core will create the sound 
card and its PCM devices. 

Shall we force the device drivers that depends on topology to be configured as 
modules?

Thanks
Mengdong 

N�r��yb�X��ǧv�^�)޺{.n�+{zX����ܨ}���Ơz�:+v���zZ+��+zf���h���~i���z��w���?�&�)ߢf��^jǫy�m��@A�a���
0��h���i

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
 wrote:
> On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
>  wrote:
>>
>> I think the options are to either :-
>>
>> 1) Don not support audio DSP drivers using topology data as built-in
>> drivers. Audio is not really a critical system required for booting
>> anyway.
>
> Yes, forcing it to be a module and not letting people compile it in by
> mistake (and then not have it work) is an option.
>
> That said, there are situations where people don't want to use
> modules. I used to eschew them for security reasons, for example - now
> I instead just do a one-time temporary key. But others may have other
> reasons to try to avoid modules.
>
>> 2) Create a default PCM for every driver that has topology data on the
>> assumption that every sound card will at least 1 PCM. This PCM can then
>> be re-configured when the FW is loaded.
>
> That would seem to be the better option if it is reasonably implementable.
>
> Of course, some kind of timer-based retry (limited *somehow*) of the
> fw loading could work too, but smells really really hacky.

Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
which should be one kind of fix, but looks there were objections at that time.

thanks,
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Linus Torvalds]

On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
 wrote:
>
> I think the options are to either :-
>
> 1) Don not support audio DSP drivers using topology data as built-in
> drivers. Audio is not really a critical system required for booting
> anyway.

Yes, forcing it to be a module and not letting people compile it in by
mistake (and then not have it work) is an option.

That said, there are situations where people don't want to use
modules. I used to eschew them for security reasons, for example - now
I instead just do a one-time temporary key. But others may have other
reasons to try to avoid modules.

> 2) Create a default PCM for every driver that has topology data on the
> assumption that every sound card will at least 1 PCM. This PCM can then
> be re-configured when the FW is loaded.

That would seem to be the better option if it is reasonably implementable.

Of course, some kind of timer-based retry (limited *somehow*) of the
fw loading could work too, but smells really really hacky.

  Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Liam Girdwood]

On Wed, 2015-08-26 at 08:29 +, Jie, Yang wrote:
> > -Original Message-
> > From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]

> > I think the options are to either :-
> > 
> > 1) Don not support audio DSP drivers using topology data as built-in 
> > drivers.
> > Audio is not really a critical system required for booting anyway.
> > 
> > 2) Create a default PCM for every driver that has topology data on the
> > assumption that every sound card will at least 1 PCM. This PCM can then be
> > re-configured when the FW is loaded.
> 
> Yep, this case is quite similar with what Linus described.
>  
> Is it possible that we can probe pcm device after firmware is loaded for this
> case?
> 

The PCM devices are defined in the topology data so it is only possible
to create the PCM device *after* the firmware is loaded in these
drivers. 

Liam

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Jie, Yang]

> -Original Message-
> From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
> Sent: Wednesday, August 26, 2015 4:07 PM
> To: Jie, Yang
> Cc: Takashi Iwai; Dmitry Torokhov; Luis R. Rodriguez;
> joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
> Rodriguez; lkml; yalin wang
> Subject: Re: Problems loading firmware using built-in drivers with kernels
> that use initramfs.
> 
> On Wed, 2015-08-26 at 07:17 +0100, Jie, Yang wrote:
> > > -Original Message-
> > > From: Takashi Iwai [mailto:ti...@suse.de]
> > > Sent: Wednesday, August 26, 2015 1:33 PM
> > > To: Jie, Yang
> > > Cc: Dmitry Torokhov; Luis R. Rodriguez; Girdwood, Liam R;
> > > joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro;
> > > Greg Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse;
> > > Luis Rodriguez; lkml
> > > Subject: Re: Problems loading firmware using built-in drivers with
> > > kernels that use initramfs.
> > >
> > > It's possible -- and even with the normal request_firmware(), in
> > > theory.  The missing piece is that you need to inform the helper to retry
> the f/w loading.
> > > Currently the direct f/w loader assumes that the file is there and
> > > returns error if not present.
> > >
> > > If we implement a retry, another question is how to trigger it, i.e.
> > > how the helper can know when a fs is mounted a new file is present.
> >
> > Got it, thanks Takashi and all.
> >
> > So looks we should(and already done for most audio firmware) implement
> > it as Dmitry and Linus proposed, that is to make sure
> > request_firmware() calls are not in driver's probe() paths.
> >
> > Yalin help posted sample code for this implementation(to request
> > firmware in device node open) and actually I also did similar for
> > Broadwell ADSP driver and will send out later, thank you all.
> 
> Keyon, we have a similar situation to that of the FPGA example that Linus
> described when the DSP driver uses topology data. i.e. the audio driver will
> have no PCM devices that can be opened (to initiate the FW
> loading) until after the FW is loaded. The topology data is part of the FW and
> contains the PCMs, kcontrols, graphs etc. i.e. a PCM device is created for
> every PCM description found in the FW topology data.
> 
> I think the options are to either :-
> 
> 1) Don not support audio DSP drivers using topology data as built-in drivers.
> Audio is not really a critical system required for booting anyway.
> 
> 2) Create a default PCM for every driver that has topology data on the
> assumption that every sound card will at least 1 PCM. This PCM can then be
> re-configured when the FW is loaded.

Yep, this case is quite similar with what Linus described.
 
Is it possible that we can probe pcm device after firmware is loaded for this
case?

What I am thinking about is postponing all firmware related(e.g. pcm probing,
dma/dsp/ipc initialization, ...) code to that rootfs is ready and FW is loaded.

~Keyon

> 
> Liam
> 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Liam Girdwood]

On Wed, 2015-08-26 at 07:17 +0100, Jie, Yang wrote:
> > -Original Message-
> > From: Takashi Iwai [mailto:ti...@suse.de]
> > Sent: Wednesday, August 26, 2015 1:33 PM
> > To: Jie, Yang
> > Cc: Dmitry Torokhov; Luis R. Rodriguez; Girdwood, Liam R;
> > joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> > Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
> > Rodriguez; lkml
> > Subject: Re: Problems loading firmware using built-in drivers with kernels
> > that use initramfs.
> > 
> > On Wed, 26 Aug 2015 07:12:46 +0200,
> > Jie, Yang wrote:
> > >
> > > > -Original Message-
> > > > From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
> > > > Sent: Wednesday, August 26, 2015 3:58 AM
> > > > To: Takashi Iwai
> > > > Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
> > > > joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro;
> > > > Greg Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse;
> > > > Luis Rodriguez; lkml
> > > > Subject: Re: Problems loading firmware using built-in drivers with
> > > > kernels that use initramfs.
> > > >
> > > > On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai  wrote:
> > > > > On Tue, 25 Aug 2015 21:34:08 +0200, Luis R. Rodriguez wrote:
> > > > >>
> > > > >> On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
> > > > >> > Luis, did you tell me the other day that you made the kernel
> > > > >> > get firmware directly from the file system? This regression
> > > > >> > would be yours
> > > > then?
> > > > >>
> > > > >> I didn't implement that, Linus did in 2012 (see commit
> > > > >> abb139e75c2c titled
> > > > >> "firmware: teach the kernel to load firmware files directly from
> > > > >> the filesystem"). But we used to fallback to a userspace helper
> > > > >> when the fw was not present and then Takashi made this optional
> > > > >> via commit 7b1269f778782d titled "firmware: Make user-mode helper
> > optional".
> > > > >> Takashi noted in the Kconfig "The user-mode helper is no longer
> > > > >> required unless you have a special firmware file that resides in
> > > > >> a non-standard path". It was not clarified why that's true
> > > > >> though, or what you'd need to do to ensure that the fw would be
> > > > >> available. It would be good for us to elaborate on that.
> > > > >
> > > > > The recent udev already dropped the firmware loading feature.
> > > >
> > > > Note that even when we had udev helper to load the firmware it was
> > > > not always reliable depending on the exact point where we requested
> > firmware.
> > > > If request happened in probe() path before we mounted root fs then
> > > > we'd never get it loaded because we'd be waiting for devices settle
> > > > before mounting rootfs.
> > >
> > > For request in probe(), is it possible to use
> > > request_firmware_nowait() to wait rootfs mounted or timeout in another
> > thread?
> > >
> > > It looks usermodehelper_disabled is 0(at probe()) at this case then no
> > > waiting occurs here in our testing.
> > 
> > It's possible -- and even with the normal request_firmware(), in theory.  
> > The
> > missing piece is that you need to inform the helper to retry the f/w 
> > loading.
> > Currently the direct f/w loader assumes that the file is there and returns
> > error if not present.
> > 
> > If we implement a retry, another question is how to trigger it, i.e. how the
> > helper can know when a fs is mounted a new file is present.
>  
> Got it, thanks Takashi and all.
> 
> So looks we should(and already done for most audio firmware) implement it 
> as Dmitry and Linus proposed, that is to make sure request_firmware() calls
> are not in driver's probe() paths.
> 
> Yalin help posted sample code for this implementation(to request firmware
> in device node open) and actually I also did similar for Broadwell ADSP driver
> and will send out later, thank you all.

Keyon, we have a similar situation to that of the FPGA example that
Linus described when the DSP driver uses topology data. i.e. the audio
driver will have no PCM devices that can be opened (to initiate the FW
loading) until after the FW is loaded. The topology data is part of the
FW and contains the PCMs, kcontrols, graphs etc. i.e. a PCM device is
created for every PCM description found in the FW topology data. 

I think the options are to either :-

1) Don not support audio DSP drivers using topology data as built-in
drivers. Audio is not really a critical system required for booting
anyway.

2) Create a default PCM for every driver that has topology data on the
assumption that every sound card will at least 1 PCM. This PCM can then
be re-configured when the FW is loaded. 

Liam


--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Jie, Yang]

> -Original Message-
> From: Takashi Iwai [mailto:ti...@suse.de]
> Sent: Wednesday, August 26, 2015 1:33 PM
> To: Jie, Yang
> Cc: Dmitry Torokhov; Luis R. Rodriguez; Girdwood, Liam R;
> joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
> Rodriguez; lkml
> Subject: Re: Problems loading firmware using built-in drivers with kernels
> that use initramfs.
> 
> On Wed, 26 Aug 2015 07:12:46 +0200,
> Jie, Yang wrote:
> >
> > > -Original Message-
> > > From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
> > > Sent: Wednesday, August 26, 2015 3:58 AM
> > > To: Takashi Iwai
> > > Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
> > > joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro;
> > > Greg Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse;
> > > Luis Rodriguez; lkml
> > > Subject: Re: Problems loading firmware using built-in drivers with
> > > kernels that use initramfs.
> > >
> > > On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai  wrote:
> > > > On Tue, 25 Aug 2015 21:34:08 +0200, Luis R. Rodriguez wrote:
> > > >>
> > > >> On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
> > > >> > Luis, did you tell me the other day that you made the kernel
> > > >> > get firmware directly from the file system? This regression
> > > >> > would be yours
> > > then?
> > > >>
> > > >> I didn't implement that, Linus did in 2012 (see commit
> > > >> abb139e75c2c titled
> > > >> "firmware: teach the kernel to load firmware files directly from
> > > >> the filesystem"). But we used to fallback to a userspace helper
> > > >> when the fw was not present and then Takashi made this optional
> > > >> via commit 7b1269f778782d titled "firmware: Make user-mode helper
> optional".
> > > >> Takashi noted in the Kconfig "The user-mode helper is no longer
> > > >> required unless you have a special firmware file that resides in
> > > >> a non-standard path". It was not clarified why that's true
> > > >> though, or what you'd need to do to ensure that the fw would be
> > > >> available. It would be good for us to elaborate on that.
> > > >
> > > > The recent udev already dropped the firmware loading feature.
> > >
> > > Note that even when we had udev helper to load the firmware it was
> > > not always reliable depending on the exact point where we requested
> firmware.
> > > If request happened in probe() path before we mounted root fs then
> > > we'd never get it loaded because we'd be waiting for devices settle
> > > before mounting rootfs.
> >
> > For request in probe(), is it possible to use
> > request_firmware_nowait() to wait rootfs mounted or timeout in another
> thread?
> >
> > It looks usermodehelper_disabled is 0(at probe()) at this case then no
> > waiting occurs here in our testing.
> 
> It's possible -- and even with the normal request_firmware(), in theory.  The
> missing piece is that you need to inform the helper to retry the f/w loading.
> Currently the direct f/w loader assumes that the file is there and returns
> error if not present.
> 
> If we implement a retry, another question is how to trigger it, i.e. how the
> helper can know when a fs is mounted a new file is present.
 
Got it, thanks Takashi and all.

So looks we should(and already done for most audio firmware) implement it 
as Dmitry and Linus proposed, that is to make sure request_firmware() calls
are not in driver's probe() paths.

Yalin help posted sample code for this implementation(to request firmware
in device node open) and actually I also did similar for Broadwell ADSP driver
and will send out later, thank you all.

Thanks,
~Keyon

> 
> 
> Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Jie, Yang]

 -Original Message-
 From: Takashi Iwai [mailto:ti...@suse.de]
 Sent: Wednesday, August 26, 2015 1:33 PM
 To: Jie, Yang
 Cc: Dmitry Torokhov; Luis R. Rodriguez; Girdwood, Liam R;
 joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
 Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
 Rodriguez; lkml
 Subject: Re: Problems loading firmware using built-in drivers with kernels
 that use initramfs.
 
 On Wed, 26 Aug 2015 07:12:46 +0200,
 Jie, Yang wrote:
 
   -Original Message-
   From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
   Sent: Wednesday, August 26, 2015 3:58 AM
   To: Takashi Iwai
   Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
   joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro;
   Greg Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse;
   Luis Rodriguez; lkml
   Subject: Re: Problems loading firmware using built-in drivers with
   kernels that use initramfs.
  
   On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai ti...@suse.de wrote:
On Tue, 25 Aug 2015 21:34:08 +0200, Luis R. Rodriguez wrote:
   
On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
 Luis, did you tell me the other day that you made the kernel
 get firmware directly from the file system? This regression
 would be yours
   then?
   
I didn't implement that, Linus did in 2012 (see commit
abb139e75c2c titled
firmware: teach the kernel to load firmware files directly from
the filesystem). But we used to fallback to a userspace helper
when the fw was not present and then Takashi made this optional
via commit 7b1269f778782d titled firmware: Make user-mode helper
 optional.
Takashi noted in the Kconfig The user-mode helper is no longer
required unless you have a special firmware file that resides in
a non-standard path. It was not clarified why that's true
though, or what you'd need to do to ensure that the fw would be
available. It would be good for us to elaborate on that.
   
The recent udev already dropped the firmware loading feature.
  
   Note that even when we had udev helper to load the firmware it was
   not always reliable depending on the exact point where we requested
 firmware.
   If request happened in probe() path before we mounted root fs then
   we'd never get it loaded because we'd be waiting for devices settle
   before mounting rootfs.
 
  For request in probe(), is it possible to use
  request_firmware_nowait() to wait rootfs mounted or timeout in another
 thread?
 
  It looks usermodehelper_disabled is 0(at probe()) at this case then no
  waiting occurs here in our testing.
 
 It's possible -- and even with the normal request_firmware(), in theory.  The
 missing piece is that you need to inform the helper to retry the f/w loading.
 Currently the direct f/w loader assumes that the file is there and returns
 error if not present.
 
 If we implement a retry, another question is how to trigger it, i.e. how the
 helper can know when a fs is mounted a new file is present.
 
Got it, thanks Takashi and all.

So looks we should(and already done for most audio firmware) implement it 
as Dmitry and Linus proposed, that is to make sure request_firmware() calls
are not in driver's probe() paths.

Yalin help posted sample code for this implementation(to request firmware
in device node open) and actually I also did similar for Broadwell ADSP driver
and will send out later, thank you all.

Thanks,
~Keyon

 
 
 Takashi
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Linus Torvalds]

On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
liam.r.girdw...@linux.intel.com wrote:

 I think the options are to either :-

 1) Don not support audio DSP drivers using topology data as built-in
 drivers. Audio is not really a critical system required for booting
 anyway.

Yes, forcing it to be a module and not letting people compile it in by
mistake (and then not have it work) is an option.

That said, there are situations where people don't want to use
modules. I used to eschew them for security reasons, for example - now
I instead just do a one-time temporary key. But others may have other
reasons to try to avoid modules.

 2) Create a default PCM for every driver that has topology data on the
 assumption that every sound card will at least 1 PCM. This PCM can then
 be re-configured when the FW is loaded.

That would seem to be the better option if it is reasonably implementable.

Of course, some kind of timer-based retry (limited *somehow*) of the
fw loading could work too, but smells really really hacky.

  Linus
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Liam Girdwood]

On Wed, 2015-08-26 at 08:29 +, Jie, Yang wrote:
  -Original Message-
  From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]

  I think the options are to either :-
  
  1) Don not support audio DSP drivers using topology data as built-in 
  drivers.
  Audio is not really a critical system required for booting anyway.
  
  2) Create a default PCM for every driver that has topology data on the
  assumption that every sound card will at least 1 PCM. This PCM can then be
  re-configured when the FW is loaded.
 
 Yep, this case is quite similar with what Linus described.
  
 Is it possible that we can probe pcm device after firmware is loaded for this
 case?
 

The PCM devices are defined in the topology data so it is only possible
to create the PCM device *after* the firmware is loaded in these
drivers. 

Liam

--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Jie, Yang]

 -Original Message-
 From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
 Sent: Wednesday, August 26, 2015 4:07 PM
 To: Jie, Yang
 Cc: Takashi Iwai; Dmitry Torokhov; Luis R. Rodriguez;
 joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
 Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
 Rodriguez; lkml; yalin wang
 Subject: Re: Problems loading firmware using built-in drivers with kernels
 that use initramfs.
 
 On Wed, 2015-08-26 at 07:17 +0100, Jie, Yang wrote:
   -Original Message-
   From: Takashi Iwai [mailto:ti...@suse.de]
   Sent: Wednesday, August 26, 2015 1:33 PM
   To: Jie, Yang
   Cc: Dmitry Torokhov; Luis R. Rodriguez; Girdwood, Liam R;
   joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro;
   Greg Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse;
   Luis Rodriguez; lkml
   Subject: Re: Problems loading firmware using built-in drivers with
   kernels that use initramfs.
  
   It's possible -- and even with the normal request_firmware(), in
   theory.  The missing piece is that you need to inform the helper to retry
 the f/w loading.
   Currently the direct f/w loader assumes that the file is there and
   returns error if not present.
  
   If we implement a retry, another question is how to trigger it, i.e.
   how the helper can know when a fs is mounted a new file is present.
 
  Got it, thanks Takashi and all.
 
  So looks we should(and already done for most audio firmware) implement
  it as Dmitry and Linus proposed, that is to make sure
  request_firmware() calls are not in driver's probe() paths.
 
  Yalin help posted sample code for this implementation(to request
  firmware in device node open) and actually I also did similar for
  Broadwell ADSP driver and will send out later, thank you all.
 
 Keyon, we have a similar situation to that of the FPGA example that Linus
 described when the DSP driver uses topology data. i.e. the audio driver will
 have no PCM devices that can be opened (to initiate the FW
 loading) until after the FW is loaded. The topology data is part of the FW and
 contains the PCMs, kcontrols, graphs etc. i.e. a PCM device is created for
 every PCM description found in the FW topology data.
 
 I think the options are to either :-
 
 1) Don not support audio DSP drivers using topology data as built-in drivers.
 Audio is not really a critical system required for booting anyway.
 
 2) Create a default PCM for every driver that has topology data on the
 assumption that every sound card will at least 1 PCM. This PCM can then be
 re-configured when the FW is loaded.

Yep, this case is quite similar with what Linus described.
 
Is it possible that we can probe pcm device after firmware is loaded for this
case?

What I am thinking about is postponing all firmware related(e.g. pcm probing,
dma/dsp/ipc initialization, ...) code to that rootfs is ready and FW is loaded.

~Keyon

 
 Liam
 

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Liam Girdwood]

On Wed, 2015-08-26 at 07:17 +0100, Jie, Yang wrote:
  -Original Message-
  From: Takashi Iwai [mailto:ti...@suse.de]
  Sent: Wednesday, August 26, 2015 1:33 PM
  To: Jie, Yang
  Cc: Dmitry Torokhov; Luis R. Rodriguez; Girdwood, Liam R;
  joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
  Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
  Rodriguez; lkml
  Subject: Re: Problems loading firmware using built-in drivers with kernels
  that use initramfs.
  
  On Wed, 26 Aug 2015 07:12:46 +0200,
  Jie, Yang wrote:
  
-Original Message-
From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
Sent: Wednesday, August 26, 2015 3:58 AM
To: Takashi Iwai
Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro;
Greg Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse;
Luis Rodriguez; lkml
Subject: Re: Problems loading firmware using built-in drivers with
kernels that use initramfs.
   
On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai ti...@suse.de wrote:
 On Tue, 25 Aug 2015 21:34:08 +0200, Luis R. Rodriguez wrote:

 On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
  Luis, did you tell me the other day that you made the kernel
  get firmware directly from the file system? This regression
  would be yours
then?

 I didn't implement that, Linus did in 2012 (see commit
 abb139e75c2c titled
 firmware: teach the kernel to load firmware files directly from
 the filesystem). But we used to fallback to a userspace helper
 when the fw was not present and then Takashi made this optional
 via commit 7b1269f778782d titled firmware: Make user-mode helper
  optional.
 Takashi noted in the Kconfig The user-mode helper is no longer
 required unless you have a special firmware file that resides in
 a non-standard path. It was not clarified why that's true
 though, or what you'd need to do to ensure that the fw would be
 available. It would be good for us to elaborate on that.

 The recent udev already dropped the firmware loading feature.
   
Note that even when we had udev helper to load the firmware it was
not always reliable depending on the exact point where we requested
  firmware.
If request happened in probe() path before we mounted root fs then
we'd never get it loaded because we'd be waiting for devices settle
before mounting rootfs.
  
   For request in probe(), is it possible to use
   request_firmware_nowait() to wait rootfs mounted or timeout in another
  thread?
  
   It looks usermodehelper_disabled is 0(at probe()) at this case then no
   waiting occurs here in our testing.
  
  It's possible -- and even with the normal request_firmware(), in theory.  
  The
  missing piece is that you need to inform the helper to retry the f/w 
  loading.
  Currently the direct f/w loader assumes that the file is there and returns
  error if not present.
  
  If we implement a retry, another question is how to trigger it, i.e. how the
  helper can know when a fs is mounted a new file is present.
  
 Got it, thanks Takashi and all.
 
 So looks we should(and already done for most audio firmware) implement it 
 as Dmitry and Linus proposed, that is to make sure request_firmware() calls
 are not in driver's probe() paths.
 
 Yalin help posted sample code for this implementation(to request firmware
 in device node open) and actually I also did similar for Broadwell ADSP driver
 and will send out later, thank you all.

Keyon, we have a similar situation to that of the FPGA example that
Linus described when the DSP driver uses topology data. i.e. the audio
driver will have no PCM devices that can be opened (to initiate the FW
loading) until after the FW is loaded. The topology data is part of the
FW and contains the PCMs, kcontrols, graphs etc. i.e. a PCM device is
created for every PCM description found in the FW topology data. 

I think the options are to either :-

1) Don not support audio DSP drivers using topology data as built-in
drivers. Audio is not really a critical system required for booting
anyway.

2) Create a default PCM for every driver that has topology data on the
assumption that every sound card will at least 1 PCM. This PCM can then
be re-configured when the FW is loaded. 

Liam


--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Ming Lei]

On Thu, Aug 27, 2015 at 2:07 AM, Linus Torvalds
torva...@linux-foundation.org wrote:
 On Wed, Aug 26, 2015 at 1:06 AM, Liam Girdwood
 liam.r.girdw...@linux.intel.com wrote:

 I think the options are to either :-

 1) Don not support audio DSP drivers using topology data as built-in
 drivers. Audio is not really a critical system required for booting
 anyway.

 Yes, forcing it to be a module and not letting people compile it in by
 mistake (and then not have it work) is an option.

 That said, there are situations where people don't want to use
 modules. I used to eschew them for security reasons, for example - now
 I instead just do a one-time temporary key. But others may have other
 reasons to try to avoid modules.

 2) Create a default PCM for every driver that has topology data on the
 assumption that every sound card will at least 1 PCM. This PCM can then
 be re-configured when the FW is loaded.

 That would seem to be the better option if it is reasonably implementable.

 Of course, some kind of timer-based retry (limited *somehow*) of the
 fw loading could work too, but smells really really hacky.

Yeah, years ago, we discussed to use -EPROBE_DEFER for the situation,
which should be one kind of fix, but looks there were objections at that time.

thanks,
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Lin, Mengdong]

 -Original Message-
 From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
 Sent: Wednesday, August 26, 2015 5:01 PM
 To: Jie, Yang
 Cc: Takashi Iwai; Dmitry Torokhov; Luis R. Rodriguez;
 joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
 Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis Rodriguez;
 lkml; yalin wang; Lin, Mengdong
 Subject: Re: Problems loading firmware using built-in drivers with kernels 
 that
 use initramfs.
 
 On Wed, 2015-08-26 at 08:29 +, Jie, Yang wrote:
   -Original Message-
   From: Liam Girdwood [mailto:liam.r.girdw...@linux.intel.com]
 
   I think the options are to either :-
  
   1) Don not support audio DSP drivers using topology data as built-in
 drivers.
   Audio is not really a critical system required for booting anyway.
  
   2) Create a default PCM for every driver that has topology data on
   the assumption that every sound card will at least 1 PCM. This PCM
   can then be re-configured when the FW is loaded.
 
  Yep, this case is quite similar with what Linus described.
 
  Is it possible that we can probe pcm device after firmware is loaded
  for this case?
 
 
 The PCM devices are defined in the topology data so it is only possible to
 create the PCM device *after* the firmware is loaded in these drivers.
 
 Liam

It seems this can prevent audio device driver in built-in mode to use topology 
drivers.

If topology is present, the vendor driver also uses request_firmware() to load 
the topology data file in the probe phase. Then ASoC core will create the sound 
card and its PCM devices. 

Shall we force the device drivers that depends on topology to be configured as 
modules?

Thanks
Mengdong 

N�r��yb�X��ǧv�^�)޺{.n�+{zX����ܨ}���Ơz�j:+v���zZ+��+zf���h���~i���z��w���?��)ߢf��^jǫy�m��@A�a���
0��h���i

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Takashi Iwai]

On Wed, 26 Aug 2015 07:12:46 +0200,
Jie, Yang wrote:
> 
> > -Original Message-
> > From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
> > Sent: Wednesday, August 26, 2015 3:58 AM
> > To: Takashi Iwai
> > Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
> > joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> > Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
> > Rodriguez; lkml
> > Subject: Re: Problems loading firmware using built-in drivers with kernels
> > that use initramfs.
> > 
> > On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai  wrote:
> > > On Tue, 25 Aug 2015 21:34:08 +0200,
> > > Luis R. Rodriguez wrote:
> > >>
> > >> On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
> > >> > Luis, did you tell me the other day that you made the kernel get
> > >> > firmware directly from the file system? This regression would be yours
> > then?
> > >>
> > >> I didn't implement that, Linus did in 2012 (see commit abb139e75c2c
> > >> titled
> > >> "firmware: teach the kernel to load firmware files directly from the
> > >> filesystem"). But we used to fallback to a userspace helper when the
> > >> fw was not present and then Takashi made this optional via commit
> > >> 7b1269f778782d titled "firmware: Make user-mode helper optional".
> > >> Takashi noted in the Kconfig "The user-mode helper is no longer
> > >> required unless you have a special firmware file that resides in a
> > >> non-standard path". It was not clarified why that's true though, or
> > >> what you'd need to do to ensure that the fw would be available. It
> > >> would be good for us to elaborate on that.
> > >
> > > The recent udev already dropped the firmware loading feature.
> > 
> > Note that even when we had udev helper to load the firmware it was not
> > always reliable depending on the exact point where we requested firmware.
> > If request happened in probe() path before we mounted root fs then we'd
> > never get it loaded because we'd be waiting for devices settle before
> > mounting rootfs.
> 
> For request in probe(), is it possible to use request_firmware_nowait() to
> wait rootfs mounted or timeout in another thread?
> 
> It looks usermodehelper_disabled is 0(at probe()) at this case then no 
> waiting occurs
> here in our testing.

It's possible -- and even with the normal request_firmware(), in
theory.  The missing piece is that you need to inform the helper to
retry the f/w loading.  Currently the direct f/w loader assumes that
the file is there and returns error if not present.

If we implement a retry, another question is how to trigger it,
i.e. how the helper can know when a fs is mounted a new file is
present.


Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[yalin wang]

> On Aug 26, 2015, at 04:26, Linus Torvalds  
> wrote:
> 
> On Tue, Aug 25, 2015 at 12:58 PM, Dmitry Torokhov
>  wrote:
>> 
>> Either build firmware in the kernel or ramdisk (so it is always
>> available), or make sure request_firmware() calls are not in driver's
>> probe() paths.
> 
> The correct answer is almost always that second one.
> 
> Drivers that load firmware in their probe parts are generally doign
> things wrong.
> 
> It's very occasionally the right thing to do - there are a few pieces
> of hardware where just about _everything_ about the device is in the
> firmware, and you simply can't even problem them before that, because
> you don't know what they *are* before the firmware has been loaded.
> The extreme case of that might be something like the base hardware
> being an FPGA that has a USB interface, but before the FPGA has been
> loaded, it basically has no identity. So there are probably valid
> cases in theory for loading firmware at probe time, but pretty much
> every single case I've ever actually seen, the probe knows what the
> actual hardware is from some identifiable piece, and the actual
> firmware loading should be delayed until the piece of hardware is
> actually opened.
> 
> So the "load firmware in probe routine" happens, and we shouldn't
> disallow it, but it should be considered a "last resort" kind of
> thing.
> 
> In general, things like sound drivers should *not* need to have their
> firmware in the initrd, even if you build them into the kernel. A disk
> driver? Yes. Maybe the root filesystem is on that disk, and you need
> the firmware for the disk driver to load it. But a sound device?
> Please just make sure that you load the firmware as late as possible.
> 
i remember lots of drivers load their firmware when some user space process
open the device node, that is to say, they load the firmware in 
->open() function, at this time , you can make sure the real filesystem is 
ready in
most time.  

i think your dsp firmware can also do like this.
you can refer to msm gnu driver:

static void load_gpu(struct drm_device *dev)
{
static DEFINE_MUTEX(init_lock);
struct msm_drm_private *priv = dev->dev_private;

mutex_lock(_lock);

if (!priv->gpu)
priv->gpu = adreno_load_gpu(dev);

mutex_unlock(_lock);
}

static int msm_open(struct drm_device *dev, struct drm_file *file)
{
struct msm_file_private *ctx;

/* For now, load gpu on open.. to avoid the requirement of having
 * firmware in the initrd.
 */
load_gpu(dev);

ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
if (!ctx)
return -ENOMEM;

file->driver_priv = ctx;

return 0;
}
it load the firmware in open() function.

Thanks











--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Jie, Yang]

> -Original Message-
> From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
> Sent: Wednesday, August 26, 2015 3:58 AM
> To: Takashi Iwai
> Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
> joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
> Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
> Rodriguez; lkml
> Subject: Re: Problems loading firmware using built-in drivers with kernels
> that use initramfs.
> 
> On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai  wrote:
> > On Tue, 25 Aug 2015 21:34:08 +0200,
> > Luis R. Rodriguez wrote:
> >>
> >> On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
> >> > Luis, did you tell me the other day that you made the kernel get
> >> > firmware directly from the file system? This regression would be yours
> then?
> >>
> >> I didn't implement that, Linus did in 2012 (see commit abb139e75c2c
> >> titled
> >> "firmware: teach the kernel to load firmware files directly from the
> >> filesystem"). But we used to fallback to a userspace helper when the
> >> fw was not present and then Takashi made this optional via commit
> >> 7b1269f778782d titled "firmware: Make user-mode helper optional".
> >> Takashi noted in the Kconfig "The user-mode helper is no longer
> >> required unless you have a special firmware file that resides in a
> >> non-standard path". It was not clarified why that's true though, or
> >> what you'd need to do to ensure that the fw would be available. It
> >> would be good for us to elaborate on that.
> >
> > The recent udev already dropped the firmware loading feature.
> 
> Note that even when we had udev helper to load the firmware it was not
> always reliable depending on the exact point where we requested firmware.
> If request happened in probe() path before we mounted root fs then we'd
> never get it loaded because we'd be waiting for devices settle before
> mounting rootfs.

For request in probe(), is it possible to use request_firmware_nowait() to
wait rootfs mounted or timeout in another thread?

It looks usermodehelper_disabled is 0(at probe()) at this case then no waiting 
occurs
here in our testing.

Thanks,
~Keyon

> 
> Either build firmware in the kernel or ramdisk (so it is always available), or
> make sure request_firmware() calls are not in driver's
> probe() paths.
> 
> Thanks.
> 
> --
> Dmitry

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Linus Torvalds]

On Tue, Aug 25, 2015 at 12:58 PM, Dmitry Torokhov
 wrote:
>
> Either build firmware in the kernel or ramdisk (so it is always
> available), or make sure request_firmware() calls are not in driver's
> probe() paths.

The correct answer is almost always that second one.

Drivers that load firmware in their probe parts are generally doign
things wrong.

It's very occasionally the right thing to do - there are a few pieces
of hardware where just about _everything_ about the device is in the
firmware, and you simply can't even problem them before that, because
you don't know what they *are* before the firmware has been loaded.
The extreme case of that might be something like the base hardware
being an FPGA that has a USB interface, but before the FPGA has been
loaded, it basically has no identity. So there are probably valid
cases in theory for loading firmware at probe time, but pretty much
every single case I've ever actually seen, the probe knows what the
actual hardware is from some identifiable piece, and the actual
firmware loading should be delayed until the piece of hardware is
actually opened.

So the "load firmware in probe routine" happens, and we shouldn't
disallow it, but it should be considered a "last resort" kind of
thing.

In general, things like sound drivers should *not* need to have their
firmware in the initrd, even if you build them into the kernel. A disk
driver? Yes. Maybe the root filesystem is on that disk, and you need
the firmware for the disk driver to load it. But a sound device?
Please just make sure that you load the firmware as late as possible.

Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai  wrote:
> On Tue, 25 Aug 2015 21:34:08 +0200,
> Luis R. Rodriguez wrote:
>>
>> On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
>> > Luis, did you tell me the other day that you made the kernel get firmware
>> > directly from the file system? This regression would be yours then?
>>
>> I didn't implement that, Linus did in 2012 (see commit abb139e75c2c titled
>> "firmware: teach the kernel to load firmware files directly from the
>> filesystem"). But we used to fallback to a userspace helper when the fw was
>> not present and then Takashi made this optional via commit 7b1269f778782d
>> titled "firmware: Make user-mode helper optional". Takashi noted in the
>> Kconfig "The user-mode helper is no longer required unless you have a
>> special firmware file that resides in a non-standard path". It was not
>> clarified why that's true though, or what you'd need to do to ensure
>> that the fw would be available. It would be good for us to elaborate
>> on that.
>
> The recent udev already dropped the firmware loading feature.

Note that even when we had udev helper to load the firmware it was not
always reliable depending on the exact point where we requested
firmware. If request happened in probe() path before we mounted root
fs then we'd never get it loaded because we'd be waiting for devices
settle before mounting rootfs.

Either build firmware in the kernel or ramdisk (so it is always
available), or make sure request_firmware() calls are not in driver's
probe() paths.

Thanks.

-- 
Dmitry
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Takashi Iwai]

On Tue, 25 Aug 2015 21:34:08 +0200,
Luis R. Rodriguez wrote:
> 
> On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
> > Luis, did you tell me the other day that you made the kernel get firmware
> > directly from the file system? This regression would be yours then?
> 
> I didn't implement that, Linus did in 2012 (see commit abb139e75c2c titled
> "firmware: teach the kernel to load firmware files directly from the
> filesystem"). But we used to fallback to a userspace helper when the fw was
> not present and then Takashi made this optional via commit 7b1269f778782d
> titled "firmware: Make user-mode helper optional". Takashi noted in the
> Kconfig "The user-mode helper is no longer required unless you have a
> special firmware file that resides in a non-standard path". It was not
> clarified why that's true though, or what you'd need to do to ensure
> that the fw would be available. It would be good for us to elaborate
> on that.

The recent udev already dropped the firmware loading feature.


Takashi
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
> Luis, did you tell me the other day that you made the kernel get firmware
> directly from the file system? This regression would be yours then?

I didn't implement that, Linus did in 2012 (see commit abb139e75c2c titled
"firmware: teach the kernel to load firmware files directly from the
filesystem"). But we used to fallback to a userspace helper when the fw was
not present and then Takashi made this optional via commit 7b1269f778782d
titled "firmware: Make user-mode helper optional". Takashi noted in the
Kconfig "The user-mode helper is no longer required unless you have a
special firmware file that resides in a non-standard path". It was not
clarified why that's true though, or what you'd need to do to ensure
that the fw would be available. It would be good for us to elaborate
on that.

More on this below.

> I can also take a look next week when I'm home from vacation.

No worries, I'll look at it.

> From:Liam Girdwood 
> Sent:Tue, 25 Aug 2015 09:05:00 +0100
> To:"Jie, Yang" ,dw...@infradead.org,"joonas.lahtinen  
> " 
> Subject:Re: Problems loading firmware using built-in drivers with kernels 
> that use initramfs.
> 
> >On Mon, 2015-08-24 at 21:50 +0100, Liam Girdwood wrote:
> >> Currently firmware loading with built-in drivers is failing on systems
> >> that mount an initramfs before the root FS is mounted (that contains the
> >> firmware file). Modular drivers work fine.
> >> 
> >> Keyon (Jie, Yang) has done some testing with the Intel audio DSP driver
> >> and found that the built-in firmware loading only works if the firmware
> >> is included as part of the initramfs.

Note that works.

> >> The same has been seen with other
> >> Intel drivers that use firmware.
> >> 
> >> In this case it looks like userspace (udevd ?) is checking initramfs for
> >> pending firmware requests and returning not found if it cant find the
> >> files, but it's not re-checking for any pending firmware requests once
> >> the real FS is mounted. Is it possible to have udevd re-check the
> >> pending firmware requests after the real FS is mounted ?

We are phasing out CONFIG_FW_LOADER_USER_HELPER support in the kernel, support
for udev helper firmware loading was removed from systemd, the only requirement
for it now is the dell rbu driver.

Without CONFIG_FW_LOADER_USER_HELPER it its up to userspace or system
integrators to do the right thing and you have a few options if you are
enabling drivers as built-in:

  * stuff firmware in initramfs
  * built firmware into the kernel

We don't do anything special for pivot_root for built-in, I am not sure if
we should, I don't think we should... can you not do any of the above
two options?

Since folks can stuff firmware into initramfs and that's outside the scope of
how we build the kernel we can't always ensure folks will do one of the above
two options at build time for built-in drivers. If we don't want to do
something for pivot_root the only thing I think we can do is document the
expecations clearly, unless there are other ideas of course.

  Luis
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[yalin wang]

 On Aug 26, 2015, at 04:26, Linus Torvalds torva...@linux-foundation.org 
 wrote:
 
 On Tue, Aug 25, 2015 at 12:58 PM, Dmitry Torokhov
 dmitry.torok...@gmail.com wrote:
 
 Either build firmware in the kernel or ramdisk (so it is always
 available), or make sure request_firmware() calls are not in driver's
 probe() paths.
 
 The correct answer is almost always that second one.
 
 Drivers that load firmware in their probe parts are generally doign
 things wrong.
 
 It's very occasionally the right thing to do - there are a few pieces
 of hardware where just about _everything_ about the device is in the
 firmware, and you simply can't even problem them before that, because
 you don't know what they *are* before the firmware has been loaded.
 The extreme case of that might be something like the base hardware
 being an FPGA that has a USB interface, but before the FPGA has been
 loaded, it basically has no identity. So there are probably valid
 cases in theory for loading firmware at probe time, but pretty much
 every single case I've ever actually seen, the probe knows what the
 actual hardware is from some identifiable piece, and the actual
 firmware loading should be delayed until the piece of hardware is
 actually opened.
 
 So the load firmware in probe routine happens, and we shouldn't
 disallow it, but it should be considered a last resort kind of
 thing.
 
 In general, things like sound drivers should *not* need to have their
 firmware in the initrd, even if you build them into the kernel. A disk
 driver? Yes. Maybe the root filesystem is on that disk, and you need
 the firmware for the disk driver to load it. But a sound device?
 Please just make sure that you load the firmware as late as possible.
 
i remember lots of drivers load their firmware when some user space process
open the device node, that is to say, they load the firmware in 
-open() function, at this time , you can make sure the real filesystem is 
ready in
most time.  

i think your dsp firmware can also do like this.
you can refer to msm gnu driver:

static void load_gpu(struct drm_device *dev)
{
static DEFINE_MUTEX(init_lock);
struct msm_drm_private *priv = dev-dev_private;

mutex_lock(init_lock);

if (!priv-gpu)
priv-gpu = adreno_load_gpu(dev);

mutex_unlock(init_lock);
}

static int msm_open(struct drm_device *dev, struct drm_file *file)
{
struct msm_file_private *ctx;

/* For now, load gpu on open.. to avoid the requirement of having
 * firmware in the initrd.
 */
load_gpu(dev);

ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
if (!ctx)
return -ENOMEM;

file-driver_priv = ctx;

return 0;
}
it load the firmware in open() function.

Thanks











--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Takashi Iwai]

On Wed, 26 Aug 2015 07:12:46 +0200,
Jie, Yang wrote:
 
  -Original Message-
  From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
  Sent: Wednesday, August 26, 2015 3:58 AM
  To: Takashi Iwai
  Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
  joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
  Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
  Rodriguez; lkml
  Subject: Re: Problems loading firmware using built-in drivers with kernels
  that use initramfs.
  
  On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai ti...@suse.de wrote:
   On Tue, 25 Aug 2015 21:34:08 +0200,
   Luis R. Rodriguez wrote:
  
   On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
Luis, did you tell me the other day that you made the kernel get
firmware directly from the file system? This regression would be yours
  then?
  
   I didn't implement that, Linus did in 2012 (see commit abb139e75c2c
   titled
   firmware: teach the kernel to load firmware files directly from the
   filesystem). But we used to fallback to a userspace helper when the
   fw was not present and then Takashi made this optional via commit
   7b1269f778782d titled firmware: Make user-mode helper optional.
   Takashi noted in the Kconfig The user-mode helper is no longer
   required unless you have a special firmware file that resides in a
   non-standard path. It was not clarified why that's true though, or
   what you'd need to do to ensure that the fw would be available. It
   would be good for us to elaborate on that.
  
   The recent udev already dropped the firmware loading feature.
  
  Note that even when we had udev helper to load the firmware it was not
  always reliable depending on the exact point where we requested firmware.
  If request happened in probe() path before we mounted root fs then we'd
  never get it loaded because we'd be waiting for devices settle before
  mounting rootfs.
 
 For request in probe(), is it possible to use request_firmware_nowait() to
 wait rootfs mounted or timeout in another thread?
 
 It looks usermodehelper_disabled is 0(at probe()) at this case then no 
 waiting occurs
 here in our testing.

It's possible -- and even with the normal request_firmware(), in
theory.  The missing piece is that you need to inform the helper to
retry the f/w loading.  Currently the direct f/w loader assumes that
the file is there and returns error if not present.

If we implement a retry, another question is how to trigger it,
i.e. how the helper can know when a fs is mounted a new file is
present.


Takashi
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

RE: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Jie, Yang]

 -Original Message-
 From: Dmitry Torokhov [mailto:dmitry.torok...@gmail.com]
 Sent: Wednesday, August 26, 2015 3:58 AM
 To: Takashi Iwai
 Cc: Luis R. Rodriguez; Girdwood, Liam R; Jie, Yang;
 joonas.lahti...@linux.intel.com; Tom Gundersen; Ming Lei; Al Viro; Greg
 Kroah-Hartman; Kay Sievers; Linus Torvalds; David Woodhouse; Luis
 Rodriguez; lkml
 Subject: Re: Problems loading firmware using built-in drivers with kernels
 that use initramfs.
 
 On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai ti...@suse.de wrote:
  On Tue, 25 Aug 2015 21:34:08 +0200,
  Luis R. Rodriguez wrote:
 
  On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
   Luis, did you tell me the other day that you made the kernel get
   firmware directly from the file system? This regression would be yours
 then?
 
  I didn't implement that, Linus did in 2012 (see commit abb139e75c2c
  titled
  firmware: teach the kernel to load firmware files directly from the
  filesystem). But we used to fallback to a userspace helper when the
  fw was not present and then Takashi made this optional via commit
  7b1269f778782d titled firmware: Make user-mode helper optional.
  Takashi noted in the Kconfig The user-mode helper is no longer
  required unless you have a special firmware file that resides in a
  non-standard path. It was not clarified why that's true though, or
  what you'd need to do to ensure that the fw would be available. It
  would be good for us to elaborate on that.
 
  The recent udev already dropped the firmware loading feature.
 
 Note that even when we had udev helper to load the firmware it was not
 always reliable depending on the exact point where we requested firmware.
 If request happened in probe() path before we mounted root fs then we'd
 never get it loaded because we'd be waiting for devices settle before
 mounting rootfs.

For request in probe(), is it possible to use request_firmware_nowait() to
wait rootfs mounted or timeout in another thread?

It looks usermodehelper_disabled is 0(at probe()) at this case then no waiting 
occurs
here in our testing.

Thanks,
~Keyon

 
 Either build firmware in the kernel or ramdisk (so it is always available), or
 make sure request_firmware() calls are not in driver's
 probe() paths.
 
 Thanks.
 
 --
 Dmitry

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Luis R. Rodriguez]

On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
 Luis, did you tell me the other day that you made the kernel get firmware
 directly from the file system? This regression would be yours then?

I didn't implement that, Linus did in 2012 (see commit abb139e75c2c titled
firmware: teach the kernel to load firmware files directly from the
filesystem). But we used to fallback to a userspace helper when the fw was
not present and then Takashi made this optional via commit 7b1269f778782d
titled firmware: Make user-mode helper optional. Takashi noted in the
Kconfig The user-mode helper is no longer required unless you have a
special firmware file that resides in a non-standard path. It was not
clarified why that's true though, or what you'd need to do to ensure
that the fw would be available. It would be good for us to elaborate
on that.

More on this below.

 I can also take a look next week when I'm home from vacation.

No worries, I'll look at it.

 From:Liam Girdwood liam.r.girdw...@intel.com
 Sent:Tue, 25 Aug 2015 09:05:00 +0100
 To:Jie, Yang yang@intel.com,dw...@infradead.org,joonas.lahtinen  
  joonas.lahti...@linux.intel.com
 Subject:Re: Problems loading firmware using built-in drivers with kernels 
 that use initramfs.
 
 On Mon, 2015-08-24 at 21:50 +0100, Liam Girdwood wrote:
  Currently firmware loading with built-in drivers is failing on systems
  that mount an initramfs before the root FS is mounted (that contains the
  firmware file). Modular drivers work fine.
  
  Keyon (Jie, Yang) has done some testing with the Intel audio DSP driver
  and found that the built-in firmware loading only works if the firmware
  is included as part of the initramfs.

Note that works.

  The same has been seen with other
  Intel drivers that use firmware.
  
  In this case it looks like userspace (udevd ?) is checking initramfs for
  pending firmware requests and returning not found if it cant find the
  files, but it's not re-checking for any pending firmware requests once
  the real FS is mounted. Is it possible to have udevd re-check the
  pending firmware requests after the real FS is mounted ?

We are phasing out CONFIG_FW_LOADER_USER_HELPER support in the kernel, support
for udev helper firmware loading was removed from systemd, the only requirement
for it now is the dell rbu driver.

Without CONFIG_FW_LOADER_USER_HELPER it its up to userspace or system
integrators to do the right thing and you have a few options if you are
enabling drivers as built-in:

  * stuff firmware in initramfs
  * built firmware into the kernel

We don't do anything special for pivot_root for built-in, I am not sure if
we should, I don't think we should... can you not do any of the above
two options?

Since folks can stuff firmware into initramfs and that's outside the scope of
how we build the kernel we can't always ensure folks will do one of the above
two options at build time for built-in drivers. If we don't want to do
something for pivot_root the only thing I think we can do is document the
expecations clearly, unless there are other ideas of course.

  Luis
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Dmitry Torokhov]

On Tue, Aug 25, 2015 at 12:46 PM, Takashi Iwai ti...@suse.de wrote:
 On Tue, 25 Aug 2015 21:34:08 +0200,
 Luis R. Rodriguez wrote:

 On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
  Luis, did you tell me the other day that you made the kernel get firmware
  directly from the file system? This regression would be yours then?

 I didn't implement that, Linus did in 2012 (see commit abb139e75c2c titled
 firmware: teach the kernel to load firmware files directly from the
 filesystem). But we used to fallback to a userspace helper when the fw was
 not present and then Takashi made this optional via commit 7b1269f778782d
 titled firmware: Make user-mode helper optional. Takashi noted in the
 Kconfig The user-mode helper is no longer required unless you have a
 special firmware file that resides in a non-standard path. It was not
 clarified why that's true though, or what you'd need to do to ensure
 that the fw would be available. It would be good for us to elaborate
 on that.

 The recent udev already dropped the firmware loading feature.

Note that even when we had udev helper to load the firmware it was not
always reliable depending on the exact point where we requested
firmware. If request happened in probe() path before we mounted root
fs then we'd never get it loaded because we'd be waiting for devices
settle before mounting rootfs.

Either build firmware in the kernel or ramdisk (so it is always
available), or make sure request_firmware() calls are not in driver's
probe() paths.

Thanks.

-- 
Dmitry
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Takashi Iwai]

On Tue, 25 Aug 2015 21:34:08 +0200,
Luis R. Rodriguez wrote:
 
 On Tue, Aug 25, 2015 at 10:17:00AM +0100, David Woodhouse wrote:
  Luis, did you tell me the other day that you made the kernel get firmware
  directly from the file system? This regression would be yours then?
 
 I didn't implement that, Linus did in 2012 (see commit abb139e75c2c titled
 firmware: teach the kernel to load firmware files directly from the
 filesystem). But we used to fallback to a userspace helper when the fw was
 not present and then Takashi made this optional via commit 7b1269f778782d
 titled firmware: Make user-mode helper optional. Takashi noted in the
 Kconfig The user-mode helper is no longer required unless you have a
 special firmware file that resides in a non-standard path. It was not
 clarified why that's true though, or what you'd need to do to ensure
 that the fw would be available. It would be good for us to elaborate
 on that.

The recent udev already dropped the firmware loading feature.


Takashi
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Problems loading firmware using built-in drivers with kernels that use initramfs.

[Linus Torvalds]

On Tue, Aug 25, 2015 at 12:58 PM, Dmitry Torokhov
dmitry.torok...@gmail.com wrote:

 Either build firmware in the kernel or ramdisk (so it is always
 available), or make sure request_firmware() calls are not in driver's
 probe() paths.

The correct answer is almost always that second one.

Drivers that load firmware in their probe parts are generally doign
things wrong.

It's very occasionally the right thing to do - there are a few pieces
of hardware where just about _everything_ about the device is in the
firmware, and you simply can't even problem them before that, because
you don't know what they *are* before the firmware has been loaded.
The extreme case of that might be something like the base hardware
being an FPGA that has a USB interface, but before the FPGA has been
loaded, it basically has no identity. So there are probably valid
cases in theory for loading firmware at probe time, but pretty much
every single case I've ever actually seen, the probe knows what the
actual hardware is from some identifiable piece, and the actual
firmware loading should be delayed until the piece of hardware is
actually opened.

So the load firmware in probe routine happens, and we shouldn't
disallow it, but it should be considered a last resort kind of
thing.

In general, things like sound drivers should *not* need to have their
firmware in the initrd, even if you build them into the kernel. A disk
driver? Yes. Maybe the root filesystem is on that disk, and you need
the firmware for the disk driver to load it. But a sound device?
Please just make sure that you load the firmware as late as possible.

Linus
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/