Re: [PATCH v3 5/5] firmware: add an extensible system data helpers

2016-01-04 Thread Kees Cook 
On Wed, Dec 23, 2015 at 1:34 PM, Luis R. Rodriguez
 wrote:
> From: "Luis R. Rodriguez" 
>
> The firmware API has evolved over the years slowly, as it
> grows we extend it by adding new routines or at times we extend
> existing routines with more or less arguments. This doesn't scale
> well, when new arguments are added to existing routines it means
> we need to traverse the kernel with a slew of collateral
> evolutions to adjust old driver users. The firmware API is also
> now being used for things outside of the scope of what typically
> would be considered "firmware", an example here is the p54 driver
> enables users to provide a custom EEPROM through this interface.
> Another example is optional CPU microcode updates.
>
> There are other subsystems which would like to make use of the
> APIs for similar things but have different requirements and
> criteria which they'd like to be met for the requested file.
> If different requirements are needed it would again mean adding
> more arguments and making a slew of collateral evolutions, or
> adding yet-another-new-API-call.
>
> Instead of extending the existing firmware API even more this
> provides an new extensible API which can be used to supercede the
> old firmware API without causing a series of collateral evolutions
> as requirements grow. This leaves the old firmware API as-is,
> ignores all consideration for usermode-helpers, labels the new API
> to reflect its broad use outside of the scope of firmware: system
> data helpers, and builds on top of the original firmware core code.
>
> The new extensible "system data" set of helpers accepts that there
> really are only two types of requests for accessing system data:
>
> a) synchronous requests
> b) asynchronous requests
>
> Both of these requests may have a different set of requirements
> which must be met. These requirements can simply be passed as a
> descriptors to each type of request. The descriptor can be extended
> over time to support different requirements as the kernel evolves.
>
> Using the new system data helpers is only necessary if you have
> requirements outside of what the existing old firmware API accepts.
> We encourage developers to leave the old API as-is and extend the
> new descriptors and system data code to provide support for new
> features.
>
> A few simple features added as part of the new set of system data
> request APIs, other than making the new API easily extensible for
> the future:
>
>  - By default the kernel will free the system data file for you after
>your callbacks are called, you however are allowed to request to that
>you wish to keep the system data file on the descriptor. This is
>dealt with by requiring a consumer callback for the system data file.
>  - Allow both asynchronous and synchronous request to specify that system data
>files are optional. With the old APIs we had added one full API call,
>request_firmware_direct() just for this purpose -- although it should be
>noted another of its goal was to also skip the usermode helper.
>The system data request APIs allow for you to annotate that a system
>data file is optional for both synchronous or asynchronous requests
>through the same two basic set of APIs.
>  - Usermode helpers are completely ignored, always
>  - The system data request APIs currently match the old synchronous firmware
>API calls to refcounted firmware_class module, but it should be easy
>to add support now to enable also refcounting the caller's module
>should it be be needed. Likewise the system data request APIs match the
>old asynchronous firmware API call and refcounts the caller's module.
>
> In order to try to help phase out user mode helpers this makes no use of
> the old user mode helper code *at all*, and if we wish to can easily
> phase this code out with time then.

So these are basically wrappers around the existing firmware loading routines?

-Kees

>
> Signed-off-by: Luis R. Rodriguez 
> ---
>  Documentation/firmware_class/system_data.txt |  79 
>  MAINTAINERS  |   4 +-
>  drivers/base/firmware_class.c| 291 
> +++
>  include/linux/sysdata.h  | 212 +++
>  4 files changed, 585 insertions(+), 1 deletion(-)
>  create mode 100644 Documentation/firmware_class/system_data.txt
>  create mode 100644 include/linux/sysdata.h
>
> diff --git a/Documentation/firmware_class/system_data.txt 
> b/Documentation/firmware_class/system_data.txt
> new file mode 100644
> index ..ab509243455a
> --- /dev/null
> +++ b/Documentation/firmware_class/system_data.txt
> @@ -0,0 +1,79 @@
> +System data requests API
> +
> +
> +As the kernel evolves we keep extending the firmware_class set of APIs
> +with more or less arguments, this creates a slew of collateral evolutions.
> +The set of users of

Re: [PATCH v3 5/5] firmware: add an extensible system data helpers

2015-12-24 Thread kbuild test robot 
Hi Luis,

[auto build test WARNING on v4.4-rc6]
[also build test WARNING on next-20151223]

url:
https://github.com/0day-ci/linux/commits/Luis-R-Rodriguez/firmware_class-extensible-firmware-API/20151224-053852
reproduce: make htmldocs

All warnings (new ones prefixed by >>):

   include/linux/init.h:1: warning: no structured comments found
   kernel/sys.c:1: warning: no structured comments found
>> drivers/base/firmware_class.c:1336: warning: No description found for 
>> parameter 'sysdata'
>> drivers/base/firmware_class.c:1336: warning: Excess function parameter 
>> 'sysdata_file' description in 'release_sysdata_file'
   drivers/dma-buf/seqno-fence.c:1: warning: no structured comments found
   drivers/dma-buf/reservation.c:1: warning: no structured comments found
   include/linux/reservation.h:1: warning: no structured comments found
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef 
member 'e_handler' description in 'hsi_client'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef 
member 'pclaimed' description in 'hsi_client'
   include/linux/hsi/hsi.h:150: warning: Excess struct/union/enum/typedef 
member 'nb' description in 'hsi_client'

vim +/sysdata +1336 drivers/base/firmware_class.c

  1320  __func__);
  1321  return -ENOMEM;
  1322  }
  1323  
  1324  ret = _request_firmware_prepare(, name, device);
  1325  if (ret >= 0)
  1326  sysdata->priv = fw;
  1327  
  1328  return ret;
  1329  }
  1330  
  1331  /**
  1332   * release_sysdata_file: - release the resource associated with the 
sysdata file
  1333   * @sysdata_file: sysdata resource to release
  1334   **/
  1335  void release_sysdata_file(const struct sysdata_file *sysdata)
> 1336  {
  1337  struct firmware *fw;
  1338  
  1339  if (sysdata) {
  1340  if (sysdata->priv) {
  1341  fw = sysdata->priv;
  1342  release_firmware(fw);
  1343  }
  1344  }

---
0-DAY kernel test infrastructureOpen Source Technology Center
https://lists.01.org/pipermail/kbuild-all   Intel Corporation


.config.gz
Description: Binary data

[PATCH v3 5/5] firmware: add an extensible system data helpers

2015-12-23 Thread Luis R. Rodriguez 
From: "Luis R. Rodriguez" 

The firmware API has evolved over the years slowly, as it
grows we extend it by adding new routines or at times we extend
existing routines with more or less arguments. This doesn't scale
well, when new arguments are added to existing routines it means
we need to traverse the kernel with a slew of collateral
evolutions to adjust old driver users. The firmware API is also
now being used for things outside of the scope of what typically
would be considered "firmware", an example here is the p54 driver
enables users to provide a custom EEPROM through this interface.
Another example is optional CPU microcode updates.

There are other subsystems which would like to make use of the
APIs for similar things but have different requirements and
criteria which they'd like to be met for the requested file.
If different requirements are needed it would again mean adding
more arguments and making a slew of collateral evolutions, or
adding yet-another-new-API-call.

Instead of extending the existing firmware API even more this
provides an new extensible API which can be used to supercede the
old firmware API without causing a series of collateral evolutions
as requirements grow. This leaves the old firmware API as-is,
ignores all consideration for usermode-helpers, labels the new API
to reflect its broad use outside of the scope of firmware: system
data helpers, and builds on top of the original firmware core code.

The new extensible "system data" set of helpers accepts that there
really are only two types of requests for accessing system data:

a) synchronous requests
b) asynchronous requests

Both of these requests may have a different set of requirements
which must be met. These requirements can simply be passed as a
descriptors to each type of request. The descriptor can be extended
over time to support different requirements as the kernel evolves.

Using the new system data helpers is only necessary if you have
requirements outside of what the existing old firmware API accepts.
We encourage developers to leave the old API as-is and extend the
new descriptors and system data code to provide support for new
features.

A few simple features added as part of the new set of system data
request APIs, other than making the new API easily extensible for
the future:

 - By default the kernel will free the system data file for you after
   your callbacks are called, you however are allowed to request to that
   you wish to keep the system data file on the descriptor. This is
   dealt with by requiring a consumer callback for the system data file.
 - Allow both asynchronous and synchronous request to specify that system data
   files are optional. With the old APIs we had added one full API call,
   request_firmware_direct() just for this purpose -- although it should be
   noted another of its goal was to also skip the usermode helper.
   The system data request APIs allow for you to annotate that a system
   data file is optional for both synchronous or asynchronous requests
   through the same two basic set of APIs.
 - Usermode helpers are completely ignored, always
 - The system data request APIs currently match the old synchronous firmware
   API calls to refcounted firmware_class module, but it should be easy
   to add support now to enable also refcounting the caller's module
   should it be be needed. Likewise the system data request APIs match the
   old asynchronous firmware API call and refcounts the caller's module.

In order to try to help phase out user mode helpers this makes no use of
the old user mode helper code *at all*, and if we wish to can easily
phase this code out with time then.

Signed-off-by: Luis R. Rodriguez 
---
 Documentation/firmware_class/system_data.txt |  79 
 MAINTAINERS  |   4 +-
 drivers/base/firmware_class.c| 291 +++
 include/linux/sysdata.h  | 212 +++
 4 files changed, 585 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/firmware_class/system_data.txt
 create mode 100644 include/linux/sysdata.h

diff --git a/Documentation/firmware_class/system_data.txt 
b/Documentation/firmware_class/system_data.txt
new file mode 100644
index ..ab509243455a
--- /dev/null
+++ b/Documentation/firmware_class/system_data.txt
@@ -0,0 +1,79 @@
+System data requests API
+
+
+As the kernel evolves we keep extending the firmware_class set of APIs
+with more or less arguments, this creates a slew of collateral evolutions.
+The set of users of firmware request APIs has also grown now to include
+users which are not looking for "firmware" per se, but instead general
+system data files which for one reason or another has been decided to be
+kept oustide of the kernel, and/or to allow dynamic updates. The system data
+request set of APIs addresses rebranding of firmware as generic system data