This is an automated email from the ASF dual-hosted git repository.
acassis pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/nuttx.git
The following commit(s) were added to refs/heads/master by this push:
new b180dc17ae1 Documentation,drivers/aie: align machine learning docs
with current code
b180dc17ae1 is described below
commit b180dc17ae17e8520840ec9cc280e74690405d74
Author: Abhishek Mishra <[email protected]>
AuthorDate: Sat Sep 19 07:24:47 2026 +0000
Documentation,drivers/aie: align machine learning docs with current code
The tflm tool registered DEPTHWISE_CONV_2D in nuttx-apps#3773, but the
docs still listed eight operators. Document the unused -C compile path,
that the sim helper uses heap I/O, and the pinned TFLM/CMSIS/NNABLA
versions. Add missing gemmlowp, KissFFT, Ruy, and FlatBuffers pages,
document the AI-engine character driver, and wire it into CMake.
Signed-off-by: Abhishek Mishra <[email protected]>
---
Documentation/applications/math/gemmlowp/index.rst | 13 +++++++
Documentation/applications/math/kissfft/index.rst | 12 ++++++
Documentation/applications/math/ruy/index.rst | 12 ++++++
.../applications/mlearning/cmsis-nn/index.rst | 5 ++-
.../applications/mlearning/cmsis/index.rst | 6 ++-
.../applications/mlearning/darknet/index.rst | 9 +++--
Documentation/applications/mlearning/index.rst | 14 +++++--
.../applications/mlearning/libnnablart/index.rst | 5 ++-
.../applications/mlearning/tflite-micro/index.rst | 43 +++++++++++++++-------
.../applications/system/flatbuffers/index.rst | 13 +++++++
Documentation/components/drivers/character/aie.rst | 22 +++++++++++
.../components/drivers/character/index.rst | 1 +
Documentation/components/drivers/index.rst | 5 +++
drivers/aie/CMakeLists.txt | 25 +++++++++++++
drivers/aie/Kconfig | 5 ++-
include/nuttx/aie/ai_engine.h | 6 ++-
16 files changed, 168 insertions(+), 28 deletions(-)
diff --git a/Documentation/applications/math/gemmlowp/index.rst
b/Documentation/applications/math/gemmlowp/index.rst
new file mode 100644
index 00000000000..20006da4e35
--- /dev/null
+++ b/Documentation/applications/math/gemmlowp/index.rst
@@ -0,0 +1,13 @@
+=======================
+``gemmlowp`` gemmlowp
+=======================
+
+``apps/math/gemmlowp`` vendors Google `gemmlowp
+<https://github.com/google/gemmlowp>`_, a small low-precision matrix
+library used by TensorFlow Lite Micro.
+
+Enable ``CONFIG_MATH_GEMMLOWP``. The build downloads a pinned snapshot
+(``719139ce755a0f31cbf1c37f7f98adcc7fc9f425``). Headers are added from
+``apps/math/gemmlowp/gemmlowp``.
+
+:doc:`../../mlearning/tflite-micro/index` depends on this package.
diff --git a/Documentation/applications/math/kissfft/index.rst
b/Documentation/applications/math/kissfft/index.rst
new file mode 100644
index 00000000000..fcae4fd94e6
--- /dev/null
+++ b/Documentation/applications/math/kissfft/index.rst
@@ -0,0 +1,12 @@
+===================
+``kissfft`` KissFFT
+===================
+
+``apps/math/kissfft`` vendors `KissFFT
+<https://github.com/mborgerding/kissfft>`_ (tag ``v130``) and applies
+``kissfft.patch``. It compiles ``kiss_fft.c`` and ``tools/kiss_fftr.c``.
+
+Enable ``CONFIG_MATH_KISSFFT``. Include path:
+``apps/math/kissfft/kissfft``.
+
+:doc:`../../mlearning/tflite-micro/index` depends on this package.
diff --git a/Documentation/applications/math/ruy/index.rst
b/Documentation/applications/math/ruy/index.rst
new file mode 100644
index 00000000000..ae4daf085f7
--- /dev/null
+++ b/Documentation/applications/math/ruy/index.rst
@@ -0,0 +1,12 @@
+=============
+``ruy`` Ruy
+=============
+
+``apps/math/ruy`` vendors Google `Ruy <https://github.com/google/ruy>`_,
+a matrix-multiplication library used by TensorFlow Lite Micro.
+
+Enable ``CONFIG_MATH_RUY``. The build downloads a pinned snapshot
+(``d37128311b445e758136b8602d1bbd2a755e115d``). Headers are added from
+``apps/math/ruy/ruy``.
+
+:doc:`../../mlearning/tflite-micro/index` depends on this package.
diff --git a/Documentation/applications/mlearning/cmsis-nn/index.rst
b/Documentation/applications/mlearning/cmsis-nn/index.rst
index c56cde351be..e5bd176c70d 100644
--- a/Documentation/applications/mlearning/cmsis-nn/index.rst
+++ b/Documentation/applications/mlearning/cmsis-nn/index.rst
@@ -9,8 +9,9 @@ TensorFlow Lite Micro uses when ``CONFIG_MLEARNING_CMSIS_NN``
is enabled.
It is distinct from :doc:`../cmsis/index`, which vendors the older CMSIS 5
tree (DSP plus NN) for NNABLA.
-Enable with ``CONFIG_MLEARNING_CMSIS_NN``. The Makefile downloads a pinned
-CMSIS-NN commit and compiles:
+Enable with ``CONFIG_MLEARNING_CMSIS_NN``. The Makefile and CMake build
+download a pinned CMSIS-NN commit
+(``72e1ebf623ab1660a3e14e4e36fdcddce46f1991``) and compile:
- Activation functions
- Basic math
diff --git a/Documentation/applications/mlearning/cmsis/index.rst
b/Documentation/applications/mlearning/cmsis/index.rst
index 4bd872ceb76..d1d7fc786d6 100644
--- a/Documentation/applications/mlearning/cmsis/index.rst
+++ b/Documentation/applications/mlearning/cmsis/index.rst
@@ -5,10 +5,14 @@
``apps/mlearning/cmsis`` fetches `CMSIS 5
<https://github.com/ARM-software/CMSIS_5>`_ and builds CMSIS-DSP and
CMSIS-NN from that tree. Default version is ``5.8.0``
-(``CONFIG_CMSIS_VER``).
+(``CONFIG_CMSIS_VER``). ARM archived the CMSIS 5 repository on 18
+December 2025; NuttX still vendors this tree because NNABLA uses the
+CHW helpers in ``cmsis-nn-support_nnabla.patch``. There is no CMake
+build for this package.
This package is aimed at NNABLA and generic DSP use. For TensorFlow Lite
Micro kernel acceleration, use :doc:`../cmsis-nn/index` instead.
+``CONFIG_CMSIS_NN`` here is not ``CONFIG_MLEARNING_CMSIS_NN``.
Enable ``CONFIG_CMSIS`` from :menuselection:`Application Configuration -->
Machine Learning Support --> CMSIS Libraries`.
diff --git a/Documentation/applications/mlearning/darknet/index.rst
b/Documentation/applications/mlearning/darknet/index.rst
index f1f129e4f2e..9f686097fe7 100644
--- a/Documentation/applications/mlearning/darknet/index.rst
+++ b/Documentation/applications/mlearning/darknet/index.rst
@@ -7,9 +7,12 @@
Only Look Once) object detection.
Enable ``CONFIG_DARKNET_YOLO``. ``CONFIG_DARKNET_YOLO_VER`` is the Git
-branch fetched at build time (default ``master``). The Makefile compiles
-the core Darknet sources (network, parser, convolutional and connected
-layers, YOLO/region/detection layers, RNN/GRU/LSTM, and image helpers).
+branch fetched at build time (default ``master``), not a commit pin.
+Upstream ``pjreddie/darknet`` is unmaintained, so the fetched tree can
+change between builds. There is no NuttX example application and no
+CMake build; the Makefile compiles the core Darknet sources (network,
+parser, convolutional and connected layers, YOLO/region/detection
+layers, RNN/GRU/LSTM, and image helpers).
Include path: ``apps/mlearning/darknet/darknet/include``.
diff --git a/Documentation/applications/mlearning/index.rst
b/Documentation/applications/mlearning/index.rst
index 54fddfd30bd..6231c28520b 100644
--- a/Documentation/applications/mlearning/index.rst
+++ b/Documentation/applications/mlearning/index.rst
@@ -15,14 +15,20 @@ Enable the libraries from::
The following packages are available:
-- :doc:`cmsis/index` — ARM CMSIS 5 DSP and NN libraries
+- :doc:`cmsis/index` — ARM CMSIS 5 DSP and NN libraries (Makefile only)
- :doc:`cmsis-nn/index` — standalone ARM CMSIS-NN kernels for TensorFlow Lite
Micro
-- :doc:`darknet/index` — Darknet YOLO object detection
-- :doc:`libnnablart/index` — Sony NNABLA C inference runtime
+- :doc:`darknet/index` — Darknet YOLO object detection (Makefile only)
+- :doc:`libnnablart/index` — Sony NNABLA C inference runtime (Makefile only)
- :doc:`tflite-micro/index` — TensorFlow Lite for Microcontrollers
+``CONFIG_CMSIS_NN`` (nested under CMSIS Libraries) is not the same
+option as ``CONFIG_MLEARNING_CMSIS_NN`` (standalone CMSIS-NN used by
+TFLM). CMake currently builds ``tflite-micro`` and ``cmsis-nn`` only.
+
A ready-to-build simulator configuration is ``sim:tflm``. See
-:doc:`/platforms/sim/sim/boards/sim/index`.
+:doc:`/platforms/sim/sim/boards/sim/index`. The kernel also has an
+optional AI-engine character driver (``CONFIG_AI_ENGINE``); see
+:doc:`/components/drivers/character/aie`.
.. toctree::
:glob:
diff --git a/Documentation/applications/mlearning/libnnablart/index.rst
b/Documentation/applications/mlearning/libnnablart/index.rst
index 489818a08f6..5f49a7a1e1f 100644
--- a/Documentation/applications/mlearning/libnnablart/index.rst
+++ b/Documentation/applications/mlearning/libnnablart/index.rst
@@ -7,7 +7,10 @@
networks produced by Neural Network Libraries.
Enable ``CONFIG_NNABLA_RT``. ``CONFIG_NNABLA_RT_VER`` selects the upstream
-tag (default ``1.24.0``). The Makefile downloads that release and compiles
+tag (default ``1.24.0``). Sony's later ``nnabla-c-runtime`` tags go
+through ``v1.38.0``; NuttX has not moved the default. Sony has placed
+NNABLA in maintenance. There is no NuttX example application and no
+CMake build. The Makefile downloads the selected release and compiles
the runtime plus a subset of operators, including:
- Activations (ReLU, sigmoid, tanh, softmax, ELU, leaky ReLU, Swish, ...)
diff --git a/Documentation/applications/mlearning/tflite-micro/index.rst
b/Documentation/applications/mlearning/tflite-micro/index.rst
index 4925081cba0..ccac7788dd2 100644
--- a/Documentation/applications/mlearning/tflite-micro/index.rst
+++ b/Documentation/applications/mlearning/tflite-micro/index.rst
@@ -6,7 +6,8 @@ TensorFlow Lite for Microcontrollers (TFLM) is a C++
interpreter for
running ``.tflite`` models on memory-constrained targets. NuttX integrates
it from ``apps/mlearning/tflite-micro``.
-The build downloads a pinned TFLM snapshot from
+The build downloads a pinned TFLM snapshot
+(``cfa4c91d1b36c37c7c104b9c664615e59f1abfe3``, 24 February 2024) from
https://github.com/tensorflow/tflite-micro and applies NuttX patches that
add INT8-only operator registrations and an ``extern "C"`` entry point for
the hello-world example.
@@ -16,10 +17,14 @@ Dependencies
``CONFIG_TFLITEMICRO`` depends on all of:
-- ``CONFIG_SYSTEM_FLATBUFFERS`` — FlatBuffers headers
(``apps/system/flatbuffers``)
-- ``CONFIG_MATH_GEMMLOWP`` — gemmlowp (``apps/math/gemmlowp``)
-- ``CONFIG_MATH_KISSFFT`` — KissFFT (``apps/math/kissfft``)
-- ``CONFIG_MATH_RUY`` — Ruy (``apps/math/ruy``)
+- ``CONFIG_SYSTEM_FLATBUFFERS`` — FlatBuffers headers
+ (:doc:`../../system/flatbuffers/index`)
+- ``CONFIG_MATH_GEMMLOWP`` — gemmlowp
+ (:doc:`../../math/gemmlowp/index`)
+- ``CONFIG_MATH_KISSFFT`` — KissFFT
+ (:doc:`../../math/kissfft/index`)
+- ``CONFIG_MATH_RUY`` — Ruy
+ (:doc:`../../math/ruy/index`)
TFLM is C++, so the configuration also needs C++ support (``CONFIG_HAVE_CXX``
and a C++ standard library such as ``CONFIG_LIBCXX``).
@@ -95,10 +100,12 @@ unpacked tree under
``apps/mlearning/tflite-micro/tflite-micro``.
Using the ``tflm`` tool
=======================
-``tflm`` loads a ``.tflite`` file from the filesystem, constructs a
-``tflite::MicroInterpreter``, calls ``AllocateTensors()``, and can invoke
-the model once for profiling or emit compiled C++ (when TFLM was built
-with ``TFLITE_MODEL_COMPILER``).
+``tflm`` is a host-oriented NSH helper for ``sim:tflm``. It loads a
+``.tflite`` file from the filesystem with ``ifstream``, allocates the
+tensor arena with ``new``, constructs a ``tflite::MicroInterpreter``,
+and calls ``AllocateTensors()``. That heap-and-filesystem path is
+intentional on the simulator. On-target applications should embed the
+model as a C array; see *Embedding a model in an application* below.
.. code-block:: console
@@ -116,10 +123,16 @@ with ``TFLITE_MODEL_COMPILER``).
``-i`` is required. ``-o`` is required only with ``-C``. Defaults are
prefix ``NXAI`` and arena size 8192 bytes.
-The built-in operator resolver registers eight generic (float and
+``-C`` appears in the help text but is not functional in NuttX builds.
+Model compilation requires ``TFLITE_MODEL_COMPILER``, which neither the
+Makefile nor the CMake integration defines. ``tflm -C`` prints
+``Not supported compiling``.
+
+The built-in operator resolver registers nine generic (float and
quantized) ops:
- ``CONV_2D``
+- ``DEPTHWISE_CONV_2D``
- ``MAX_POOL_2D``
- ``QUANTIZE``
- ``DEQUANTIZE``
@@ -184,7 +197,7 @@ and the NuttX apps tree next to ``nuttx`` (``../apps`` or
nsh> tflm -E -i /path/to/model.tflite -a 8192
The tool fails with ``AllocateTensors failed`` if the arena is too
- small or the model uses operators outside the eight registered ops.
+ small or the model uses operators outside the nine registered ops.
CMake is equivalent: ``cmake -B build -DBOARD_CONFIG=sim:tflm -GNinja``
then ``cmake --build build`` and ``./build/nuttx``.
@@ -192,9 +205,11 @@ then ``cmake --build build`` and ``./build/nuttx``.
Embedding a model in an application
===================================
-TFLM is designed for targets without a filesystem and without dynamic
-allocation for the model itself. Typical NuttX applications compile the
-``.tflite`` file into a C array and pass it to ``tflite::GetModel()``.
+The TFLM library is designed for targets without a filesystem and
+without dynamic allocation for the model itself. The ``tflm`` NSH tool
+is an exception used on ``sim:tflm``. Typical on-target applications
+compile the ``.tflite`` file into a C array and pass it to
+``tflite::GetModel()``.
The CMake helper ``tflite_generate_data()`` in
``apps/mlearning/tflite-micro/CMakeLists.txt`` wraps ``xxd -i`` for that
diff --git a/Documentation/applications/system/flatbuffers/index.rst
b/Documentation/applications/system/flatbuffers/index.rst
new file mode 100644
index 00000000000..a7ad75b0375
--- /dev/null
+++ b/Documentation/applications/system/flatbuffers/index.rst
@@ -0,0 +1,13 @@
+=============================
+``flatbuffers`` FlatBuffers
+=============================
+
+``apps/system/flatbuffers`` vendors Google `FlatBuffers
+<https://github.com/google/flatbuffers>`_ ``v23.5.26`` and applies
+``flatbuffers.patch``. TensorFlow Lite Micro uses the headers to read
+``.tflite`` model schemas.
+
+Enable ``CONFIG_SYSTEM_FLATBUFFERS``. Include path:
+``apps/system/flatbuffers/flatbuffers/include``.
+
+:doc:`../../mlearning/tflite-micro/index` depends on this package.
diff --git a/Documentation/components/drivers/character/aie.rst
b/Documentation/components/drivers/character/aie.rst
new file mode 100644
index 00000000000..385d1256111
--- /dev/null
+++ b/Documentation/components/drivers/character/aie.rst
@@ -0,0 +1,22 @@
+========================
+AI Engine (AIE) Drivers
+========================
+
+``drivers/aie`` is an upper-half character driver for hardware neural
+processing units. Enable ``CONFIG_AI_ENGINE``. There is no in-tree
+lower-half or board configuration yet; a platform must implement
+``struct aie_ops_s`` and call ``aie_register()``.
+
+The public header is ``include/nuttx/aie/ai_engine.h``. Ioctl commands:
+
+- ``AIE_CMD_LOAD`` — load a model. The argument is a model pointer
+ passed to ``ops->init()``. A second load on the same file returns
+ ``-EINVAL``.
+- ``AIE_CMD_FEED_INPUT`` — feed one input tensor via ``ops->feed_input()``.
+- ``AIE_CMD_GET_OUTPUT`` — read one output tensor via ``ops->get_output()``.
+
+Other ioctl numbers are forwarded to ``ops->control()`` when that
+callback is provided, otherwise ``-ENOSYS``.
+
+This kernel driver is independent of the application-level TinyML
+packages under :doc:`/applications/mlearning/index`.
diff --git a/Documentation/components/drivers/character/index.rst
b/Documentation/components/drivers/character/index.rst
index ab597a36a72..3518e9c73c2 100644
--- a/Documentation/components/drivers/character/index.rst
+++ b/Documentation/components/drivers/character/index.rst
@@ -55,6 +55,7 @@ Character device drivers have these properties:
:maxdepth: 2
1wire.rst
+ aie.rst
analog/index.rst
bch.rst
can.rst
diff --git a/Documentation/components/drivers/index.rst
b/Documentation/components/drivers/index.rst
index 743e7254083..dd2e70daebe 100644
--- a/Documentation/components/drivers/index.rst
+++ b/Documentation/components/drivers/index.rst
@@ -41,6 +41,11 @@ Subdirectories of ``nuttx/drivers``
1wire device drivers.
+* ``aie/`` :doc:`character/aie`
+
+ Upper-half character driver for hardware AI / NPU engines.
+ See ``include/nuttx/aie/ai_engine.h``.
+
* ``analog/`` :doc:`character/analog/index`
This directory holds implementations of analog device drivers.
diff --git a/drivers/aie/CMakeLists.txt b/drivers/aie/CMakeLists.txt
new file mode 100644
index 00000000000..dca2146aa1b
--- /dev/null
+++ b/drivers/aie/CMakeLists.txt
@@ -0,0 +1,25 @@
+#
##############################################################################
+# drivers/aie/CMakeLists.txt
+#
+# SPDX-License-Identifier: Apache-2.0
+#
+# Licensed to the Apache Software Foundation (ASF) under one or more
contributor
+# license agreements. See the NOTICE file distributed with this work for
+# additional information regarding copyright ownership. The ASF licenses this
+# file to you under the Apache License, Version 2.0 (the "License"); you may
not
+# use this file except in compliance with the License. You may obtain a copy
of
+# the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+# License for the specific language governing permissions and limitations under
+# the License.
+#
+#
##############################################################################
+
+if(CONFIG_AI_ENGINE)
+ target_sources(drivers PRIVATE ai_engine.c)
+endif()
diff --git a/drivers/aie/Kconfig b/drivers/aie/Kconfig
index 57a708b03d4..dcbebe1642b 100644
--- a/drivers/aie/Kconfig
+++ b/drivers/aie/Kconfig
@@ -7,4 +7,7 @@ config AI_ENGINE
bool "AI Engine Acceleration Support"
default n
---help---
- Drivers for various AI engine devices.
+ Upper-half character driver for hardware AI / NPU engines.
+ A board or architecture lower-half must implement struct
+ aie_ops_s and call aie_register(). There is no in-tree
+ lower-half yet. See include/nuttx/aie/ai_engine.h.
diff --git a/include/nuttx/aie/ai_engine.h b/include/nuttx/aie/ai_engine.h
index 1b100d7f00e..dbb09184494 100644
--- a/include/nuttx/aie/ai_engine.h
+++ b/include/nuttx/aie/ai_engine.h
@@ -1,6 +1,8 @@
/****************************************************************************
* include/nuttx/aie/ai_engine.h
*
+ * SPDX-License-Identifier: Apache-2.0
+ *
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership. The
@@ -85,10 +87,10 @@ struct aie_lowerhalf_s
****************************************************************************/
/****************************************************************************
- * Name: ai_engine_register
+ * Name: aie_register
*
* Description:
- * Register all ai engine related drivers.
+ * Register an AI engine character driver at path.
*
****************************************************************************/
