Hi all,
I'd like to start a separate discussion on a topic that keeps coming
up in two recent threads: the esp-hal-3rdparty governance thread [1]
and the RZ/V2H FSP HAL thread [2].
Both threads have mostly been about where a vendor HAL should live
(ASF, github.com/NuttX or the vendor's repo) and who controls it. I
think we should also look at the interface between NuttX and the HAL,
because that's what decides most of the maintenance cost.
Between the two threads, we've listed most of the problems already.
Vendor code doesn't follow our coding style. We can't easily fix it.
Vendors change priorities. Refactors inside NuttX end up touching
vendor code. Patching the HAL breaks the vendor's sync with upstream
(Tiago explained this well for esp-hal-3rdparty [1]). And some HALs
install their own interrupt handlers or locks that the RTOS doesn't
know about. A native port avoids all of this, and it should stay the
preferred option. But some vendors won't write one, and for large
HALs like FSP or the ESP-IDF components I understand why.
So here's the proposal:
1. NuttX defines a standard HAL interface. This is a documented,
versioned contract describing what NuttX expects from a low-level
peripheral layer (UART, SPI, I2C, GPIO, timers, DMA, ...).
2. For each out-of-tree vendor HAL we can't bring in tree, there is
an adapter that maps the vendor API onto this interface. It could
live in the main repo or in a repo under github.com/NuttX. I don't
have a strong opinion on that, and we can decide it separately.
3. NuttX treats the vendor HAL as a black box, pinned to a specific
version. We never patch it.
4. The vendor is free to update their HAL whenever they want. Vendors
usually keep their public API stable because their own customers
depend on it. As long as the API holds, the adapter doesn't
change. When it doesn't hold, only the adapter changes.
5. NuttX supports any HAL that has an adapter meeting the interface.
The interface also comes with rules. An adapter that can't meet them
doesn't get merged:
- NuttX owns the interrupts. The HAL doesn't install its own vectors.
- The HAL doesn't spawn its own threads or use hidden locks. The
adapter uses NuttX primitives.
- Memory allocation, DMA and cache maintenance are visible to NuttX,
not buried in the HAL.
- Callbacks run in a defined context (interrupt or thread).
- The adapter follows the NuttX coding style and passes NuttX CI.
- CI builds every adapter against its pinned HAL version.
Why I think this works:
- A vendor HAL update never touches NuttX code. At worst it touches
the adapter.
- A NuttX refactor never touches vendor code. It stops at the
interface.
- The vendor source stays untouched, so their sync with upstream keeps
working. This fits the sync/release branch model Tiago described
for esp-hal-3rdparty [1].
- If a vendor walks away, we're left with a small adapter and a pinned
HAL, not vendor code spread through arch/.
- We don't have to force vendors to follow a standard, which was
Alan's question in the RZ/V2H thread [2]. The adapter absorbs
whatever the vendor does.
To be clear, this doesn't replace native ports. Chips we maintain in
tree don't need to go through this interface. They can keep a tighter
integration with NuttX and get the performance and footprint benefits
that come with it. The adapter route is only for complex vendor HALs
that we realistically can't bring in tree.
Most of this interface already exists in practice. The lower-half ops
structures (uart_ops_s, spi_ops_s, i2c_ops_s, ...) already do this
job. What's missing is turning them into an explicit, stable,
versioned contract, adding the rules above, and having CI build each
adapter against a pinned HAL.
In the esp-hal thread [1], Tiago asked some good questions that needed
answers before any vote. Here is how this proposal answers them:
- Does NuttX officially support third-party HALs?
Yes, but only through an adapter to the standard interface.
- Who maintains what?
NuttX committers maintain the adapters, and the vendor maintains
the HAL. Each adapter has at least one vendor contact in
CODEOWNERS.
- What gets forked?
Nothing. We keep an unmodified mirror of each pinned HAL version
under github.com/NuttX, so builds don't break if the vendor moves or
deletes their repo. This also addresses the supply-chain concern
Alin raised [2], and Greg's point that losing the connection to
vendor repos was fatal in the past [2].
- Who defines the rules?
The community, in the interface document, with changes approved by
the PMC.
- Licensing?
The HAL must have an Apache-2.0 compatible license. Adapters are
Apache-2.0.
- CI?
Each adapter builds against its pinned HAL. A HAL update is just a
PR that bumps the pin.
- Where is this documented?
In the contribution guide, with a clear order of preference: native
port first, adapter second.
If people think this is worth pursuing, I'm happy to write a first
draft of the interface document. The RZ/V2H port with FSP looks like a
good pilot to validate the model. Once the details are settled here,
we can open a proper [VOTE] thread with a clear statement of what +1
means.
Looking forward to your thoughts.
Best regards,
Marco Casaroli
References:
[1] Discussing the governance of NuttX's esp-hal-3rdparty dependency
https://lists.apache.org/thread/rqc833zxlty106nv9oh5th3ls8z1vvns
[2] Early feedback on RZ/V2H port and RZ FSP HAL dependency
https://lists.apache.org/thread/gf4c3z4ds94lon91op4c6zmclqnwtvcf
