aviralgarg05 opened a new pull request, #3642: URL: https://github.com/apache/nuttx-apps/pull/3642
*Note: Please adhere to [Contributing Guidelines](https://github.com/apache/nuttx/blob/master/CONTRIBUTING.md).* ## Summary This is another slice of the ongoing GSoC 2026 Dynamic ELF loading and `nxpkg` work for NuttX, following on from the initial package lifecycle helper merged in #3474. An earlier draft PR (#3474) carries the initial `nxpkg` package lifecycle helper — local metadata parsing, a local `install` / `list` flow, compatibility checks, transaction-state handling, and integrity verification. That initial slice was intentionally local-only: no network sync, no update/rollback execution, no remove. Series order for this PR: - base series context: #3474 (merged) - technical dependency: after #3474 - independent of: #3519, #3526, #3531 This PR fills in most of what #3474 explicitly deferred, plus hardening found while getting the package path solid enough to build a GUI frontend on top of it (a separate follow-on PR). Concretely, it: - adds bounded memory-safety helpers (`pkg_zalloc`/`pkg_malloc`/`pkg_free`) and explicit size limits used throughout the rest of the package path - hardens on-device storage read/write against untrusted or partially written input, rather than assuming every file it reads back was written by a well-behaved prior run - rejects path-traversal sequences in manifest `name`/`version` before they are spliced into an on-device path — a manifest's `name`/`version` ultimately comes from a repository index, which should not be implicitly trusted to be well-formed - supports loading a package index from an arbitrary path, not just a single hardcoded location - adds real network sync and artifact acquisition (a plain HTTP GET, verified via SHA-256 rather than requiring TLS or any auth — the intent is that any static file host works, no special server needed) - rewrites `install` to support network sources and to reclaim state left behind by an interrupted previous install/download, instead of getting stuck on stale partial state - wires `sync` / `remove` / `rollback` / `available` into the CLI — previously stub error paths - raises `PKG_INDEX_MAX` now that the in-memory index is heap-allocated rather than a fixed-size stack/static array - adds an optional manifest `icon` field and the source-resolution logic for it (consumed by the nxstore GUI frontend) - fixes a lost-update race on the shared installed-packages database: two independent installs of different packages could each read the on-disk installed-state file, mutate their own in-memory copy, and write back — the second writer's write would silently discard the first writer's already-applied change. Fixed with a global installed- state lock so concurrent installs serialize around that shared file instead of clobbering each other - documents the repository layout and how to stand up a local package server (`system/nxpkg/README.txt`) - fixes a real build break: `pkg_runtime_compat()` unconditionally referenced `CONFIG_ARCH_BOARD`, a Kconfig string symbol with no `default` clause under `ARCH_BOARD_CUSTOM` — on a custom board it is left completely undefined rather than defined-but-empty, so the previous code failed to compile the moment `CONFIG_SYSTEM_NXPKG` was enabled on a custom-board config. Fixed by falling back to `CONFIG_ARCH_BOARD_CUSTOM_NAME` (only `#define`'d when `ARCH_BOARD_CUSTOM` is selected), and routed `pkg_error()`/`pkg_info()` through `syslog` instead of stdio, since neither is visible to a supervisor process with no attached console ## Impact User impact: - `nxpkg` now supports the full lifecycle the initial slice deferred: `sync`, `install` (local or network source), `update`, `remove`, `rollback`, `list`, `available` - fixes a real compile failure on any board selected via `ARCH_BOARD_CUSTOM` (a custom board directory, as opposed to one of the named boards `ARCH_BOARD` has a `default` clause for) with `CONFIG_SYSTEM_NXPKG` enabled Build impact: - no new Kconfig dependencies beyond what #3474 already required (`NETUTILS_CJSON`); network sync additionally needs `NETUTILS_WEBCLIENT` Runtime impact: - adds network I/O and a global install-state lock to the install path; local-only installs are otherwise unaffected in behavior - `pkg_error()`/`pkg_info()` now log via `syslog` instead of directly to stdio Compatibility impact: - none for existing applications unless `CONFIG_SYSTEM_NXPKG` is enabled - the on-disk installed-state format gained fields (icon source, lock file); already-installed state from #3474's format is not migrated, since #3474 itself is not yet in a released version ## Testing Host used for build: - OS: macOS 26.5 arm64 - compiler: `xtensa-esp-elf-gcc (crosstool-NG esp-14.2.0_20241119) 14.2.0` Target used for verification: - arch: `xtensa` - board/config used during runtime verification: `esp32s3-touch-lcd-7:nsh` - board: Waveshare ESP32-S3-Touch-LCD-7 (custom board directory, not yet upstream) This series was developed and continuously exercised on real hardware over the course of building the `nxstore` GUI frontend on top of it (a separate follow-on PR) — every `nxpkg` code path this PR touches is what that frontend calls into for every install/launch/close cycle across dozens of manual test passes on the physical board, including: - network sync against a local static-file repository server - installing packages whose version strings are longer than a single character (this specifically exercises a since-fixed FAT 8.3 short-filename issue in a related, already-shipped part of the storage path — not part of this diff, but this PR's install/sync rewrite is what exercises that path end-to-end) - concurrent-install handling for the installed-state lock fix, by triggering installs of two different packages back to back and confirming both ended up correctly recorded in the installed database afterward, rather than one silently overwriting the other Focused verification for the final commit in this PR (the `ARCH_BOARD_CUSTOM` build fix): ``` $ xtensa-esp32s3-elf-gcc -c -fsyntax-only \ -I <nuttx>/include -isystem <nuttx>/include -D__NuttX__ -D__KERNEL__ \ -I <apps>/include -I system/nxpkg \ system/nxpkg/pkg_compat.c system/nxpkg/pkg_log.c ``` compiles clean against this project's own generated `include/nuttx/config.h` for `esp32s3-touch-lcd-7:nsh`, where `CONFIG_ARCH_BOARD` is confirmed absent and only `CONFIG_ARCH_BOARD_CUSTOM_NAME` is defined — reproducing the exact build-break condition and confirming the fallback resolves it. This is a draft PR — opening now to get the series in front of reviewers; happy to split further if any single commit above should be its own PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
