I was wondering if ostest is even supposed to work on AVR so I gave it
a
try... it didn't do much, the only result I got when I ran ostest from
NSH was:
stdio_test: write fd=1
stdio_test: write fd=2
And that's it.
(There may be something wrong with printf when the application gets
executed through NSH though - just noticed that my other testing apps
also print nothing when executed from NSH while being able to printf
when executed as init entrypoint. Will try to update apps to something
more recent but if anyone has any idea what may be causing this, it
would be appreciated.)
Anyway, the compilation also throws a lot of warnings, for example
semtimed.c:87:32: warning: integer overflow in expression of type
‘int’
results in ‘16960’ [-Woverflow]
87 | tp->tv_nsec >= 1000 * 1000 * 1000)
and there are also warnings regarding to size mismatch in
pointer/integer conversion.
So it may be that ostest doesn't work well for AVR (and probably for
other 8 bit MCUs) already. It also already barely fits the flash of
the
largest AVR DA chip.
(Not suggesting to make it even more faulty, just mentioning it FYI.)
On 2025-12-22 13:35, Alan C. Assis wrote:
> Is it too complicated to rewrite the cmocka tests in Unity?
>
> If cmoka is not feasible for small microcontrollers, like AVRs and
> others,
> our test coverage will be faulty.
>
> BR,
>
> Alan
>
> On Mon, Dec 22, 2025 at 6:14 AM raiden00pl <[email protected]>
> wrote:
>
>> I'm a fan of Unity and have been using it in my projects for years,
>> but I
>> think
>> we should stick with one testing framework. Currently, Cmocka is used
>> in
>> nuttx-apps, and we should stick with it. The only question is how well
>> Cmocka
>> is suitable for running on small systems? Will ostest still be able to
>> run
>> on small MCUs?
>> With Unity this would be quite likely, so maybe an exception for Unity
>> and
>> ostest would
>> be acceptable, but using it for other test cases will only cause
>> chaos.
>>
>> pon., 22 gru 2025 o 10:05 Guiding Li <[email protected]> napisał(a):
>>
>> > Hi Matteo,
>> >
>> > Good idea !
>> >
>> > But it is best to stick with a single testing framework. Currently,
many
>> > test cases are using cmocka, so I suggest reusing cmocka for
consistency.
>> >
>> > ostest is a minimal basic test suite, typically used for CT
(continuous
>> > testing) of each patch.
>> > If we move ostest to cmocka, it would make ostest dependent on a
specific
>> > framework before it can be run.
>> > And It may increase the code size, which is not very friendly for
>> > configurations with small flash memory.
>> >
>> > As Greg mentioned: the answer depends on a tradeoff between effort and
>> test
>> > coverage.
>> >
>> > We need to clarify whether we define ostest as a lightweight,
>> ready-to-run
>> > test set for quick verification, or as part of the full test suite.
>> >
>> >
>> > The follow is my compiling for cmocka:
>> >
>> > *ligd@BAI:~/platform/mainline/nuttx$ ./tools/configure.sh sim:citest
>> -j8*
>> > Copy files
>> > Select CONFIG_HOST_LINUX=y
>> > Refreshing...
>> > CP: arch/dummy/Kconfig to
>> > /home/ligd/platform/mainline/nuttx/arch/dummy/dummy_kconfig
>> > CP: boards/dummy/Kconfig to
>> > /home/ligd/platform/mainline/nuttx/boards/dummy/dummy_kconfig
>> > LN: include/arch to arch/sim/include
>> > LN: drivers/platform to
/home/ligd/platform/mainline/nuttx/drivers/dummy
>> > LN: arch/sim/src/chip to
>> > /home/ligd/platform/mainline/nuttx/arch/sim/src/sim
>> > LN: arch/sim/src/board to
>> > /home/ligd/platform/mainline/nuttx/boards/sim/sim/sim/src
>> > LN: include/arch/board to
>> > /home/ligd/platform/mainline/nuttx/boards/sim/sim/sim/include
>> > LN: include/arch/chip to
>> > /home/ligd/platform/mainline/nuttx/arch/sim/include/sim
>> > LN: platform/board to /home/ligd/platform/mainline/apps/platform/dummy
>> > mkkconfig in /home/ligd/platform/mainline/apps/audioutils
>> > mkkconfig in /home/ligd/platform/mainline/apps/benchmarks
>> > mkkconfig in /home/ligd/platform/mainline/apps/boot
>> > mkkconfig in /home/ligd/platform/mainline/apps/canutils
>> > mkkconfig in /home/ligd/platform/mainline/apps/crypto
>> > mkkconfig in /home/ligd/platform/mainline/apps/database
>> > mkkconfig in /home/ligd/platform/mainline/apps/examples/rust
>> > mkkconfig in /home/ligd/platform/mainline/apps/examples/sotest
>> > mkkconfig in /home/ligd/platform/mainline/apps/examples/module
>> > mkkconfig in /home/ligd/platform/mainline/apps/examples/mcuboot
>> > mkkconfig in /home/ligd/platform/mainline/apps/examples
>> > mkkconfig in /home/ligd/platform/mainline/apps/fsutils
>> > mkkconfig in /home/ligd/platform/mainline/apps/games
>> > mkkconfig in /home/ligd/platform/mainline/apps/graphics
>> > mkkconfig in /home/ligd/platform/mainline/apps/industry
>> > mkkconfig in /home/ligd/platform/mainline/apps/inertial
>> > mkkconfig in /home/ligd/platform/mainline/apps/interpreters/luamodules
>> > mkkconfig in /home/ligd/platform/mainline/apps/interpreters
>> > mkkconfig in /home/ligd/platform/mainline/apps/logging
>> > mkkconfig in /home/ligd/platform/mainline/apps/lte
>> > mkkconfig in /home/ligd/platform/mainline/apps/math
>> > mkkconfig in /home/ligd/platform/mainline/apps/mlearning
>> > mkkconfig in /home/ligd/platform/mainline/apps/netutils
>> > mkkconfig in /home/ligd/platform/mainline/apps/sdr
>> > mkkconfig in /home/ligd/platform/mainline/apps/system
>> > mkkconfig in /home/ligd/platform/mainline/apps/tee
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/drivers
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/sched
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/fs
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/mm
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/libc
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/arch
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing/cxx
>> > mkkconfig in /home/ligd/platform/mainline/apps/testing
>> > mkkconfig in /home/ligd/platform/mainline/apps/videoutils
>> > mkkconfig in /home/ligd/platform/mainline/apps/wireless/bluetooth
>> > mkkconfig in /home/ligd/platform/mainline/apps/wireless/ieee802154
>> > mkkconfig in /home/ligd/platform/mainline/apps/wireless
>> > mkkconfig in /home/ligd/platform/mainline/apps
>> > Loaded configuration '.config'
>> > Configuration saved to '.config'
>> > ligd@BAI:~/platform/mainline/nuttx$
>> > *ligd@BAI:~/platform/mainline/nuttx$ make*
>> > Create version.h
>> > Downloading: libcxx-17.0.6.src.tar.xz LN: platform/board to
>> > /home/ligd/platform/mainline/apps/platform/dummy
>> > Downloading: v1.46.0.zip
>> > ...
>> > patching file
>> >
>> >
>>
testcases/open_posix_testsuite/conformance/interfaces/pthread_sigmask/18-1.c
>> > Hunk #1 succeeded at 160 with fuzz 2 (offset -8 lines).
>> >
>> > CP: /home/ligd/platform/mainline/nuttx/include/nuttx/config.h
>> > CP: /home/ligd/platform/mainline/nuttx/include/nuttx/fs/hostfs.h
>> > LD: nuttx
>> > Pac SIM with dynamic libs..
>> > '/lib/x86_64-linux-gnu/libz.so.1' -> 'sim-pac/libs/libz.so.1'
>> > '/lib/x86_64-linux-gnu/libstdc++.so.6' ->
'sim-pac/libs/libstdc++.so.6'
>> > '/lib/x86_64-linux-gnu/libc.so.6' -> 'sim-pac/libs/libc.so.6'
>> > '/lib/x86_64-linux-gnu/libm.so.6' -> 'sim-pac/libs/libm.so.6'
>> > '/lib/x86_64-linux-gnu/libgcc_s.so.1' -> 'sim-pac/libs/libgcc_s.so.1'
>> > '/lib64/ld-linux-x86-64.so.2' -> 'sim-pac/ld-linux-x86-64.so.2'
>> > SIM elf with dynamic libs archive in nuttx.tgz
>> > ligd@BAI:~/platform/mainline/nuttx$
>> > ligd@BAI:~/platform/mainline/nuttx$
>> > ligd@BAI:~/platform/mainline/nuttx$
>> > *ligd@BAI:~/platform/mainline/nuttx$ ./nuttx *
>> > CHelloWorld: Constructor: mSecret=42
>> > nsh: init: open failed: 2
>> >
>> > NuttShell (NSH) NuttX-12.11.0
>> > MOTD: username=admin password=Administrator
>> > nsh>
>> > nsh> help
>> > help usage: help [-v] [<cmd>]
>> >
>> > . df ifconfig mkfifo readlink umount
>> > [ dmesg ifdown mkrd rm unset
>> > ? echo ifup mount rmdir uptime
>> > alias env insmod mv rmmod usleep
>> > unalias exec kill nslookup set watch
>> > basename exit pkill pidof sleep xd
>> > break expr losetup pmconfig source wait
>> > cat false ln poweroff test
>> > cd fdinfo ls quit time
>> > cp free lsmod printf true
>> > cmp help mkdir ps truncate
>> > dirname hexdump mkfatfs pwd uname
>> >
>> > Builtin Apps:
>> > cmocka
>> > cmocka_driver_block
>> > cmocka_driver_gpio
>> > cmocka_driver_pm
>> > cmocka_driver_pm_runtime
>> > cmocka_driver_regulator
>> >
>> > Matteo Golin <[email protected]> 于2025年12月22日周一 10:15写道：
>> >
>> > > I think so. Best to leave OS test alone and slowly replace it with a
>> new
>> > > tool that has improvements. The only reason I can see to modify OS
test
>> > now
>> > > would just be to have clearer output about what passed/failed.
>> > >
>> > > I'll come up with an improved proof of concept for a new tool to
share
>> in
>> > > the next few days.
>> > >
>> > > Matteo
>> > >
>> > > On Sun, Dec 21, 2025, 9:13 PM Tomek CEDRO <[email protected]> wrote:
>> > >
>> > > > Small steps with measurable results may be the best approach
here, to
>> > > > create some new solution next to existing ostest but not
replacing it
>> > > > until ready? :-)
>> > > >
>> > > > --
>> > > > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>> > > >
>> > > > On Mon, Dec 22, 2025 at 3:06 AM Matteo Golin <
[email protected]
>> >
>> > > > wrote:
>> > > > >
>> > > > > I agree, I think it would take a long time to have proper
coverage
>> of
>> > > > NuttX
>> > > > > as a whole. Perhaps even impossible, since Unity is more of a
unit
>> > > > testing
>> > > > > framework and thus there are definitely test cases for NuttX
that
>> > would
>> > > > be
>> > > > > outside of its realm of "applicability". There is still a need
for
>> a
>> > > > > variety of testing applications and bench-marking applications
for
>> > > those
>> > > > > scenarios.
>> > > > >
>> > > > > That makes sense, prerequisites would need to be considered. I
>> think
>> > > > > targeting the low-hanging fruit of unit-testable code would be
easy
>> > in
>> > > > that
>> > > > > case, since prerequisites are mostly encoded in the Kconfig
>> > definitions
>> > > > > existing or not for features to be tested. Any complex behaviour
>> > > > > (interrupts, multi-threading, etc.) might be out of the realm of
>> > what's
>> > > > > testable by Unity in the first place. We'll always have OS test
in
>> > that
>> > > > > case, but maybe then it's better to do what Alan suggested and
at
>> > least
>> > > > > make its output more clear in the meantime. I suppose RTOS
testing
>> > is a
>> > > > > whole field in itself.
>> > > > >
>> > > > > On Sun, Dec 21, 2025 at 8:40 PM Gregory Nutt <
[email protected]>
>> > > > wrote:
>> > > > >
>> > > > > > I think that the answer depends on a tradeoff between effort
and
>> > test
>> > > > > > coverage. I could imagine that a proper job could be
man-years
>> of
>> > > > effort.
>> > > > > >
>> > > > > > My only concern with a re-design is to be certain that tests
are
>> > > > > > arranged so that the prerequisites of later test are tested
>> first.
>> > > If
>> > > > > > tests are grouped by, for example, functional area, then
failures
>> > can
>> > > > be
>> > > > > > difficult to understand if there is a failure in an unverified
>> test
>> > > > > > prerequisite.
>> > > > > >
>> > > > > > Some of the "tests" are bogus and only exist to provide some
>> > minimum
>> > > > > > confidence that we have the functionality to run the test.
For
>> > > > > > example: Can I start tasks? Can I pass parameters? Can I
>> survive
>> > a
>> > > > > > context switch. Some of the tests were intended only to
support
>> > > debug.
>> > > > > > Some were created only to track down specific bugs.
>> > > > > >
>> > > > > > On 12/21/25 18:39, Matteo Golin wrote:
>> > > > > > > LTS being the Long-Term-Stable release of NuttX?
>> > > > > > >
>> > > > > > > I'm starting to think that it would be best to create new
suite
>> > of
>> > > > tests
>> > > > > > > using Unity, using what's in OS Test as a basis and then
>> > expanding
>> > > it
>> > > > > > > further. Greg has pointed out that OS test is not intended
to
>> be
>> > a
>> > > > > > complete
>> > > > > > > test, so maybe a more solid testing framework base in a new
>> form
>> > > > would be
>> > > > > > > in order. At that point it would also be good to document
and
>> > > educate
>> > > > > > users
>> > > > > > > about other test options, since OS test is a current
favourite
>> > for
>> > > > > > showing
>> > > > > > > PRs haven't broken anything. Maybe that's not the best
choice
>> > > anymore
>> > > > > > with
>> > > > > > > this information.
>> > > > > > >
>> > > > > > > I don't want to re-invent the wheel of what's already in
>> > > `testsuite`,
>> > > > > > but I
>> > > > > > > think Unity might end up being lighter-weight in the long
run?
>> > And
>> > > > it is
>> > > > > > > also not a Xiaomi owned/maintained framework so it may
receive
>> > > better
>> > > > > > > external support (i.e. more users rely on it?). I tried to
>> > compile
>> > > > the
>> > > > > > > sim:citest configuration to run the testsuite example and
see
>> > what
>> > > > it was
>> > > > > > > like, but I was met with a long list of C++ compilation
errors
>> > that
>> > > > I'd
>> > > > > > > rather not deal with. There also seems to be zero docs
about it
>> > or
>> > > > cmocka
>> > > > > > > in our NuttX docs at the moment.
>> > > > > > >
>> > > > > > > What do you think? Start a Unity test suite, add some
switches
>> to
>> > > > choose
>> > > > > > > which test cases get compiled into the binary, open for
>> > extensions
>> > > by
>> > > > > > users
>> > > > > > > who know more about different subsystems? This way we can
keep
>> OS
>> > > > test
>> > > > > > how
>> > > > > > > it is for a quick, low-resource sanity check of "board
bringup"
>> > > while
>> > > > > > > having a dedicated testing application for patches? Someone
>> > > changing
>> > > > file
>> > > > > > > system stuff can just compile to include file system test
cases
>> > and
>> > > > run
>> > > > > > > that suite only as proof that their PR works. The output is
>> > really
>> > > > easy
>> > > > > > to
>> > > > > > > glance at to determine pass/fail results.
>> > > > > > >
>> > > > > > > Anyways, here's my draft demonstrating how Unity works with
the
>> > FPU
>> > > > test
>> > > > > > > suite from OS test:
>> > https://github.com/apache/nuttx-apps/pull/3259
>> > > > > > >
>> > > > > > > Matteo
>> > > > > > >
>> > > > > > > On Sun, Dec 21, 2025 at 7:13 PM Tomek CEDRO <
[email protected]>
>> > > > wrote:
>> > > > > > >
>> > > > > > >> On Mon, Dec 22, 2025 at 1:02 AM Gregory Nutt <
>> > [email protected]
>> > > >
>> > > > > > wrote:
>> > > > > > >>> (..)
>> > > > > > >>> Starting with bits and pieces of something like the LTS
might
>> > > make
>> > > > more
>> > > > > > >>> sense other than its licensing problems. Since it would
>> never
>> > be
>> > > > > > >>> included in a distributed binary, the licensing really
should
>> > not
>> > > > > > matter
>> > > > > > >>> much in practice.
>> > > > > > >> Unless its GPL and some company would like to have their
own
>> > > > internal
>> > > > > > >> bits and pieces.. luckily its MIT and the whole thing is
>> > > open-source
>> > > > > > >> anyways :-)
>> > > > > > >>
>> > > > > > >> Yes we had many discussions about the LTS and this could be
>> > great
>> > > > start!
>> > > > > > >> :-)
>> > > > > > >>
>> > > > > > >> --
>> > > > > > >> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
>> > > > > > >>
>> > > > > >
>> > > >
>> > >
>> >
>>
