How to unsbcribe? Pierre-Noël Bouteville Envoyé de mon iPhone
> Le 22 déc. 2025 à 19:13, raiden00pl <[email protected]> a écrit : > > Cmocka tests can run on any target. It doesn't matter if it's sim, qemu or > real hardware. > You just need to prepare a defconfig and have the right amount of FLASH. > > For small systems we can always try to reduce the size of the cmock, e.g. > by disabling > unnecessary prints. I doubt that the logic of running tests in cmoka is > heavy, running > test cases are not a complicated operation, I think most of the required > FLASH is wasted > on strings. > > NTFC supports serial port communication and sample test scenario based on > nucleo-h743z > with cmocka tests enabled. The only thing missing for full test automation > (compile, flash, > test, result) is cmake support for testsuites. > > pon., 22 gru 2025 o 18:08 Matteo Golin <[email protected]> napisał(a): > >> Hello everyone, >> >> Thanks for all the feedback! I'm glad I discussed this here first. I'll try >> to address everything that was brought up. >> >> Thanks Guiding for the input! I must just be missing some library on my >> machine if you're able to compile it just fine. Maybe I'll track down that >> error and see if you can recognize what's wrong. >> >> I don't intend to replace OS test anymore. I think it serves its purpose as >> a minimal sanity check. Instead, I think we need something a little more >> involved to test PRs since OS test can't catch many issues. It is currently >> the standard everyone uses for verifying their patches and it would be >> better to have more complete, alternate options. I do think that we can all >> agree that OS test needs a revision of its console output so that it is >> much easier to see pass/fail results, but that is just a matter of print >> statements and can be done in a different effort. >> >> I think the overwhelming concern here is mixing two unit testing frameworks >> and no longer having one resource where we can go for tests. I share raiden >> and Guiding's concern about that, which is why I'm glad I opened for >> discussion here. On the one hand, Xiaomi has a pretty substantial >> collection of tests in the `testsuite` application which they are using >> internally I believe (in their CI). These run well on virtual machines, >> which I assume is why the only configurations with that test suite included >> is sim and rv-virt. I do not know nearly enough about cmocka to make any >> claim that it *wouldn't* work on smaller hardware devices, but it also >> definitely looks to be less light-weight from the feature set in the >> documentation. On the other hand, the Xiaomi test suites are also not a >> complete silver bullet since they seem not to be intended to run on >> hardware, and there are *a lot *of tests that need hardware verification. >> In PRs we often request more than just a simulator test for certain >> changes. For instance, sim isn't able to have `/dev/rand`, so that needs to >> be tested on a different target. >> >> Maybe someone more familiar with Xiaomi's testing practices can weigh in on >> whether cmocka has been run on hardware. Going purely based off of their >> PRs, which almost exclusively cite tests on simulators, my guess is that >> cmocka was not designed with embedded in mind when it was written. >> >> As for size: unit testing can get large with Unity or cmocka (as raiden >> mentioned) as more tests are added. Hence my effort to enable/disable >> individual test suites from getting compiled. It might not be possible to >> ever get good testing ability for AVR just because the devices are so >> small. Whatever solution is used there is probably going to involve >> flashing the device multiple times with different tests. The size goal >> should just be to target the most NuttX devices possible. >> >> Then there are the pros and cons of each framework (again, I don't know >> much about cmocka so take it with a grain of salt): >> >> Unity's cons: >> - Designed for each test-suite to be a standalone binary. This is clunky on >> NuttX, so `nest` is currently implemented as a mega-suite where test cases >> just follow a naming convention to make it clear what is being tested >> - setUp/tearDown can't really be leveraged because of the above >> - I've noticed some funky behaviour using TEST_IGNORE in certain scenarios >> but that's possibly my own configuration issue >> - Would be in conflict with existing cmocka tests >> >> Unity's pros: >> - Highly configurable, like NuttX >> - Light-weight and works for embedded devices >> - Significantly larger user base than cmocka >> >> Cmocka's cons: >> - Possibly (?) not light-weight >> - No documentation for existing test cases in NuttX >> - Small user base >> >> Cmocka's pros: >> - Seems to have a larger set of features, including setup/teardown >> functions per-test case >> - Existing tests in NuttX >> >> Irrespective of whether we go with Unity or cmocka, the collection of tests >> that we write will be intended to run on the actual hardware devices >> themselves for the purposes of testing patches and doing regression tests. >> As a result, the tests will be separate from the Xiaomi 'testsuite' >> application which is intended for CI testing. I think this realistically >> nullifies choosing which framework to use on the basis of consistency, >> since the tests are not coexisting with the cmocka tests Xiaomi has >> written. Instead, the choice should be based on what best achieves the goal >> from a functionality perspective. Unity seems like a really great choice >> for this in my opinion, but I am bothered by the fact that it's designed >> with multiple test binaries in mind. >> >> Let me know what you think, >> Matteo >> >>> On Mon, Dec 22, 2025 at 11:02 AM Alan C. Assis <[email protected]> wrote: >>> >>> Yes, I agree! We need data size from each solution for comparison, this >> is >>> the first step. >>> >>> Maybe the conversion could be facilitated if we create equivalent >> functions >>> like those used in the apps/testing/testsuites: >>> >>> cmocka_run_group_tests -> unity_run_group_tests >>> cmocka_unit_test_setup_teardown -> unity_unit_test_setup_teardown >>> etc... >>> >>> This way we avoid end up with two unit test framework, we already have >> the >>> Makefile vs CMake precedent! :-D >>> >>> On Mon, Dec 22, 2025 at 11:27 AM raiden00pl <[email protected]> >> wrote: >>> >>>> The discussion about small systems doesn't make much sense without >>>> data comparing resources of the same test cases for cmock and unity. >>>> >>>> But let's assume that Unity is better. Should we use two frameworks >>>> to implement unit tests for OS? Or choose one framework? If we choose >>> only >>>> Unity, who >>>> will migrate the current cmocka tests to Unity? Remember that cmoka >> tests >>>> are used in CI, so all CI tools must be also migrated. >>>> Or should we use two different frameworks to test different things and >>> make >>>> even >>>> more mess in apps/testing as it is now? >>>> >>>> No one is forcing anyone to do anything. This is an open-source >> project, >>>> everyone >>>> should do what they feel is right. But whether the change should be >>> merged >>>> is >>>> a different question. In this case, it's a very important question, >>> because >>>> unit testing >>>> isn't just about test cases. It's about the entire infrastructure that >>>> runs, captures and >>>> handles test results later. Writing the test cases is the least of the >>>> problems. >>>> The infrastructure for cmoka testing is already in place, the >>>> infrastructure for Unity >>>> testing isn't. >>>> >>>> pon., 22 gru 2025 o 14:16 Alan C. Assis <[email protected]> >> napisał(a): >>>> >>>>> Mateusz, >>>>> but some chips like AVR DA have big flash, but not so much RAM. >>>>> >>>>> It is better to have some possible way to test on a small system, >> using >>>>> cmocka we are removing this option from the table. >>>>> >>>>> I liked the test Matteo did with Unity and I think we should let him >>>>> continue. Forcing him to migrate to cmocka just because there's a >> test >>>>> suite from Xiaomi doesn't seem prudent to me. >>>>> >>>>> This is the advantage of an open mind open-source project: "the >>>> possibility >>>>> to test new ideas", if someone had said to Michal Lenc he couldn't >>> create >>>>> NXBoot because where is already MCUBoot integrated on NuttX, we >> didn't >>>> have >>>>> the option to use NXBoot today. ;-) >>>>> >>>>> BR, >>>>> >>>>> Alan >>>>> >>>>> On Mon, Dec 22, 2025 at 10:02 AM raiden00pl <[email protected]> >>>> wrote: >>>>> >>>>>> Unit testing by itself isn't suitable for small systems. Ostest is >>>>>> something >>>>>> different (system level testing), so running it on small systems >>> makes >>>>>> sense. >>>>>> Running unit tests on small systems is pointless because you won't >>> fit >>>> in >>>>>> FLASH >>>>>> either way. You can run the same on large targets (preferably on >>> host: >>>>> sim, >>>>>> qemu) >>>>>> and achieve the same result. >>>>>> >>>>>> pon., 22 gru 2025 o 13:35 Alan C. Assis <[email protected]> >>>> napisał(a): >>>>>> >>>>>>> Is it too complicated to rewrite the cmocka tests in Unity? >>>>>>> >>>>>>> If cmoka is not feasible for small microcontrollers, like AVRs >> and >>>>>> others, >>>>>>> our test coverage will be faulty. >>>>>>> >>>>>>> BR, >>>>>>> >>>>>>> Alan >>>>>>> >>>>>>> On Mon, Dec 22, 2025 at 6:14 AM raiden00pl <[email protected] >>> >>>>> wrote: >>>>>>> >>>>>>>> I'm a fan of Unity and have been using it in my projects for >>> years, >>>>>> but I >>>>>>>> think >>>>>>>> we should stick with one testing framework. Currently, Cmocka >> is >>>> used >>>>>> in >>>>>>>> nuttx-apps, and we should stick with it. The only question is >> how >>>>> well >>>>>>>> Cmocka >>>>>>>> is suitable for running on small systems? Will ostest still be >>> able >>>>> to >>>>>>> run >>>>>>>> on small MCUs? >>>>>>>> With Unity this would be quite likely, so maybe an exception >> for >>>>> Unity >>>>>>> and >>>>>>>> ostest would >>>>>>>> be acceptable, but using it for other test cases will only >> cause >>>>> chaos. >>>>>>>> >>>>>>>> pon., 22 gru 2025 o 10:05 Guiding Li <[email protected]> >>>> napisał(a): >>>>>>>> >>>>>>>>> Hi Matteo, >>>>>>>>> >>>>>>>>> Good idea ! >>>>>>>>> >>>>>>>>> But it is best to stick with a single testing framework. >>>> Currently, >>>>>>> many >>>>>>>>> test cases are using cmocka, so I suggest reusing cmocka for >>>>>>> consistency. >>>>>>>>> >>>>>>>>> ostest is a minimal basic test suite, typically used for CT >>>>>> (continuous >>>>>>>>> testing) of each patch. >>>>>>>>> If we move ostest to cmocka, it would make ostest dependent >> on >>> a >>>>>>> specific >>>>>>>>> framework before it can be run. >>>>>>>>> And It may increase the code size, which is not very friendly >>> for >>>>>>>>> configurations with small flash memory. >>>>>>>>> >>>>>>>>> As Greg mentioned: the answer depends on a tradeoff between >>>> effort >>>>>> and >>>>>>>> test >>>>>>>>> coverage. >>>>>>>>> >>>>>>>>> We need to clarify whether we define ostest as a lightweight, >>>>>>>> ready-to-run >>>>>>>>> test set for quick verification, or as part of the full test >>>> suite. >>>>>>>>> >>>>>>>>> >>>>>>>>> The follow is my compiling for cmocka: >>>>>>>>> >>>>>>>>> *ligd@BAI:~/platform/mainline/nuttx$ ./tools/configure.sh >>>>> sim:citest >>>>>>>> -j8* >>>>>>>>> Copy files >>>>>>>>> Select CONFIG_HOST_LINUX=y >>>>>>>>> Refreshing... >>>>>>>>> CP: arch/dummy/Kconfig to >>>>>>>>> /home/ligd/platform/mainline/nuttx/arch/dummy/dummy_kconfig >>>>>>>>> CP: boards/dummy/Kconfig to >>>>>>>>> /home/ligd/platform/mainline/nuttx/boards/dummy/dummy_kconfig >>>>>>>>> LN: include/arch to arch/sim/include >>>>>>>>> LN: drivers/platform to >>>>>>> /home/ligd/platform/mainline/nuttx/drivers/dummy >>>>>>>>> LN: arch/sim/src/chip to >>>>>>>>> /home/ligd/platform/mainline/nuttx/arch/sim/src/sim >>>>>>>>> LN: arch/sim/src/board to >>>>>>>>> /home/ligd/platform/mainline/nuttx/boards/sim/sim/sim/src >>>>>>>>> LN: include/arch/board to >>>>>>>>> /home/ligd/platform/mainline/nuttx/boards/sim/sim/sim/include >>>>>>>>> LN: include/arch/chip to >>>>>>>>> /home/ligd/platform/mainline/nuttx/arch/sim/include/sim >>>>>>>>> LN: platform/board to >>>>>> /home/ligd/platform/mainline/apps/platform/dummy >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/audioutils >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/benchmarks >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/boot >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/canutils >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/crypto >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/database >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/examples/rust >>>>>>>>> mkkconfig in >> /home/ligd/platform/mainline/apps/examples/sotest >>>>>>>>> mkkconfig in >> /home/ligd/platform/mainline/apps/examples/module >>>>>>>>> mkkconfig in >> /home/ligd/platform/mainline/apps/examples/mcuboot >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/examples >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/fsutils >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/games >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/graphics >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/industry >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/inertial >>>>>>>>> mkkconfig in >>>>>> /home/ligd/platform/mainline/apps/interpreters/luamodules >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/interpreters >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/logging >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/lte >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/math >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/mlearning >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/netutils >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/sdr >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/system >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/tee >>>>>>>>> mkkconfig in >> /home/ligd/platform/mainline/apps/testing/drivers >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing/sched >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing/fs >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing/mm >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing/libc >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing/arch >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing/cxx >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/testing >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/videoutils >>>>>>>>> mkkconfig in >>> /home/ligd/platform/mainline/apps/wireless/bluetooth >>>>>>>>> mkkconfig in >>>> /home/ligd/platform/mainline/apps/wireless/ieee802154 >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps/wireless >>>>>>>>> mkkconfig in /home/ligd/platform/mainline/apps >>>>>>>>> Loaded configuration '.config' >>>>>>>>> Configuration saved to '.config' >>>>>>>>> ligd@BAI:~/platform/mainline/nuttx$ >>>>>>>>> *ligd@BAI:~/platform/mainline/nuttx$ make* >>>>>>>>> Create version.h >>>>>>>>> Downloading: libcxx-17.0.6.src.tar.xz LN: platform/board to >>>>>>>>> /home/ligd/platform/mainline/apps/platform/dummy >>>>>>>>> Downloading: v1.46.0.zip >>>>>>>>> ... >>>>>>>>> patching file >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> testcases/open_posix_testsuite/conformance/interfaces/pthread_sigmask/18-1.c >>>>>>>>> Hunk #1 succeeded at 160 with fuzz 2 (offset -8 lines). >>>>>>>>> >>>>>>>>> CP: >> /home/ligd/platform/mainline/nuttx/include/nuttx/config.h >>>>>>>>> CP: >>> /home/ligd/platform/mainline/nuttx/include/nuttx/fs/hostfs.h >>>>>>>>> LD: nuttx >>>>>>>>> Pac SIM with dynamic libs.. >>>>>>>>> '/lib/x86_64-linux-gnu/libz.so.1' -> 'sim-pac/libs/libz.so.1' >>>>>>>>> '/lib/x86_64-linux-gnu/libstdc++.so.6' -> >>>>>> 'sim-pac/libs/libstdc++.so.6' >>>>>>>>> '/lib/x86_64-linux-gnu/libc.so.6' -> 'sim-pac/libs/libc.so.6' >>>>>>>>> '/lib/x86_64-linux-gnu/libm.so.6' -> 'sim-pac/libs/libm.so.6' >>>>>>>>> '/lib/x86_64-linux-gnu/libgcc_s.so.1' -> >>>>> 'sim-pac/libs/libgcc_s.so.1' >>>>>>>>> '/lib64/ld-linux-x86-64.so.2' -> >> 'sim-pac/ld-linux-x86-64.so.2' >>>>>>>>> SIM elf with dynamic libs archive in nuttx.tgz >>>>>>>>> ligd@BAI:~/platform/mainline/nuttx$ >>>>>>>>> ligd@BAI:~/platform/mainline/nuttx$ >>>>>>>>> ligd@BAI:~/platform/mainline/nuttx$ >>>>>>>>> *ligd@BAI:~/platform/mainline/nuttx$ ./nuttx * >>>>>>>>> CHelloWorld: Constructor: mSecret=42 >>>>>>>>> nsh: init: open failed: 2 >>>>>>>>> >>>>>>>>> NuttShell (NSH) NuttX-12.11.0 >>>>>>>>> MOTD: username=admin password=Administrator >>>>>>>>> nsh> >>>>>>>>> nsh> help >>>>>>>>> help usage: help [-v] [<cmd>] >>>>>>>>> >>>>>>>>> . df ifconfig mkfifo readlink >>>>>> umount >>>>>>>>> [ dmesg ifdown mkrd rm >>>>> unset >>>>>>>>> ? echo ifup mount rmdir >>>>>> uptime >>>>>>>>> alias env insmod mv rmmod >>>>>> usleep >>>>>>>>> unalias exec kill nslookup set >>>>> watch >>>>>>>>> basename exit pkill pidof sleep >>>> xd >>>>>>>>> break expr losetup pmconfig source >>>>> wait >>>>>>>>> cat false ln poweroff test >>>>>>>>> cd fdinfo ls quit time >>>>>>>>> cp free lsmod printf true >>>>>>>>> cmp help mkdir ps truncate >>>>>>>>> dirname hexdump mkfatfs pwd uname >>>>>>>>> >>>>>>>>> Builtin Apps: >>>>>>>>> cmocka >>>>>>>>> cmocka_driver_block >>>>>>>>> cmocka_driver_gpio >>>>>>>>> cmocka_driver_pm >>>>>>>>> cmocka_driver_pm_runtime >>>>>>>>> cmocka_driver_regulator >>>>>>>>> >>>>>>>>> Matteo Golin <[email protected]> 于2025年12月22日周一 >> 10:15写道： >>>>>>>>> >>>>>>>>>> I think so. Best to leave OS test alone and slowly replace >> it >>>>> with >>>>>> a >>>>>>>> new >>>>>>>>>> tool that has improvements. The only reason I can see to >>> modify >>>>> OS >>>>>>> test >>>>>>>>> now >>>>>>>>>> would just be to have clearer output about what >>> passed/failed. >>>>>>>>>> >>>>>>>>>> I'll come up with an improved proof of concept for a new >> tool >>>> to >>>>>>> share >>>>>>>> in >>>>>>>>>> the next few days. >>>>>>>>>> >>>>>>>>>> Matteo >>>>>>>>>> >>>>>>>>>> On Sun, Dec 21, 2025, 9:13 PM Tomek CEDRO < >> [email protected]> >>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Small steps with measurable results may be the best >>> approach >>>>>> here, >>>>>>> to >>>>>>>>>>> create some new solution next to existing ostest but not >>>>>> replacing >>>>>>> it >>>>>>>>>>> until ready? :-) >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info >>>>>>>>>>> >>>>>>>>>>> On Mon, Dec 22, 2025 at 3:06 AM Matteo Golin < >>>>>>> [email protected] >>>>>>>>> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> I agree, I think it would take a long time to have >> proper >>>>>>> coverage >>>>>>>> of >>>>>>>>>>> NuttX >>>>>>>>>>>> as a whole. Perhaps even impossible, since Unity is >> more >>>> of a >>>>>>> unit >>>>>>>>>>> testing >>>>>>>>>>>> framework and thus there are definitely test cases for >>>> NuttX >>>>>> that >>>>>>>>> would >>>>>>>>>>> be >>>>>>>>>>>> outside of its realm of "applicability". There is >> still a >>>>> need >>>>>>> for >>>>>>>> a >>>>>>>>>>>> variety of testing applications and bench-marking >>>>> applications >>>>>>> for >>>>>>>>>> those >>>>>>>>>>>> scenarios. >>>>>>>>>>>> >>>>>>>>>>>> That makes sense, prerequisites would need to be >>>> considered. >>>>> I >>>>>>>> think >>>>>>>>>>>> targeting the low-hanging fruit of unit-testable code >>> would >>>>> be >>>>>>> easy >>>>>>>>> in >>>>>>>>>>> that >>>>>>>>>>>> case, since prerequisites are mostly encoded in the >>> Kconfig >>>>>>>>> definitions >>>>>>>>>>>> existing or not for features to be tested. Any complex >>>>>> behaviour >>>>>>>>>>>> (interrupts, multi-threading, etc.) might be out of the >>>> realm >>>>>> of >>>>>>>>> what's >>>>>>>>>>>> testable by Unity in the first place. We'll always have >>> OS >>>>> test >>>>>>> in >>>>>>>>> that >>>>>>>>>>>> case, but maybe then it's better to do what Alan >>> suggested >>>>> and >>>>>> at >>>>>>>>> least >>>>>>>>>>>> make its output more clear in the meantime. I suppose >>> RTOS >>>>>>> testing >>>>>>>>> is a >>>>>>>>>>>> whole field in itself. >>>>>>>>>>>> >>>>>>>>>>>> On Sun, Dec 21, 2025 at 8:40 PM Gregory Nutt < >>>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I think that the answer depends on a tradeoff between >>>>> effort >>>>>>> and >>>>>>>>> test >>>>>>>>>>>>> coverage. I could imagine that a proper job could be >>>>>> man-years >>>>>>>> of >>>>>>>>>>> effort. >>>>>>>>>>>>> >>>>>>>>>>>>> My only concern with a re-design is to be certain >> that >>>>> tests >>>>>>> are >>>>>>>>>>>>> arranged so that the prerequisites of later test are >>>> tested >>>>>>>> first. >>>>>>>>>> If >>>>>>>>>>>>> tests are grouped by, for example, functional area, >>> then >>>>>>> failures >>>>>>>>> can >>>>>>>>>>> be >>>>>>>>>>>>> difficult to understand if there is a failure in an >>>>>> unverified >>>>>>>> test >>>>>>>>>>>>> prerequisite. >>>>>>>>>>>>> >>>>>>>>>>>>> Some of the "tests" are bogus and only exist to >> provide >>>>> some >>>>>>>>> minimum >>>>>>>>>>>>> confidence that we have the functionality to run the >>>> test. >>>>>> For >>>>>>>>>>>>> example: Can I start tasks? Can I pass parameters? >>> Can >>>> I >>>>>>>> survive >>>>>>>>> a >>>>>>>>>>>>> context switch. Some of the tests were intended only >>> to >>>>>>> support >>>>>>>>>> debug. >>>>>>>>>>>>> Some were created only to track down specific bugs. >>>>>>>>>>>>> >>>>>>>>>>>>> On 12/21/25 18:39, Matteo Golin wrote: >>>>>>>>>>>>>> LTS being the Long-Term-Stable release of NuttX? >>>>>>>>>>>>>> >>>>>>>>>>>>>> I'm starting to think that it would be best to >> create >>>> new >>>>>>> suite >>>>>>>>> of >>>>>>>>>>> tests >>>>>>>>>>>>>> using Unity, using what's in OS Test as a basis and >>>> then >>>>>>>>> expanding >>>>>>>>>> it >>>>>>>>>>>>>> further. Greg has pointed out that OS test is not >>>>> intended >>>>>> to >>>>>>>> be >>>>>>>>> a >>>>>>>>>>>>> complete >>>>>>>>>>>>>> test, so maybe a more solid testing framework base >>> in a >>>>> new >>>>>>>> form >>>>>>>>>>> would be >>>>>>>>>>>>>> in order. At that point it would also be good to >>>> document >>>>>> and >>>>>>>>>> educate >>>>>>>>>>>>> users >>>>>>>>>>>>>> about other test options, since OS test is a >> current >>>>>>> favourite >>>>>>>>> for >>>>>>>>>>>>> showing >>>>>>>>>>>>>> PRs haven't broken anything. Maybe that's not the >>> best >>>>>> choice >>>>>>>>>> anymore >>>>>>>>>>>>> with >>>>>>>>>>>>>> this information. >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't want to re-invent the wheel of what's >> already >>>> in >>>>>>>>>> `testsuite`, >>>>>>>>>>>>> but I >>>>>>>>>>>>>> think Unity might end up being lighter-weight in >> the >>>> long >>>>>>> run? >>>>>>>>> And >>>>>>>>>>> it is >>>>>>>>>>>>>> also not a Xiaomi owned/maintained framework so it >>> may >>>>>>> receive >>>>>>>>>> better >>>>>>>>>>>>>> external support (i.e. more users rely on it?). I >>> tried >>>>> to >>>>>>>>> compile >>>>>>>>>>> the >>>>>>>>>>>>>> sim:citest configuration to run the testsuite >> example >>>> and >>>>>> see >>>>>>>>> what >>>>>>>>>>> it was >>>>>>>>>>>>>> like, but I was met with a long list of C++ >>> compilation >>>>>>> errors >>>>>>>>> that >>>>>>>>>>> I'd >>>>>>>>>>>>>> rather not deal with. There also seems to be zero >>> docs >>>>>> about >>>>>>> it >>>>>>>>> or >>>>>>>>>>> cmocka >>>>>>>>>>>>>> in our NuttX docs at the moment. >>>>>>>>>>>>>> >>>>>>>>>>>>>> What do you think? Start a Unity test suite, add >> some >>>>>>> switches >>>>>>>> to >>>>>>>>>>> choose >>>>>>>>>>>>>> which test cases get compiled into the binary, open >>> for >>>>>>>>> extensions >>>>>>>>>> by >>>>>>>>>>>>> users >>>>>>>>>>>>>> who know more about different subsystems? This way >> we >>>> can >>>>>>> keep >>>>>>>> OS >>>>>>>>>>> test >>>>>>>>>>>>> how >>>>>>>>>>>>>> it is for a quick, low-resource sanity check of >>> "board >>>>>>> bringup" >>>>>>>>>> while >>>>>>>>>>>>>> having a dedicated testing application for patches? >>>>> Someone >>>>>>>>>> changing >>>>>>>>>>> file >>>>>>>>>>>>>> system stuff can just compile to include file >> system >>>> test >>>>>>> cases >>>>>>>>> and >>>>>>>>>>> run >>>>>>>>>>>>>> that suite only as proof that their PR works. The >>>> output >>>>> is >>>>>>>>> really >>>>>>>>>>> easy >>>>>>>>>>>>> to >>>>>>>>>>>>>> glance at to determine pass/fail results. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Anyways, here's my draft demonstrating how Unity >>> works >>>>> with >>>>>>> the >>>>>>>>> FPU >>>>>>>>>>> test >>>>>>>>>>>>>> suite from OS test: >>>>>>>>> https://github.com/apache/nuttx-apps/pull/3259 >>>>>>>>>>>>>> >>>>>>>>>>>>>> Matteo >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Sun, Dec 21, 2025 at 7:13 PM Tomek CEDRO < >>>>>>> [email protected]> >>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Mon, Dec 22, 2025 at 1:02 AM Gregory Nutt < >>>>>>>>> [email protected] >>>>>>>>>>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> (..) >>>>>>>>>>>>>>>> Starting with bits and pieces of something like >> the >>>> LTS >>>>>>> might >>>>>>>>>> make >>>>>>>>>>> more >>>>>>>>>>>>>>>> sense other than its licensing problems. Since >> it >>>>> would >>>>>>>> never >>>>>>>>> be >>>>>>>>>>>>>>>> included in a distributed binary, the licensing >>>> really >>>>>>> should >>>>>>>>> not >>>>>>>>>>>>> matter >>>>>>>>>>>>>>>> much in practice. >>>>>>>>>>>>>>> Unless its GPL and some company would like to have >>>> their >>>>>> own >>>>>>>>>>> internal >>>>>>>>>>>>>>> bits and pieces.. luckily its MIT and the whole >>> thing >>>> is >>>>>>>>>> open-source >>>>>>>>>>>>>>> anyways :-) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Yes we had many discussions about the LTS and this >>>> could >>>>>> be >>>>>>>>> great >>>>>>>>>>> start! >>>>>>>>>>>>>>> :-) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> CeDeROM, SQ7MHZ, http://www.tomek.cedro.info >>>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>
