It also appears that "testsuites" ( https://github.com/apache/nuttx-apps/tree/c4da1b5b9f3c6ab24014f919320bad8a5ee845d7/testing/testsuites) uses cmocka. I've never actually seen these tests run for PRs but they look quite comprehensive? Maybe it is better to be consistent and use cmocka instead of Unity, although I still think Unity is much more lightweight.
How come we haven't added more to the testsuites? Such as the new hrtimer tests? Are these for something specific only? I might be out of the loop. Matteo On Sun, Dec 21, 2025 at 5:34 PM Matteo Golin <[email protected]> wrote: > Okay, after a limited time looking at the cmocka documentation > <https://api.cmocka.org/index.html#main-features> to compare it against > Unity, I have determined the following: > > - cmocka is more geared towards mocking interfaces/C objects > - cmocka seems to be less light-weight, with a lot of abstractions for > mocking and the ability to output test results in a bunch of formats, > including XML and JUnit > > I think Unity is probably the framework to move forward with in this case > since it's lighter-weight, and I don't think our test cases in OS test > would really benefit from the mocking features of cmocka. It seems more > geared towards a simplistic approach to unit testing, and I think it would > also be more flexible. I agree though that whatever proof of concept I put > forward should have some memory usage comparison with writing the test-case > without the Unity tools. If there is too significant overhead we might need > a different solution. > > By the way, for anyone interested, the Unity framework has this basic > webpage with some information: https://www.throwtheswitch.org/unity > Most of their docs are on the GitHub page though: > https://github.com/ThrowTheSwitch/Unity > > Matteo > > On Sun, Dec 21, 2025 at 4:43 PM Matteo Golin <[email protected]> > wrote: > >> I don't think Unity should take much extra memory to run, since most of >> the test infrastructure is very simple assertions/console output macros, >> but I agree that I will need to compare the memory usage. I don't want to >> compromise any flexibility of OS test since it is meant to run on all of >> the boards (and new, low memory devices for board port tests). >> >> Matteo >> >> On Sun, Dec 21, 2025, 4:32 PM Alan C. Assis <[email protected]> wrote: >> >>> Hi Matteo, >>> that is a great idea! This is something that I always missed on ostest: >>> just say which test it is executing and if it passed or failed (just like >>> the CONFIG_TESTING_LTP does). >>> >>> Although initially using Unity seems cool, I think it could require more >>> memory to get it running inside a microcontroller with few KBs of >>> RAM/Flash >>> where ostest currently can run (search for ostest board profile inside >>> boards to see some of those). >>> >>> Maybe since we know the expected result, we just need to include a simple >>> comparison test and print the result as FAILED or PASSED (of course, some >>> tests are more complex). >>> >>> Or maybe Unity will not require much memory as I'm thinking. So, it is >>> important to get these numbers to make the right decision. >>> >>> BR, >>> >>> Alan >>> >>> On Sun, Dec 21, 2025 at 4:38 PM Matteo Golin <[email protected]> >>> wrote: >>> >>> > Hello everyone, >>> > >>> > I wanted to get some feedback on my proposal for re-factor of OS test: >>> > https://github.com/apache/nuttx-apps/issues/3258 >>> > >>> > Since it's the holidays and I have some time on my hands, I figured >>> that >>> > OS test could use some improvements, especially >>> > since it is the primary test used to check for regressions/correct >>> > functionality of kernel logic on NuttX. >>> > >>> > My proposal basically aims to use the Unity test framework to organize >>> all >>> > of the existing tests in OS test so that >>> > there is a logical code structure, and also logical console output >>> which >>> > makes it much clearer to tell if a test has >>> > passed/failed. The goal is that a) this should make reviewing test >>> results >>> > much easier in PRs and b) the improved code >>> > readability/extendability should make it easier for contributors to >>> > identify gaps in the testing and also add their own >>> > suggested tests. >>> > >>> > I have used the Unity support on NuttX for embedded testing for >>> rocketry >>> > and really enjoyed it; I think it would be a >>> > huge improvement for NuttX. >>> > >>> > If this refactor is accepted by the community, once it is done I plan >>> on >>> > completing more documentation for the OS test >>> > on the NuttX website. There is currently a lack of information about >>> all >>> > of the test suites that exist for the OS and >>> > what exactly is being verified. >>> > >>> > Please let me know what you think! >>> > >>> > -- >>> > Matteo Golin >>> > >>> >>
