Okay, after a limited time looking at the cmocka documentation <https://api.cmocka.org/index.html#main-features> to compare it against Unity, I have determined the following:
- cmocka is more geared towards mocking interfaces/C objects - cmocka seems to be less light-weight, with a lot of abstractions for mocking and the ability to output test results in a bunch of formats, including XML and JUnit I think Unity is probably the framework to move forward with in this case since it's lighter-weight, and I don't think our test cases in OS test would really benefit from the mocking features of cmocka. It seems more geared towards a simplistic approach to unit testing, and I think it would also be more flexible. I agree though that whatever proof of concept I put forward should have some memory usage comparison with writing the test-case without the Unity tools. If there is too significant overhead we might need a different solution. By the way, for anyone interested, the Unity framework has this basic webpage with some information: https://www.throwtheswitch.org/unity Most of their docs are on the GitHub page though: https://github.com/ThrowTheSwitch/Unity Matteo On Sun, Dec 21, 2025 at 4:43 PM Matteo Golin <[email protected]> wrote: > I don't think Unity should take much extra memory to run, since most of > the test infrastructure is very simple assertions/console output macros, > but I agree that I will need to compare the memory usage. I don't want to > compromise any flexibility of OS test since it is meant to run on all of > the boards (and new, low memory devices for board port tests). > > Matteo > > On Sun, Dec 21, 2025, 4:32 PM Alan C. Assis <[email protected]> wrote: > >> Hi Matteo, >> that is a great idea! This is something that I always missed on ostest: >> just say which test it is executing and if it passed or failed (just like >> the CONFIG_TESTING_LTP does). >> >> Although initially using Unity seems cool, I think it could require more >> memory to get it running inside a microcontroller with few KBs of >> RAM/Flash >> where ostest currently can run (search for ostest board profile inside >> boards to see some of those). >> >> Maybe since we know the expected result, we just need to include a simple >> comparison test and print the result as FAILED or PASSED (of course, some >> tests are more complex). >> >> Or maybe Unity will not require much memory as I'm thinking. So, it is >> important to get these numbers to make the right decision. >> >> BR, >> >> Alan >> >> On Sun, Dec 21, 2025 at 4:38 PM Matteo Golin <[email protected]> >> wrote: >> >> > Hello everyone, >> > >> > I wanted to get some feedback on my proposal for re-factor of OS test: >> > https://github.com/apache/nuttx-apps/issues/3258 >> > >> > Since it's the holidays and I have some time on my hands, I figured that >> > OS test could use some improvements, especially >> > since it is the primary test used to check for regressions/correct >> > functionality of kernel logic on NuttX. >> > >> > My proposal basically aims to use the Unity test framework to organize >> all >> > of the existing tests in OS test so that >> > there is a logical code structure, and also logical console output which >> > makes it much clearer to tell if a test has >> > passed/failed. The goal is that a) this should make reviewing test >> results >> > much easier in PRs and b) the improved code >> > readability/extendability should make it easier for contributors to >> > identify gaps in the testing and also add their own >> > suggested tests. >> > >> > I have used the Unity support on NuttX for embedded testing for rocketry >> > and really enjoyed it; I think it would be a >> > huge improvement for NuttX. >> > >> > If this refactor is accepted by the community, once it is done I plan on >> > completing more documentation for the OS test >> > on the NuttX website. There is currently a lack of information about all >> > of the test suites that exist for the OS and >> > what exactly is being verified. >> > >> > Please let me know what you think! >> > >> > -- >> > Matteo Golin >> > >> >
