Cmocka tests seems to be mostly used and developed by Xiaomi. Maybe they use it internally only for runtime tests aisde from ostest? No clue too, I haven't played that yet :-)
Last question about Unity - it uses MIT license so there is no conflict with Apache and we can include that code safely in the NuttX Apps code base without special switches? -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info On Sun, Dec 21, 2025 at 11:44 PM Matteo Golin <[email protected]> wrote: > > It also appears that "testsuites" ( > https://github.com/apache/nuttx-apps/tree/c4da1b5b9f3c6ab24014f919320bad8a5ee845d7/testing/testsuites) > uses cmocka. I've never actually seen these tests run for PRs but they look > quite comprehensive? Maybe it is better to be consistent and use cmocka > instead of Unity, although I still think Unity is much more lightweight. > > How come we haven't added more to the testsuites? Such as the new hrtimer > tests? Are these for something specific only? I might be out of the loop. > > Matteo > > On Sun, Dec 21, 2025 at 5:34 PM Matteo Golin <[email protected]> wrote: > > > Okay, after a limited time looking at the cmocka documentation > > <https://api.cmocka.org/index.html#main-features> to compare it against > > Unity, I have determined the following: > > > > - cmocka is more geared towards mocking interfaces/C objects > > - cmocka seems to be less light-weight, with a lot of abstractions for > > mocking and the ability to output test results in a bunch of formats, > > including XML and JUnit > > > > I think Unity is probably the framework to move forward with in this case > > since it's lighter-weight, and I don't think our test cases in OS test > > would really benefit from the mocking features of cmocka. It seems more > > geared towards a simplistic approach to unit testing, and I think it would > > also be more flexible. I agree though that whatever proof of concept I put > > forward should have some memory usage comparison with writing the test-case > > without the Unity tools. If there is too significant overhead we might need > > a different solution. > > > > By the way, for anyone interested, the Unity framework has this basic > > webpage with some information: https://www.throwtheswitch.org/unity > > Most of their docs are on the GitHub page though: > > https://github.com/ThrowTheSwitch/Unity > > > > Matteo > > > > On Sun, Dec 21, 2025 at 4:43 PM Matteo Golin <[email protected]> > > wrote: > > > >> I don't think Unity should take much extra memory to run, since most of > >> the test infrastructure is very simple assertions/console output macros, > >> but I agree that I will need to compare the memory usage. I don't want to > >> compromise any flexibility of OS test since it is meant to run on all of > >> the boards (and new, low memory devices for board port tests). > >> > >> Matteo > >> > >> On Sun, Dec 21, 2025, 4:32 PM Alan C. Assis <[email protected]> wrote: > >> > >>> Hi Matteo, > >>> that is a great idea! This is something that I always missed on ostest: > >>> just say which test it is executing and if it passed or failed (just like > >>> the CONFIG_TESTING_LTP does). > >>> > >>> Although initially using Unity seems cool, I think it could require more > >>> memory to get it running inside a microcontroller with few KBs of > >>> RAM/Flash > >>> where ostest currently can run (search for ostest board profile inside > >>> boards to see some of those). > >>> > >>> Maybe since we know the expected result, we just need to include a simple > >>> comparison test and print the result as FAILED or PASSED (of course, some > >>> tests are more complex). > >>> > >>> Or maybe Unity will not require much memory as I'm thinking. So, it is > >>> important to get these numbers to make the right decision. > >>> > >>> BR, > >>> > >>> Alan > >>> > >>> On Sun, Dec 21, 2025 at 4:38 PM Matteo Golin <[email protected]> > >>> wrote: > >>> > >>> > Hello everyone, > >>> > > >>> > I wanted to get some feedback on my proposal for re-factor of OS test: > >>> > https://github.com/apache/nuttx-apps/issues/3258 > >>> > > >>> > Since it's the holidays and I have some time on my hands, I figured > >>> that > >>> > OS test could use some improvements, especially > >>> > since it is the primary test used to check for regressions/correct > >>> > functionality of kernel logic on NuttX. > >>> > > >>> > My proposal basically aims to use the Unity test framework to organize > >>> all > >>> > of the existing tests in OS test so that > >>> > there is a logical code structure, and also logical console output > >>> which > >>> > makes it much clearer to tell if a test has > >>> > passed/failed. The goal is that a) this should make reviewing test > >>> results > >>> > much easier in PRs and b) the improved code > >>> > readability/extendability should make it easier for contributors to > >>> > identify gaps in the testing and also add their own > >>> > suggested tests. > >>> > > >>> > I have used the Unity support on NuttX for embedded testing for > >>> rocketry > >>> > and really enjoyed it; I think it would be a > >>> > huge improvement for NuttX. > >>> > > >>> > If this refactor is accepted by the community, once it is done I plan > >>> on > >>> > completing more documentation for the OS test > >>> > on the NuttX website. There is currently a lack of information about > >>> all > >>> > of the test suites that exist for the OS and > >>> > what exactly is being verified. > >>> > > >>> > Please let me know what you think! > >>> > > >>> > -- > >>> > Matteo Golin > >>> > > >>> > >>
