v01d commented on issue #3216: URL: https://github.com/apache/incubator-nuttx/issues/3216#issuecomment-809018785
> For ostest, I think a simple (p)expect script should be enough. We also have this [apache/incubator-nuttx-testing#70](https://github.com/apache/incubator-nuttx-testing/pull/70) which is still under review. Yeah, I think simply detecting the test app exiting with non-zero status, detecting any error printed to console or sim crashing would go a long way. My suggestion about eventually moving tests to unity is to adopt a standard way to write the test apps. It is quite simply actually: http://www.throwtheswitch.org/unity > I have a PoC that I started validating the next release against that uses LabGrid (https://labgrid.readthedocs.io/en/latest/index.html) to run the sim and qemu tests. That is pytest driven for the validating. It is really basic, but since I can now grab the build artifacts from CI it is actually fairly quick to run. Does this mean that the test job would not run here? I'm not familiar with LabGrid. In my mind I would add a "test" step in CI, running some predefined tests after each sim build of interest. > One thing that I noticed doing this is it would be really nice if nsh would report the return code for the last command much like bash can with `echo $?`. If not already, maybe we can make the sim exit with nsh's exit status, so that one can simply run nuttx sim binary and have the test run. Maybe it is also possible to pipe commands to nuttx binary, like: `echo "ostest; poweroff $?" | nuttx/nuttx`. This can be useful for more involved tests which require a series of commands, such as putting a net interface app, etc. > If there is interest I could put this up on the testing repo or something? I was going to wait until I ironed some of the issues out on this release, but I guess there is no real reason to wait. Whatever brings us closer to this being enabled is welcomed IMHO -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
