btashton commented on issue #3216: URL: https://github.com/apache/incubator-nuttx/issues/3216#issuecomment-809027877
> > I have a PoC that I started validating the next release against that uses LabGrid (https://labgrid.readthedocs.io/en/latest/index.html) to run the sim and qemu tests. That is pytest driven for the validating. It is really basic, but since I can now grab the build artifacts from CI it is actually fairly quick to run. > > Does this mean that the test job would not run here? I'm not familiar with LabGrid. > In my mind I would add a "test" step in CI, running some predefined tests after each sim build of interest. > For things that do not require actual hardware it can just be run as a regular pytest. What is really nice is that you can also use it to interact with real hardware including via openocd. So I could see a future where some of it happens in the normal CI run and some of it just is part of a nightly run or something that calls out to a physical test rack. > > One thing that I noticed doing this is it would be really nice if nsh would report the return code for the last command much like bash can with `echo $?`. > > If not already, maybe we can make the sim exit with nsh's exit status, so that one can simply run nuttx sim binary and have the test run. Maybe it is also possible to pipe commands to nuttx binary, like: `echo "ostest; poweroff $?" | nuttx/nuttx`. This can be useful for more involved tests which require a series of commands, such as putting a net interface app, etc. I mean what I have setup is that I just start the nuttx binary and interact with it like it is a shell. What I was getting at is it would be be nice if in the shell you could query the exit code rather than having to write a special parser for every command. `nsh> ostest; echo "pass: $?"` > > > If there is interest I could put this up on the testing repo or something? I was going to wait until I ironed some of the issues out on this release, but I guess there is no real reason to wait. > > Whatever brings us closer to this being enabled is welcomed IMHO Alright I'll put something up this week that at least does some of the basic execution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
