HI all , ostest is a nice first step on validating our RTOS but I think that having a distributed testing farm on real HW will help us scale and diversify the testing
My proposal is 1. have a build machine that builds every day all configs and provides the binaries for download so that they can be validated by the test clusters 2. each test cluster, configured for available HW in that cluster, downloads the corresponding binaries, flashes, tests and reports back the status for the tested boards on github 3. on our main site we link the github status report form (table with all boards/configs and the status for the last build) What do you think about this approach? Best regards Alin On Fri, Aug 9, 2024 at 3:42 AM Nathan Hartman <hartman.nat...@gmail.com> wrote: > On Thu, Aug 8, 2024 at 8:58 PM Gregory Nutt <spudan...@gmail.com> wrote: > > > > > On 8/8/2024 6:48 PM, Nathan Hartman wrote: > > > A dedicated "citest" program is a good idea! > > I think that ostest could meet all of the needs of a "citest" program. > > I just needs more control of the verbosity and format of the output. It > > already meets 95% of the requirement. > > > > Ok so perhaps it could use a command line arg to instruct it how much > output to produce. Many unix programs have the convention of --quiet for no > output, --verbose for full output, and by default only necessary messages. > But a CI test might need different output altogether, since it needs to be > compared somehow. So, maybe we need a --ci argument that puts the output in > a format suitable for automatic CI testing. My thinking is to provide one > mode for CI and another (more user friendly) mode for manual testing. I > think that's needed because if the CI tests fail, then we would likely want > to run it manually and see what isn't working. > > Just a thought. > > Cheers > Nathan >
