On Fri, Aug 9, 2024 at 3:00 PM Nathan Hartman <hartman.nat...@gmail.com> wrote: > I like Tomek's idea of testing Tiers, with Tier0 being "critical tests" > that MUST pass, on all architectures and boards. > > I pike Alin's idea of an automatically updated test status report form, > which can be viewed on GitHub. > > I recommend to list ALL supported boards in the report form, with "Need > Test Hardware" (or something like that) shown for boards that are not > included in any HW test cluster. The idea is to help find volunteers to > test those boards. > > What happens if multiple people have the same board in their test clusters? > And what happens if tests pass for one person but fail for another, on the > same board model? We might need to be able to show: Pass, Fail, Need Test > Hardware, or a percentage, like 2/3 (passing on 2 instances out of 3). That > might indicate bugs that are sensitive to timing.
Thanks Nathan :-) I am sure we can create something useful and helpful and I really like all those constructive "Think Before You Do"^TM discussions here on the NuttX Dev mailing list :-) :-) For CI testing on the GitHub I guess we are limited to build and simulator / qemu only. Distributed testing would be perfect so anyone could just launch a script with some real world boards attached then gathered results could be uploaded into a central point with total summary tests analysis. That way we could cover almost any hardware supported by NuttX because "someone somewhere should have that board" :-) Also it would be prone to single point of failure (except the central point). What a great idea! :-) But we should measure strengths for goals, with ENOTIME and ENOCASH throwing all around, lets start from something simple that just works does the job we need and allow extensions in future :-) Some design considerations right now could make that project elegant too :-) Alan mentioned USB Hub - I have popular and not that expensive (~50EUR) iTec USB 3.0 16-port 10W max with external power supply and manual power switch at every port (PN:U3CHARGEHUB16). Did not have time to use it yet.. and did not know that single hub ports also could be controlled from a PC (I know FreeBSD USB stack can reset and power off given device but that device first needs to be connected to a port). Good hint thanks Alan :-) Have a good weekend folks! :-) -- CeDeROM, SQ7MHZ, http://www.tomek.cedro.info
