tmedicci commented on issue #15730: URL: https://github.com/apache/nuttx/issues/15730#issuecomment-2679324130
> > [@tmedicci](https://github.com/tmedicci): Just some additional thoughts about our CI organization. Although it's recommended to keep the distributed build farm, we should avoid as many as possible bad commits from being merged upstream. To do that, we need to test every single PR. This costs a lot, so we need to make it more efficient. _How?_ By splitting the CI into more workflows prone to fail (and stop subsequent jobs). > > First, we build the most complete `defconfig` for each chip (or board). Then, we test it (runtime testing). After that, we can continue and build all the other `defconfigs` (and, eventually, test some of these configs on QEMU and/or real HW). > > Let's use our GH runners to build the firmware and run the QEMU testing. If QEMU testing is successful, we can even use self-hosted runners to test the HW (see, the security concerns here are mitigated as it'd only be tested after QEMU). > > I created a simple diagram about what I think should be our optimal CI in the future: > >  > > It doesn't matter that much if It takes 3 or more hours to run the complete CI as long as it fails as soon as it detects a failure. Is this possible? I don't know, we have to make it step-by-step. My current proposal is implementing the following: > >  > > And evaluate how much GH runner's usage we'd save by doing that... > > Sounds cool :-) Go for it in sync with [@lupyuen](https://github.com/lupyuen) as he knows most about current CI setup and functions :-) The sooner error can be detected and CI does not consume resources unnecessary the better, we almost lost CI on GitHub due to over use [@lupyuen](https://github.com/lupyuen) saved us from the doom. Also each push to the PR triggers all of the builds again and again.. so yes catching errors as soon as possible is more than welcome :-) > > Btw what tool do you use to make that nice charts [@tmedicci](https://github.com/tmedicci) ? :-) @lupyuen before making a prototype, what do you think about it? Are these major goals achievable? Any considerations about it? I really would like to have self-hosted GH runners for HW testing (not building) in the future (considering that we have the code tested on QEMU) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@nuttx.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
