or starting to implement a monster that only prevents anyone from changing. "No change anymore" can be implemented easier, just stop working and go on vacation, haha. But no joke, that's also a way to send a project to death.
-- Hard- and Softwaredevelopment Consultant Geschäftsführung: Simon Filgis USt-IdNr.: DE305343278 ISO9001:2015 <https://activities.ingenieurbuero-filgis.de/certifications> On Wed, Feb 5, 2025 at 2:39 PM Simon Filgis <si...@ingenieurbuero-filgis.de> wrote: > Dear all, > > each arch, driver and app tested once would be enough I think. A matrix > can help to identify test-gaps. Double testing is nice and triple testing > is not of benefit any more. > > The goal should be to have fast access to results with transparency. I > fear starting to maintain a useless monster ;) > > Simon > > -- > Hard- and Softwaredevelopment Consultant > > Geschäftsführung: Simon Filgis > USt-IdNr.: DE305343278 > ISO9001:2015 <https://activities.ingenieurbuero-filgis.de/certifications> > > > On Wed, Feb 5, 2025 at 1:27 PM Alan C. Assis <acas...@gmail.com> wrote: > >> I think we should test the most complete/complex boards from each arch. It >> will cover most of the issues that could after other boards. >> >> BR, >> >> Alan >> >> On Wed, Feb 5, 2025 at 4:23 AM raiden00pl <raiden0...@gmail.com> wrote: >> >> > As mentioned earlier, testing all boards is pointless, especially since >> the >> > project >> > has very limited resources. Choosing a few boards that will allow us to >> > test as many >> > things as possible is the most optimal approach. >> > >> > But first we should determine what things we want to test, not what >> boards. >> > Knowing what things we want to test, we can design test cases and >> possibly >> > use what is already available. >> > >> > But e-mail and github don't seem to me to be a good tool for >> brainstorming >> > and ideas. Maybe Confluence pages would be better? I haven't used >> > Confluence >> > for a few years, I just hope it's not as slow as it used to be :) >> > >> > I can create a Confluence page and describe my ideas about testing, I >> have >> > some >> > thoughts about NuttX testing written somewhere in my private notes. >> Others >> > can do >> > the same. >> > >> > Email can be good for decision making and maybe gathering more feedback >> > from >> > the community, but it's a shitty tool for more complex work. Or I'm too >> > young >> > to use it comfortably :P >> > >> > wt., 4 lut 2025 o 12:03 Tomek CEDRO <to...@cedro.info> napisał(a): >> > >> > > On Tue, Feb 4, 2025 at 11:42 AM Luchian Mihai < >> luchiann.mi...@gmail.com> >> > > wrote: >> > > > Hi! >> > > > First thing, I'm fairly new to nuttx so I might be off subject but >> here >> > > is >> > > > my hot take on this subject. >> > > >> > > Welcome and have fun Mihai! :-) >> > > >> > > >> > > > NuttX is offering support for a lot of boards, more than what DRUNX >> > > should >> > > > require. >> > > > >> > > > Eg. stm32f3 family, offering support for all the boards would >> benefit >> > the >> > > > boards more than the NuttX codebase. >> > > > These boards share 80% of the code but each in its own files, so >> most >> > > > differences are due to lack of backporting. >> > > > >> > > > My suggestion is to take a single board for each mcu family as >> > mandatory >> > > > NuttX support, others as optional. >> > > > I'm not saying to drop support, or offer less support for those, >> just >> > to >> > > > treat the mandatory as higher prio. >> > > > For the moment we can choose what is mandatory, and at a later time, >> > when >> > > > DRUNX would be stable, move the optional ones to DRUNX repo (for >> > > example). >> > > >> > > The idea is that everyone can test what they have at hand and then >> > > gather the results, that should sum up to full board list one day :-) >> > > Also different people will use different build hosts, different >> > > compilers, etc, so even if the same board is tested in different build >> > > environment that can also reveal potential issues to be fixed :-) >> > > >> > > Yes, for sure we need at least one board from each family for start, >> > > then adding more boards, we all do release testing that way or >> > > another, by hand or scripted, we now have to find a way to make it >> > > distributed easy to setup fire-and-forget :-) >> > > >> > > >> > > > TLDR: I think less is more, less "official" support, more "official" >> > > > coverage. >> > > >> > > More means more. Less means less. Lets keep thing simple in this >> > > inverted world where word have no meaning anymore :D :D :D >> > > >> > > Its a small project based on voluntary work with zero financial >> > > support from big companies. We will define a testing architecture for >> > > sure, but for now its a fresh concept and each one of us try different >> > > area to create small building blocks that will give us the solution we >> > > need, one day, hopefully ;-) >> > > >> > > Please go ahead Mihai and try your boards, start with one that you >> > > know best, use PyTest to create build and runtime automation, create >> > > "selftest" board config that will run a script with `uname -a; help; >> > > ostest, coremark`, gather the logs, parse the results, see how >> > > nuttx-dashboard works based on gist, see what problems we have, see >> > > how can we solve them hands-on :-) Maybe there is something better >> > > than PyTest. Maybe there are other ways. You try it out and share the >> > > feedback :-) >> > > >> > > If this is too much, use smaller steps, think of it as automation tool >> > > for release testing on different boards :-) >> > > >> > > Thank you and take care :-) >> > > Tomek >> > > >> > > -- >> > > CeDeROM, SQ7MHZ, http://www.tomek.cedro.info >> > > >> > >> >
