On 05/05/2016 09:10, Michal Hrusecky wrote: > John Crispin - 8:56 5.05.16 wrote: >> Hi, >> >> It would be nice if there was a process in place that would allow >> community members to easily be able to test images on devices and report >> the results some place. after some time thinking of different ways to do >> this I came up with one possible solution and wanted to know what others >> think about this. >> >> we setup a file similar to how the MAINTAINERS file in the kernel. >> anyone can add his name/email and boards he would like to be a tester >> of. every time an image is built for said board, an email is generated >> and sent out to the according persons. this email would contains an otp. >> with this credential you could then log into some web frontend and get a >> simple mask along the lines of >> >> * wifi worked >> * ethernet worked >> * leds worked >> * buttons worked >> * iperf >> * ssl benchmark >> ... >> >> this could all be done in a rather trivial manner and i dont think the >> work behind such a setup is really that huge. using otps would for >> example eliminate the need for user credential management. results could >> simply be stored in files on the backend and then harvested later by a >> secondary script etc etc... >> >> next it would be possible to generate static content based on aggregated >> data that will show a traffic light style support status for various >> boards, listing when it was last tested, with what revision and what the >> test results were. >> >> would something like this make sense ? > > It kinda makes sense, although it would be even nicer if the tests were > automated as well. That is something that Purple is working on. There is a > BoardFarm[1] test framework that is designed to test routers and there could > be > created test suites that reflects some of the criteria you described and in > the > best case, whenever image is produced, automatic test could be triggered and > results uploaded. It outputs junit and there is plenty of parsers of junit - > we > have it nicely integrated in Jenkins. > > [1] https://github.com/qca/boardfarm
i doubt they will test the 900+ boards. i would expect them to test their own boards. additionally the proposal does not tell you how to test. it describes a way to notify when testing is required and where the images are located. i think you misread/understood the purpose of what i proposed or i worded it badly. John _______________________________________________ Lede-dev mailing list Lede-dev@lists.infradead.org http://lists.infradead.org/mailman/listinfo/lede-dev