On Wed, 2022-10-26 at 12:49 -0400, Sean Anderson wrote: > On 10/26/22 10:08, Richard Purdie wrote: > > This looks like a pretty nice improvement, I've been worried about how > > entangled all this was becoming. Is there anything better we could do > > with testing of this code? We rely increasingly on the tests to spot > > regressions and I doubt the testcases we have cover the code > > complexity. > > FWIW, I found the test suite to be fairly helpful when debugging this > whole process, if agonizingly slow. I found many bugs in my initial > implementation which were uncovered by the test suites, > > The best way to extend the test suite would probably be to have QEMU try > to boot using the Linux and U-Boot. This would help cases where all the > artifacts are created correctly but e.g. the wrong U-Boot is packaged. > However, I am rather loathe to add more test cases like this, because it > already takes around 10 minutes (with sstate!) to run the tests. This > basically kills the speed of iterative development.
Out of interest how were you running the tests? Did you look at the parallelism options? Were you just running specific tests or specific suites of tests? I do think we need something like you describe and adding a linux+uboot approach is something I've wanted to do for a long time. Usually the time is taken on our automated CI rather than impacting users, unless they're working in that area at which point the tests are hopefully helpful. We haven't really had anyone try and optimise the tests either, I'm sure there will be things in there which can help. Please don't let the speed put you off trying to improve things and extend our coverage! Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172164): https://lists.openembedded.org/g/openembedded-core/message/172164 Mute This Topic: https://lists.openembedded.org/mt/94487631/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
