On 14/06/2023 14:15:54+0200, Alexis Lothoré wrote: > On 6/14/23 12:31, Richard Purdie wrote: > > On Wed, 2023-06-14 at 10:56 +0200, Alexis Lothoré via > > lists.yoctoproject.org wrote: > >> From: Alexis Lothoré <[email protected]> > >> > >> This series is a follow-up for the 4.3_M1.rc1 regression report issue. > >> > >> It has been observed that the report is empty. This issue is linked to > >> configuration description in yocto-autobuilder-helper, and has been > >> identified through the following steps: > >> - empty report is supposed to be a comparison between yocto-4.2 (4.2.rc3) > >> and 4.3_M1.rc1 > >> - yocto-4.2 results are almost empty: we only find test results from Intel > >> QA (pushed _after_ the AB build) and not the AB test results > >> - tests results are managed by send-qa-email.send-qa-email uses resulttool > >> to systematically gather and store test results in local git directory > >> - however, it looks for basebranch/comparebranch to know if those results > >> can be pushed onto git server, and those variables depend on config.json > >> content > >> - yocto-4.2 (4.2.rc3) has been built on release branch mickledore > >> (https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/5212) > >> - since mickledore is not yet described in config.json, send-qa-email > >> considers it as a "work" branch (contrary to a "release" branch) and does > >> not push test results > >> > >> As a consequence: > >> - first commit brings in python logger > >> - second commit adds a warning when such case happen, since we are able to > >> detect it > >> - third fix actually adds mickledore as a release branch to properly store > >> again test results > >> > >> There must be a more robust rework to do (because the issue will likely > >> happen on each major delivery), but I aimed for the quick and small fix to > >> quickly bring back tests results storage without breaking other things in > >> the process > >> > >> Alexis Lothoré (3): > >> scripts/send-qa-email: use logger instead of raw prints > >> scripts/send-qa-email: print warning when test results are not stored > >> config.json: add mickledore as direct push branch for test results > > > > Thanks for the analysis. I agree we need to somehow fix this properly. > > One solution might be to always push for poky if the branch name > > doesn't end with -next? > > That might work indeed. If we are sure enough that no custom/feature branch > will > be used in poky with send-qa-email (ie, only in poky-contrib), I can do the > fix > this way
I sometimes use a different branch name when testing things out (like 64 bit time) but as long as we all know, we can probably ensure this ends in -next. > > > > Since we have the release artefacts for the release, could we add the > > test results after the fact now?> > > Id' be interested to see the 4.3 M1 to 4.2 comparison rerun with that > > added. > > I am not sure about where to find those artifacts for yocto-4.2 ? If you are > referring to https://autobuilder.yocto.io/pub/, yocto-4.2 has already been > removed from there. And if you are referring to the archived release on main > site > (https://downloads.yoctoproject.org/releases/yocto/yocto-4.2/poky-21790e71d55f417f27cd51fae9dd47549758d4a0.tar.bz2), > it does contain a single, 40 line testresults.json, so that's definitely not > the > full AB tests results. > > > > > Cheers, > > > > Richard > > > > > > > > > > > > -- > Alexis Lothoré, Bootlin > Embedded Linux and Kernel engineering > https://bootlin.com > -- Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60303): https://lists.yoctoproject.org/g/yocto/message/60303 Mute This Topic: https://lists.yoctoproject.org/mt/99523809/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
