>-----Original Message-----
>From: [email protected] <[email protected]>
>Sent: Tuesday, 23 April, 2019 10:31 PM
>To: Jain, Sangeeta <[email protected]>; '[email protected]'
><[email protected]>; Eggleton, Paul <[email protected]>; 'Michael
>Halstead' <[email protected]>; Erway, Tracey M
><[email protected]>; '[email protected]'
><[email protected]>; 'openembedded-core-
>[email protected]' <openembedded-core-
>[email protected]>
>Cc: Sangal, Apoorv <[email protected]>; Yeoh, Ee Peng
><[email protected]>; Graydon, Tracy <[email protected]>;
>Tummalapalli, Vineela <[email protected]>
>Subject: Re: QA cycle report for 2.5.3 RC3
>
>On Thu, 2019-04-18 at 04:58 +0000, Jain, Sangeeta wrote:
>>
>> QA cycle report for 2.5.3 RC3:
>>
>> No high milestone defects.
>> Test results are available at following location:
>> · For results of all automated tests, refer to results at
>> public AB [1].
>> · For other test results, refer to attachment [2].
>> · For test report for test cases run by Intel and WR team,
>> refer attachment [3]
>> · For full test report, refer attachment [4]
>> · For ptest results, please refer to results at public AB [5]
>> · For ptest report, refer to attachment [6]
>> No new defects are found in this cycle.
>> Number of existing issues observed in this release is 2- toaster [7]
>> and Build-appliance [8] For ptest, regression data is not available
>> for this release. No timeout issues.
>> Test result report on Public AB shows no failures.
>
>I've been discussing with Tracy and Vineela how to best handle the test
>results for
>the release for 2.5.3. In the end I:
>
>a) Copied in the ptest results to the right place. This will happen
>automagically in
>all future builds:
>
>https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults/qemux86-64-
>ptest/
>
>b) Copied in the intel test results to their own directory:
>
>https://autobuilder.yocto.io/pub/releases/yocto-2.5.3/testresults-intel/
>
>c) Manually generated an updated test report for the combined results with the
>commands:
>
>$ cd /srv/autobuilder/autobuilder.yoctoproject.org/pub/releases/yocto-2.5.3
>$resulttool report . > testreport.txt
>
>d) Added a header to the report which consisted of the report from QA with
>details of the bugs etc.
>
>This means we have a top level testreport file which contains all the test
>information about the release.
>
>I'd like to make this the standard procedure for release. The release notes and
>announcement can refer to the test report included with the release and the
>test
>results as its there all together.
>
>Ultimately I'd like to improve the formatting of the report (separate out ptest
>regressions from the other regressions, maybe html, maybe graphs, better
>regression information) but that is something for the future.
>
>Does that work for everyone?
That looks great to me.
As a further improvement for sharing of test results, I am working to create a
wiki page for consolidated
test results. For 2.5.3, it can be viewed at:
https://wiki.yoctoproject.org/wiki/WW16_-_2019-04-18_-_Full_Test_Cycle_2.5.3_RC3
After you have added more results on AB, I will change some existing links on
this web page with the new ones.
Another thought I have is that since for any release, all data will be removed
from public AB after few days of release,
Should I include here a link to "yocto-testresults" git repo? It will make
this wiki page more useful for reference in future as well.
>
>Cheers,
>
>Richard
>
Thanks & Regards,
Sangeeta Jain
--
_______________________________________________
yocto mailing list
[email protected]
https://lists.yoctoproject.org/listinfo/yocto
