Re: [yocto] Autobuilder changes
On Sat, 2018-11-24 at 07:40 -0700, akuster wrote: > On 11/23/18 10:02 AM, Richard Purdie wrote: > > We now run "a-quick" on master at 1am every day which will make > > What Time zone for 1am? I'm not 100% sure but I think its UTC. > > Building older releases is now broken until I sort out the > > corresponding helper changes for those branches but one thing at a > > time! > > Does that include Thud ? I expect sumo and below to be problems. > > Do the changes need to roll from newest to oldest or are they > independent? I was hoping to get Sumo tested using the original QA > process to buy us time to get the automation working on it. Does this > changes that ? Good news is I've fixed the older releases now and run test builds to check things. There were a couple of fixes needed but I've sorted those and done a test artefact deployment too. This just updates everything to use the new builder names. The QA process is unchanged. If a given build generates a test report, whichever release it is, it will be collected up but if not that is also fine. This means we can opt in to the new code if/as/when we backport anything. So we're good for sumo. We do need to sort these gpg issues across various stable releases though and probably tweak master too. There were some test results ordering changes I'd be tempted to backport just as they make reading the build logs so much easier... Cheers, Richard -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Autobuilder changes
On 11/23/18 10:02 AM, Richard Purdie wrote: > I've take this quiet period as an opportunity to make some significant > autobuilder changes: > > The targets have all been renamed and split up. "nightly-arm" became > "qemuarm" and "beaglebone". The "nightly-" prefixes were dropped. > > The nightly target was dropped and replaced by "a-quick" and "a-full". > The difference are a number of targets which don't often find problems. > We're also ramping down on the amount of testing qemumips-lsb/qemuppc- > lsb get for example as they're more minor architectures as things > stand. The "a-" prefix is for sorting in the UI and to separate them > out from the mass of other builds. > > There are now selftest targets per "distro" which can replace the > testing that Intel QA have been doing. This is only done for "a-full" > and the selftest on a distro at random is done by "a-quick". > > Buildhistory is only being generated for the qemu* targets. > > The "template" used for testing qemu or "real-hardware" targets was > standardised as there were weird differences for now good reason. > > The structure of "non-release" builds on > https://autobuilder.yocto.io/pub/ was cleaned up and now appears under > a single date+number under the non-release directory which was how it > was meant to work. All the other spurious output that was there has > been cleaned up. > > XML test results are now being published for all builds, e.g.: > https://autobuilder.yocto.io/pub/non-release/20181122-11/qemuarm64/ > > Ptest was added to "a-full" builds for arm and x86 although the arm > builds currently time out. The plan will be to enable arm when we have > an arm build server with KVM support in our rack (next week?). ptest > execution for x86 takes around 3 hours and some sample result output > is: > > https://autobuilder.yocto.io/pub/non-release/20181122-11/qemux86-64-ptest/ > > "meta-oe" and "meta-virt" test targets were added for experimentation, > testing meta-openembedded and meta-virtualization respectively. These > are work in progress currently just doing a "world -k" build for > qemux86-64 which fails (they take around 2.5 hours). > > We now run "a-quick" on master at 1am every day which will make What Time zone for 1am? > buildhistory more up to date and useful for the master branch and give > us better comparison data for -next and mut. > > wine is installed onto the ubuntu1804 workers and the mingw testing was > split into a meta-mingw target, now with 32 and 64 bit build tests and > 64 bit runtime tests. With master-next of meta-mingw this appears to > run the test. Huge kudos to Joshua Watt for putting that testing > together. If we install more onto the workers we could probably test 32 > bit as well, I'm torn on that right now as its a lot more dependencies > on the workers. > > Building older releases is now broken until I sort out the > corresponding helper changes for those branches but one thing at a > time! Does that include Thud ? I expect sumo and below to be problems. Do the changes need to roll from newest to oldest or are they independent? I was hoping to get Sumo tested using the original QA process to buy us time to get the automation working on it. Does this changes that ? - armin > > If anyone would like to help with xml results file analysis tooling, > that is where we need work next to be able to summarise the ptest > results and do results comparison. I know Ee Peng has some work in this > area which I will be looking at next. > > Cheers, > > Richard > > > -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Autobuilder changes
I've take this quiet period as an opportunity to make some significant autobuilder changes: The targets have all been renamed and split up. "nightly-arm" became "qemuarm" and "beaglebone". The "nightly-" prefixes were dropped. The nightly target was dropped and replaced by "a-quick" and "a-full". The difference are a number of targets which don't often find problems. We're also ramping down on the amount of testing qemumips-lsb/qemuppc- lsb get for example as they're more minor architectures as things stand. The "a-" prefix is for sorting in the UI and to separate them out from the mass of other builds. There are now selftest targets per "distro" which can replace the testing that Intel QA have been doing. This is only done for "a-full" and the selftest on a distro at random is done by "a-quick". Buildhistory is only being generated for the qemu* targets. The "template" used for testing qemu or "real-hardware" targets was standardised as there were weird differences for now good reason. The structure of "non-release" builds on https://autobuilder.yocto.io/pub/ was cleaned up and now appears under a single date+number under the non-release directory which was how it was meant to work. All the other spurious output that was there has been cleaned up. XML test results are now being published for all builds, e.g.: https://autobuilder.yocto.io/pub/non-release/20181122-11/qemuarm64/ Ptest was added to "a-full" builds for arm and x86 although the arm builds currently time out. The plan will be to enable arm when we have an arm build server with KVM support in our rack (next week?). ptest execution for x86 takes around 3 hours and some sample result output is: https://autobuilder.yocto.io/pub/non-release/20181122-11/qemux86-64-ptest/ "meta-oe" and "meta-virt" test targets were added for experimentation, testing meta-openembedded and meta-virtualization respectively. These are work in progress currently just doing a "world -k" build for qemux86-64 which fails (they take around 2.5 hours). We now run "a-quick" on master at 1am every day which will make buildhistory more up to date and useful for the master branch and give us better comparison data for -next and mut. wine is installed onto the ubuntu1804 workers and the mingw testing was split into a meta-mingw target, now with 32 and 64 bit build tests and 64 bit runtime tests. With master-next of meta-mingw this appears to run the test. Huge kudos to Joshua Watt for putting that testing together. If we install more onto the workers we could probably test 32 bit as well, I'm torn on that right now as its a lot more dependencies on the workers. Building older releases is now broken until I sort out the corresponding helper changes for those branches but one thing at a time! If anyone would like to help with xml results file analysis tooling, that is where we need work next to be able to summarise the ptest results and do results comparison. I know Ee Peng has some work in this area which I will be looking at next. Cheers, Richard -- ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
[yocto] Autobuilder changes and Milestone2
All, This morning I stopped the autobuilder and fixed a few scripts that caused our first milestone build to not generate a source release tarball. I also removed a few unneeded sdk targets from milestone-internal and fixed the CURRENT linking issue. I'll have these all packaged up and ready to pull into the poky-autobuilder repo next week but I wanted to go through the changes before I left for the weekend (so I don't forget what I did :) ) The first change was with poky-autobuild-generate-sources-tarball. The tarball generation failed due to git trying to archive a hardcoded branch (master). This script now takes an optional third arg of $BRANCH. If the third arg isn't set, it defaults to 'master'. This should fix the source tarball generation. The second change was to poky-autobuild-generate-release. This script now takes an optional third arg of 'current', which causes it to just create a CURRENT symlink to the latest release directory. I want to remove this functionality from this script as issuing a 'current' doesn't actually generate a release and the functionality should be extracted and put in it's own script. It was just a convenient place to put it for now. I also removed all the non-IA arch sdk builds from milestone-internal. Building these happens on milestone-external and is not needed. M2 build 2 is now running and judging from the last build, milestone-external should be done around 12 noon PST tomorrow and milestone-internal finished around 8am PST. If you have any questions or concerns, I'll be checking email intermittently this weekend. -b ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Autobuilder changes and Milestone2
I also removed all the non-IA arch sdk builds from milestone-internal. Building these happens on milestone-external and is not needed. Beth, so all the non-IA arch sdk will be built for M2 but just will be built on milestone-external, right? Thanks, Jessica smime.p7s Description: S/MIME cryptographic signature ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
Re: [yocto] Autobuilder changes and Milestone2
On 12/18/2010 10:14 AM, Flanagan, Elizabeth wrote: All, This morning I stopped the autobuilder and fixed a few scripts that caused our first milestone build to not generate a source release tarball. I also removed a few unneeded sdk targets from milestone-internal and fixed the CURRENT linking issue. I'll have these all packaged up and ready to pull into the poky-autobuilder repo next week but I wanted to go through the changes before I left for the weekend (so I don't forget what I did :) ) The first change was with poky-autobuild-generate-sources-tarball. The tarball generation failed due to git trying to archive a hardcoded branch (master). This script now takes an optional third arg of $BRANCH. If the third arg isn't set, it defaults to 'master'. This should fix the source tarball generation. The second change was to poky-autobuild-generate-release. This script now takes an optional third arg of 'current', which causes it to just create a CURRENT symlink to the latest release directory. I want to remove this functionality from this script as issuing a 'current' doesn't actually generate a release and the functionality should be extracted and put in it's own script. It was just a convenient place to put it for now. I also removed all the non-IA arch sdk builds from milestone-internal. Building these happens on milestone-external and is not needed. M2 build 2 is now running and judging from the last build, milestone-external should be done around 12 noon PST tomorrow and milestone-internal finished around 8am PST. If you have any questions or concerns, I'll be checking email intermittently this weekend. -b ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto Beth, Thanks for getting this kicked off over the weekend. Sau! ___ yocto mailing list yocto@yoctoproject.org https://lists.yoctoproject.org/listinfo/yocto
