On Thu, 2023-10-26 at 15:11 +0200, David Pierret wrote:
> Hi,
>
> I've made separate steps for setup and run
> I didn't yet modify the python script in upgrade-helper to include
> Kevin reviews.
>
> follow some questions I have about the autobuilder.
>
> On Thu, Oct 26, 2023 at 3:04 PM David Pierret via
> lists.yoctoproject.org <david.pierret=smile...@lists.yoctoproject.org>
> wrote:
> >
> > - Modify the setup-auh to use repositories cloned by the auto-builder.
> > Manage the case where meta-openembedded is not used.
> >
> > - Modify the run-auh to manage the optional additional layer.
> > Only 1 layer can be provided.
> >
> > - Add auh-meta-openembedded job with 1 step per layer.
> >
> > Signed-off-by: David Pierret <david.pier...@smile.fr>
> > ---
> > config.json | 102 ++++++++++++++++++++++++++++++++++++++++++++--
> > scripts/run-auh | 39 ++++++++++++------
> > scripts/setup-auh | 34 ++++++++++------
> > 3 files changed, 146 insertions(+), 29 deletions(-)
> >
> > diff --git a/config.json b/config.json
> > index 3acb710..a168e48 100644
> > --- a/config.json
> > +++ b/config.json
> > @@ -1416,9 +1416,99 @@
> > "TEMPLATE" : "buildperf"
> > },
> > "auh" : {
> > - "EXTRAPLAINCMDS" : [
> > - "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR};
> > ${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR} ${WEBPUBLISH_DIR}/pub/auh/"
> > - ]
> > + "NEEDREPOS" : ["poky", "auto-upgrade-helper"],
> > + "step1" : {
> > + "shortname" : "setup AUH",
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}"
> > + ]
> > + },
> > + "step2" : {
> > + "shortname" : "run AUH poky",
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/"
> > + ]
> > + }
> > + },
> > + "auh-meta-openembedded" : {
> > + "NEEDREPOS" : ["poky", "meta-openembedded",
> > "auto-upgrade-helper"],
> > + "step1" : {
> > + "shortname" : "setup AUH",
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/setup-auh ${HELPERBUILDDIR}"
> > + ]
> > + },
> > + "step2" : {
> > + "shortname" : "run AUH for layer meta-oe",
> > +
> > + "ADDLAYER" : ["${BUILDDIR}/../meta-openembedded/meta-oe"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-oe"
> > + ]
> > + },
> > + "step3" : {
> > + "shortname" : "run AUH for layer meta-python",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-python"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-python"
> > + ]
> > + },
> > + "step4" : {
> > + "shortname" : "run AUH for layer meta-perl",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-perl"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-perl"
> > + ]
> > + },
> > + "step5" : {
> > + "shortname" : "run AUH for layer meta-networking",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-networking"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-networking"
> > + ]
> > + },
> > + "step6" : {
> > + "shortname" : "run AUH for layer meta-multimedia",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-multimedia"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-multimedia"
> > + ]
> > + },
> > + "step7" : {
> > + "shortname" : "run AUH for layer meta-gnome",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-gnome"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-gnome"
> > + ]
> > + },
> > + "step8" : {
> > + "shortname" : "run AUH for layer meta-xfce",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-xfce"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded meta-xfce"
> > + ]
> > + },
> > + "step9" : {
> > + "shortname" : "run AUH for layer meta-filesystems",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-filesystems"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-filesystems"
> > + ]
> > + },
> > + "step10" : {
> > + "shortname" : "run AUH for layer meta-initramfs",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-initramfs"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-initramfs"
> > + ]
> > + },
> > + "step11" : {
> > + "shortname" : "run AUH for layer meta-webserver",
> > + "ADDLAYER" :
> > ["${BUILDDIR}/../meta-openembedded/meta-webserver"],
> > + "EXTRAPLAINCMDS" : [
> > + "${SCRIPTSDIR}/run-auh ${HELPERBUILDDIR}
> > ${WEBPUBLISH_DIR}/pub/auh/ ${HELPERBUILDDIR}/meta-openembedded
> > meta-webserver"
> > + ]
> > + }
> > },
> > "a-quick" : {
> > "TEMPLATE" : "trigger-build"
> > @@ -1531,6 +1621,12 @@
> > "branch" : "master",
> > "revision" : "HEAD",
> > "no-layer-add" : true
> > + },
> > + "auto-upgrade-helper" : {
> > + "url" : "git://git.yoctoproject.org/auto-upgrade-helper",
> > + "branch" : "master",
> > + "revision" : "HEAD",
> > + "no-layer-add" : true
> > }
> > },
> > "buildtools" : {
> > diff --git a/scripts/run-auh b/scripts/run-auh
> > index 0419dd1..f59c561 100755
> > --- a/scripts/run-auh
> > +++ b/scripts/run-auh
> > @@ -7,28 +7,41 @@
> > # Called with $1 - the directory where the setup was created
> >
> > if [ -z $1 ]; then
> > - echo "Use: $0 auh_setup_dir [publish_dir]"
> > + echo "Use: $0 [auh_setup_dir] [publish_dir] [meta_dir] [meta_list]"
> > exit 1
> > fi
> >
> > -full_dir=$(readlink -e $1)
> > +build_dir=$(readlink -e $1)
> >
> > -auh_dir=$full_dir/auto-upgrade-helper
> > -poky_dir=$full_dir/poky
> > -build_dir=$full_dir/build
> > -sstate_dir=$full_dir/build/sstate-cache
> > +auh_dir=$build_dir/../auto-upgrade-helper
> > +poky_dir=$build_dir/../poky
> > +sstate_dir=$build_dir/sstate-cache
> >
> > pushd $poky_dir
> > + # Base the upgrades on poky master
> > + git fetch origin
> > + git checkout -B tmp-auh-upgrades origin/master
> > +popd
> > +
> > +LAYERS_ARGS=''
> > +# if layer arguments are provided
> > +if [ ! -z $3 ]; then
> > + meta_dir=$3
> > + meta_layer=$4
> > + machine_list="qemux86 qemux86-64 qemuarm qemumips qemuppc qemux86_musl"
>
> Do we have a variable that contains the machine provided in the config.json ?
I think this is different. The MACHINE value set in config.json is used
in the OE build itself. The list above is the set of MACHINE values auh
iterates over during testing. This is quite auh specific.
The easier way to customise this would be to set something in the
upgrade-helper.conf file or through the commandline. This was why I was
thinking of two conf files, one for meta-oe and one for oe-core.
It looks like Alex has changed the defaults in the script so we don't
need to make this piece meta-oe specific though.
>
> > +
> > + pushd $meta_dir || exit 1
> > + # Base the upgrades on master branch
> > + git fetch origin
> > + git checkout -B tmp-auh-upgrades origin/main
> > + popd
> >
> > -# Base the upgrades on poky master
> > -git fetch origin
> > -git checkout -B tmp-auh-upgrades origin/master
> > + LAYERS_ARGS="--layer-dir ${meta_dir} --layer-names ${meta_layer}
> > --layer-machines ${machine_list}"
> > +fi
> >
> > -source $poky_dir/oe-init-build-env $build_dir
> > -$auh_dir/upgrade-helper.py -e all
> > +# execute upgrade-helper script on all recipes
> > +$auh_dir/upgrade-helper.py -e ${LAYERS_ARGS} -- all
> >
> > if [ -n $2 ]; then
> > cp -rf $build_dir/upgrade-helper/* $2
> > fi
>
> Should this part be put in a step ?
It could be, not sure if it is worth it or not. Was there a particular
reason you think it should be?
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61497): https://lists.yoctoproject.org/g/yocto/message/61497
Mute This Topic: https://lists.yoctoproject.org/mt/102199062/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe:
https://lists.yoctoproject.org/g/yocto/leave/6691583/21656/737036229/xyzzy
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
