Re: [OE-core] [PATCH v2] bitbake-buildall: automate build testing for qemu MACHINEs
On Tue, Feb 11, 2020 at 6:06 AM Trevor Gamblin wrote: > > > On 2/10/20 7:28 PM, Khem Raj wrote: > > On Mon, Feb 10, 2020 at 3:47 PM Trevor Gamblin > > wrote: > >> bitbake-buildall simplifies the process of build testing an upgraded or > >> patched recipe by cycling through the build steps for each available qemu > >> target, with desired LIBC specified by the user as an option (defaulting > >> to both glibc and musl if no option is provided). While building, a log > >> file with the name "-buildall.log" is written, containing a PASS > >> or FAIL line upon completion of the build for each architecture. For now, > >> qemu targets are not selectable (i.e. the recipe is built for all qemu > >> targets found in meta/conf/machine), but this functionality can be added > >> in the future along with other useful options. > >> > >> The log file created by bitbake-buildall also includes some basic system > >> info (e.g. build start time, hostname, host OS, host kernel). Since it is > >> not guaranteed that tools such as lsb_release will be available on the > >> host (it isn't by default on my Fedora machine), this information is > >> collected manually. Additionally, the previous run's log is retained for > >> comparison by renaming it in the format -buildall.log.old once a > >> new set of builds is triggered for the same recipe. > >> > >> Finally, in v2, the line "set -o pipefail" has been removed to be fully > >> POSIX-compatible. > >> > >> Sample log output: > >> > >> BITBAKE-BUILDALL LOG FOR aspell > >> START TIME: 2020-02-05_15:57:41 > >> HOSTNAME: yow-tgamblin-fedora2 > >> HOST OS: Fedora 31 (Server Edition) > >> HOST KERNEL: 5.4.10-200.fc31.x86_64 > >> === > >> BUILD RESULTS: > >> [glibc] > >> PASS: qemuarmv5 > >> PASS: qemux86 > >> PASS: qemuppc > >> PASS: qemumips64 > >> FAIL: qemux86-64 > >> FAIL: qemumips > >> FAIL: qemuarm > >> FAIL: qemuarm64 > >> FAIL: qemuriscv64 > >> [musl] > >> PASS: qemuarmv5 > >> PASS: qemux86 > >> PASS: qemuppc > >> PASS: qemumips64 > >> PASS: qemux86-64 > >> PASS: qemumips > >> PASS: qemuarm > >> PASS: qemuarm64 > >> PASS: qemuriscv64 > >> === > >> PASSED: 13 > >> FAILED: 5 > >> > >> Signed-off-by: Trevor Gamblin > >> --- > >> scripts/bitbake-buildall | 120 +++ > >> 1 file changed, 120 insertions(+) > >> create mode 100755 scripts/bitbake-buildall > > if its building qemu machines only, then naming it build-qemuall.sh or > > some such would be > > better, I would avoid usine bitbake- prefix. That can confuse with bitbake > > tools > > The intent is that it can be extended to machines other than just qemu > with later patches, so I'm hesitant to give it a qemu-specific name. > its not clear if its generic enough, its grepping into hardcoded paths in oe-core repo which perhaps makes it specific. > I could rename it to something like buildall-targets. > since we use bitbake- for general tooling around bitbake, this seems not fit that bill. > > > >> diff --git a/scripts/bitbake-buildall b/scripts/bitbake-buildall > >> new file mode 100755 > >> index 00..74a5994f2f > >> --- /dev/null > >> +++ b/scripts/bitbake-buildall > >> @@ -0,0 +1,120 @@ > >> +#!/bin/sh > >> +# Copyright (c) 2020 Wind River Systems, Inc. > >> +# > >> +# SPDX-License-Identifier: GPL-2.0-only > >> +# > >> +# bitbake-buildall: a tool for automating build testing of recipes > >> +# TODO: Add support for selecting which qemu architectures to build > >> +# TODO: Add support for queueing up multiple recipe builds > >> +# TODO: Add more logging options (e.g. local.conf info, bitbake env info) > >> + > >> +usage () > >> +{ > >> +base=$(basename "$0") > >> +echo "Usage: $base [options] [recipename/target]" > >> +echo "Executes a build of a given target for selected LIBCs. With no > >> options, default to both libc and musl." > >> +echo "Options:" > >> +echo "-l, --libcSpecify one of \"libc\" or \"musl\"" > >> +} > >> + > >> + > >> +buildall () > >> +{ > >> +# Get path to oe-core directory. Since oe-init-build-env prepends > >> $PATH with > >> +# the path to the scripts directory, get it from there > >> +SCRIPTS_PATH="$(echo "$PATH" | cut -d ":" -f 1)" > >> +OE_CORE_PATH=$(echo "$SCRIPTS_PATH" | sed 's|\(.*\)/.*|\1|') > >> + > >> +# Get target list and host machine information > >> +TARGET_LIST=$(find "$OE_CORE_PATH"/meta/conf/machine -maxdepth 1 > >> -type f | grep qemu | sed 's|.*/||' | sed -e 's/\.conf//') > >> + > >> +# Set LIBC value to use for the builds based on options provided by > >> the user > >> +if [ -n "$2" ] > >> +then > >> + LIBC_LIST="$2" > >> + echo "$LIBC_LIST" > >> +else > >> + LIBC_LIST="glibc musl" > >> + echo "$LIBC_LIST" > >> +fi > >> + > >> +START_TIME=$(date "+%Y-%m-%d_%H:%M:%S") > >> +LOG_FILE="$1-buildall.log" > >> +OS_INFO=$(grep "PRETTY_NAME=" /etc/os-release | awk -F "=" '{print > >> $2}' | sed -e
Re: [OE-core] [PATCH v2] bitbake-buildall: automate build testing for qemu MACHINEs
On 2/10/20 7:28 PM, Khem Raj wrote: On Mon, Feb 10, 2020 at 3:47 PM Trevor Gamblin wrote: bitbake-buildall simplifies the process of build testing an upgraded or patched recipe by cycling through the build steps for each available qemu target, with desired LIBC specified by the user as an option (defaulting to both glibc and musl if no option is provided). While building, a log file with the name "-buildall.log" is written, containing a PASS or FAIL line upon completion of the build for each architecture. For now, qemu targets are not selectable (i.e. the recipe is built for all qemu targets found in meta/conf/machine), but this functionality can be added in the future along with other useful options. The log file created by bitbake-buildall also includes some basic system info (e.g. build start time, hostname, host OS, host kernel). Since it is not guaranteed that tools such as lsb_release will be available on the host (it isn't by default on my Fedora machine), this information is collected manually. Additionally, the previous run's log is retained for comparison by renaming it in the format -buildall.log.old once a new set of builds is triggered for the same recipe. Finally, in v2, the line "set -o pipefail" has been removed to be fully POSIX-compatible. Sample log output: BITBAKE-BUILDALL LOG FOR aspell START TIME: 2020-02-05_15:57:41 HOSTNAME: yow-tgamblin-fedora2 HOST OS: Fedora 31 (Server Edition) HOST KERNEL: 5.4.10-200.fc31.x86_64 === BUILD RESULTS: [glibc] PASS: qemuarmv5 PASS: qemux86 PASS: qemuppc PASS: qemumips64 FAIL: qemux86-64 FAIL: qemumips FAIL: qemuarm FAIL: qemuarm64 FAIL: qemuriscv64 [musl] PASS: qemuarmv5 PASS: qemux86 PASS: qemuppc PASS: qemumips64 PASS: qemux86-64 PASS: qemumips PASS: qemuarm PASS: qemuarm64 PASS: qemuriscv64 === PASSED: 13 FAILED: 5 Signed-off-by: Trevor Gamblin --- scripts/bitbake-buildall | 120 +++ 1 file changed, 120 insertions(+) create mode 100755 scripts/bitbake-buildall if its building qemu machines only, then naming it build-qemuall.sh or some such would be better, I would avoid usine bitbake- prefix. That can confuse with bitbake tools The intent is that it can be extended to machines other than just qemu with later patches, so I'm hesitant to give it a qemu-specific name. I could rename it to something like buildall-targets. diff --git a/scripts/bitbake-buildall b/scripts/bitbake-buildall new file mode 100755 index 00..74a5994f2f --- /dev/null +++ b/scripts/bitbake-buildall @@ -0,0 +1,120 @@ +#!/bin/sh +# Copyright (c) 2020 Wind River Systems, Inc. +# +# SPDX-License-Identifier: GPL-2.0-only +# +# bitbake-buildall: a tool for automating build testing of recipes +# TODO: Add support for selecting which qemu architectures to build +# TODO: Add support for queueing up multiple recipe builds +# TODO: Add more logging options (e.g. local.conf info, bitbake env info) + +usage () +{ +base=$(basename "$0") +echo "Usage: $base [options] [recipename/target]" +echo "Executes a build of a given target for selected LIBCs. With no options, default to both libc and musl." +echo "Options:" +echo "-l, --libcSpecify one of \"libc\" or \"musl\"" +} + + +buildall () +{ +# Get path to oe-core directory. Since oe-init-build-env prepends $PATH with +# the path to the scripts directory, get it from there +SCRIPTS_PATH="$(echo "$PATH" | cut -d ":" -f 1)" +OE_CORE_PATH=$(echo "$SCRIPTS_PATH" | sed 's|\(.*\)/.*|\1|') + +# Get target list and host machine information +TARGET_LIST=$(find "$OE_CORE_PATH"/meta/conf/machine -maxdepth 1 -type f | grep qemu | sed 's|.*/||' | sed -e 's/\.conf//') + +# Set LIBC value to use for the builds based on options provided by the user +if [ -n "$2" ] +then + LIBC_LIST="$2" + echo "$LIBC_LIST" +else + LIBC_LIST="glibc musl" + echo "$LIBC_LIST" +fi + +START_TIME=$(date "+%Y-%m-%d_%H:%M:%S") +LOG_FILE="$1-buildall.log" +OS_INFO=$(grep "PRETTY_NAME=" /etc/os-release | awk -F "=" '{print $2}' | sed -e 's/^"//' -e 's/"$//') + +# Append an existing log file for this build with .old if one exists +if [ -f "${LOG_FILE}" ] +then + mv "${LOG_FILE}" "${LOG_FILE}.old" +else + touch "${LOG_FILE}" +fi + +# Fill the log file with build and host info +echo "BITBAKE-BUILDALL LOG FOR $1" >> "${LOG_FILE}" +echo "START TIME: ${START_TIME}" >> "${LOG_FILE}" +echo "HOSTNAME: $(uname -n)" >> "${LOG_FILE}" +echo "HOST OS: ${OS_INFO}" >> "${LOG_FILE}" +echo "HOST KERNEL: $(uname -r)" >> "${LOG_FILE}" +echo "===" >> "${LOG_FILE}" +echo "BUILD RESULTS:" >> "${LOG_FILE}" + +# start the builds for each MACHINE and TCLIBC +for j in ${LIBC_LIST} +do + echo "[$j]" >> "${LOG_FILE}" + for i in ${TARGET_LIST} + do + echo "$i" "$j"; \ + TCLIBC=$j MA
Re: [OE-core] [PATCH v2] bitbake-buildall: automate build testing for qemu MACHINEs
On Mon, Feb 10, 2020 at 3:47 PM Trevor Gamblin wrote: > > bitbake-buildall simplifies the process of build testing an upgraded or > patched recipe by cycling through the build steps for each available qemu > target, with desired LIBC specified by the user as an option (defaulting > to both glibc and musl if no option is provided). While building, a log > file with the name "-buildall.log" is written, containing a PASS > or FAIL line upon completion of the build for each architecture. For now, > qemu targets are not selectable (i.e. the recipe is built for all qemu > targets found in meta/conf/machine), but this functionality can be added > in the future along with other useful options. > > The log file created by bitbake-buildall also includes some basic system > info (e.g. build start time, hostname, host OS, host kernel). Since it is > not guaranteed that tools such as lsb_release will be available on the > host (it isn't by default on my Fedora machine), this information is > collected manually. Additionally, the previous run's log is retained for > comparison by renaming it in the format -buildall.log.old once a > new set of builds is triggered for the same recipe. > > Finally, in v2, the line "set -o pipefail" has been removed to be fully > POSIX-compatible. > > Sample log output: > > BITBAKE-BUILDALL LOG FOR aspell > START TIME: 2020-02-05_15:57:41 > HOSTNAME: yow-tgamblin-fedora2 > HOST OS: Fedora 31 (Server Edition) > HOST KERNEL: 5.4.10-200.fc31.x86_64 > === > BUILD RESULTS: > [glibc] > PASS: qemuarmv5 > PASS: qemux86 > PASS: qemuppc > PASS: qemumips64 > FAIL: qemux86-64 > FAIL: qemumips > FAIL: qemuarm > FAIL: qemuarm64 > FAIL: qemuriscv64 > [musl] > PASS: qemuarmv5 > PASS: qemux86 > PASS: qemuppc > PASS: qemumips64 > PASS: qemux86-64 > PASS: qemumips > PASS: qemuarm > PASS: qemuarm64 > PASS: qemuriscv64 > === > PASSED: 13 > FAILED: 5 > > Signed-off-by: Trevor Gamblin > --- > scripts/bitbake-buildall | 120 +++ > 1 file changed, 120 insertions(+) > create mode 100755 scripts/bitbake-buildall if its building qemu machines only, then naming it build-qemuall.sh or some such would be better, I would avoid usine bitbake- prefix. That can confuse with bitbake tools > > diff --git a/scripts/bitbake-buildall b/scripts/bitbake-buildall > new file mode 100755 > index 00..74a5994f2f > --- /dev/null > +++ b/scripts/bitbake-buildall > @@ -0,0 +1,120 @@ > +#!/bin/sh > +# Copyright (c) 2020 Wind River Systems, Inc. > +# > +# SPDX-License-Identifier: GPL-2.0-only > +# > +# bitbake-buildall: a tool for automating build testing of recipes > +# TODO: Add support for selecting which qemu architectures to build > +# TODO: Add support for queueing up multiple recipe builds > +# TODO: Add more logging options (e.g. local.conf info, bitbake env info) > + > +usage () > +{ > +base=$(basename "$0") > +echo "Usage: $base [options] [recipename/target]" > +echo "Executes a build of a given target for selected LIBCs. With no > options, default to both libc and musl." > +echo "Options:" > +echo "-l, --libcSpecify one of \"libc\" or \"musl\"" > +} > + > + > +buildall () > +{ > +# Get path to oe-core directory. Since oe-init-build-env prepends $PATH > with > +# the path to the scripts directory, get it from there > +SCRIPTS_PATH="$(echo "$PATH" | cut -d ":" -f 1)" > +OE_CORE_PATH=$(echo "$SCRIPTS_PATH" | sed 's|\(.*\)/.*|\1|') > + > +# Get target list and host machine information > +TARGET_LIST=$(find "$OE_CORE_PATH"/meta/conf/machine -maxdepth 1 -type f > | grep qemu | sed 's|.*/||' | sed -e 's/\.conf//') > + > +# Set LIBC value to use for the builds based on options provided by the > user > +if [ -n "$2" ] > +then > + LIBC_LIST="$2" > + echo "$LIBC_LIST" > +else > + LIBC_LIST="glibc musl" > + echo "$LIBC_LIST" > +fi > + > +START_TIME=$(date "+%Y-%m-%d_%H:%M:%S") > +LOG_FILE="$1-buildall.log" > +OS_INFO=$(grep "PRETTY_NAME=" /etc/os-release | awk -F "=" '{print $2}' > | sed -e 's/^"//' -e 's/"$//') > + > +# Append an existing log file for this build with .old if one exists > +if [ -f "${LOG_FILE}" ] > +then > + mv "${LOG_FILE}" "${LOG_FILE}.old" > +else > + touch "${LOG_FILE}" > +fi > + > +# Fill the log file with build and host info > +echo "BITBAKE-BUILDALL LOG FOR $1" >> "${LOG_FILE}" > +echo "START TIME: ${START_TIME}" >> "${LOG_FILE}" > +echo "HOSTNAME: $(uname -n)" >> "${LOG_FILE}" > +echo "HOST OS: ${OS_INFO}" >> "${LOG_FILE}" > +echo "HOST KERNEL: $(uname -r)" >> "${LOG_FILE}" > +echo "===" >> "${LOG_FILE}" > +echo "BUILD RESULTS:" >> "${LOG_FILE}" > + > +# start the builds for each MACHINE and TCLIBC > +for j in ${LIBC_LIST} > +do > + echo "[$j]" >> "${LOG_FILE}" > + for i in ${TARGET_LIST} > + do > + echo "$i" "$j