On Mon, Jul 26, 2021 at 6:37 PM Christopher Clark < [email protected]> wrote:
> On Tue, Jul 20, 2021 at 7:49 PM Bruce Ashfield <[email protected]> > wrote: > > > > On Thu, Jul 8, 2021 at 12:43 AM Christopher Clark > > <[email protected]> wrote: > > > > > > From: Christopher Clark <[email protected]> > > > > > > The Xen hypervisor built for Arm 64-bit targets can be launched with > > > runqemu by providing a Device Tree binary and configuration for Qemu, > > > which enables interactive testing of Xen images. > > > > > > Add qemuboot-xen-dtb.bbclass to add a new bitbake task for generating > > > the dtb file by using lopper on a device tree produced by Qemu. > > > > > > Add qemuboot-xen-defaults.bbclass to supply working default parameters > > > for the qemuarm64 machine and general support for qemuboot for Xen. > > > > > > Add the new qemuboot-xen classes to the xen-image-minimal image, along > > > with variable settings needed for qemuboot to run the image for > > > qemuarm64. > > > > > > The memory setting for Qemu has been increased to 512M after testing > > > with successful boots. Default fstype for qemux86-64 was found to need > a > > > stronger override in testing. Provide an override for the serial > console > > > setting for the qemuarm64 machine when the Xen distro feature is > > > enabled for the console in qemuboot, following the example of similar > > > machine settings in meta-arm-bsp. > > > > > > Development aided by this script by Stewart Hildebrand of DornerWorks: > > > https://gist.github.com/stewdk/110f43e0cc1d905fc6ed4c7e10d8d35e > > > > > > Signed-off-by: Christopher Clark <[email protected]> > > > --- > > > classes/qemuboot-xen-defaults.bbclass | 29 +++ > > > classes/qemuboot-xen-dtb.bbclass | 209 +++++++++++++++++++ > > > conf/distro/include/meta-virt-xen.inc | 3 + > > > recipes-extended/images/xen-image-minimal.bb | 6 +- > > > 4 files changed, 245 insertions(+), 2 deletions(-) > > > create mode 100644 classes/qemuboot-xen-defaults.bbclass > > > create mode 100644 classes/qemuboot-xen-dtb.bbclass > > > > > > diff --git a/classes/qemuboot-xen-defaults.bbclass > b/classes/qemuboot-xen-defaults.bbclass > > > new file mode 100644 > > > index 0000000..82cad49 > > > --- /dev/null > > > +++ b/classes/qemuboot-xen-defaults.bbclass > > > @@ -0,0 +1,29 @@ > > > +# Set defaults for booting Xen images with qemuboot > > > + > > > +# Xen and Dom0 command line options > > > +QB_XEN_CMDLINE_EXTRA ??= "dom0_mem=192M" > > > +QB_XEN_DOM0_BOOTARGS ??= \ > > > + "console=hvc0 earlyprintk clk_ignore_unused root=/dev/vda" > > > + > > > +# Launch with one initial domain, dom0, with one boot module, the > kernel > > > +DOM0_KERNEL="${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE}" > > > +DOM0_KERNEL_LOAD_ADDR="0x45000000" > > > +QB_XEN_DOMAIN_MODULES = > "${DOM0_KERNEL}:${DOM0_KERNEL_LOAD_ADDR}:multiboot,kernel" > > > + > > > +# Qemuboot for 64-bit Arm uses the QB_DEFAULT_KERNEL method to load > Xen > > > +# and the device loader option for the dom0 kernel: > > > +QB_OPT_APPEND_append_aarch64 = " \ > > > + -device loader,file=${DOM0_KERNEL},addr=${DOM0_KERNEL_LOAD_ADDR} \ > > > + " > > > +QB_DEFAULT_KERNEL_aarch64 = "xen-${MACHINE}" > > > + > > > +# 64-bit Arm: gic version 3 > > > +QB_MACHINE_aarch64 = "-machine virt,gic-version=3 -machine > virtualization=true" > > > + > > > +# Use overrides for stronger assignment here as default memory set by > the > > > +# machines is not high enough > > > +QB_MEM_qemuarm64 = "-m 512" > > > > What about making that memory value a variable (with 512 as the > > default) ? That allows us to change it without needing to always > > modify the bbclass. > > > > Almost all of my K8S runqemu launches need 512 or 1G of memory, so > > this is definitely not uncommon! > > ack - will do that for v2 > > > > > > + > > > +# 64-bit Arm: qemuboot with a device tree binary > > > +QB_DTB_aarch64 = "${IMAGE_NAME}.qemuboot.dtb" > > > +QB_DTB_LINK_aarch64 = "${IMAGE_LINK_NAME}.qemuboot.dtb" > > > diff --git a/classes/qemuboot-xen-dtb.bbclass > b/classes/qemuboot-xen-dtb.bbclass > > > new file mode 100644 > > > index 0000000..08f9b02 > > > --- /dev/null > > > +++ b/classes/qemuboot-xen-dtb.bbclass > > > @@ -0,0 +1,209 @@ > > > +# Enable booting Xen with qemuboot / runqemu: generate device tree > > > +# > > > +# Copyright (c) 2021 Star Lab Corp. All rights reserved. > > > +# > > > +# Author: Christopher Clark <[email protected]> > > > + > > > +# Interface variables: > > > +# > > > +# QB_DTB : defined in qemuboot.bbclass. > > > +# If set, this class will generate the specified device tree file. > > > +# > > > +# QB_XEN_CMDLINE_EXTRA : > > > +# A string to be appended to the default Xen hypervisor boot command > line, > > > +# for supplying Xen boot options. > > > +# The device tree that this bbclass generates will contain Xen command > > > +# line options to connect the Xen console to the Qemu serial port. > > > +# > > > +# QB_XEN_DOMAIN_MODULES : > > > +# A space-separated list of colon-separated entries: > > > +# "<file for the module>:<load memory address>:<module > compatibility string>" > > > +# > > > +# QB_XEN_DOM0_BOOTARGS : > > > +# A string for specifying Dom0 boot options for the Xen section of > the device > > > +# tree. > > > +# > > > +# QB_XEN_DTB_TASK_DEPENDS : > > > +# The task dependencies for the DTB generation. A default is provided. > > > +# > > > +# See also: Other QB_ variables as defined by the qemuboot.bbclass. > > > + > > > +write_lops_xen_section() { > > > + cat <<EOF >"$1" > > > +/dts-v1/; > > > +/ { > > > + compatible = "system-device-tree-v1"; > > > + lops { > > > + /* Connect the Xen console to Qemu dtuart */ > > > + lop_1 { > > > + compatible = "system-device-tree-v1,lop,code-v1"; > > > + code = " > > > + chosen = node.tree['/chosen'] > > > + stdout_path = str(chosen['stdout-path'].value[0]) > > > + chosen['xen,xen-bootargs'] = \ > > > + 'console=dtuart dtuart=%s' % stdout_path > > > + return True > > > + "; > > > + }; > > > + lop_2 { > > > + compatible = "system-device-tree-v1,lop,modify"; > > > + modify = > "/chosen:xen,dom0-bootargs:${QB_XEN_DOM0_BOOTARGS}"; > > > + }; > > > + lop_3 { > > > + compatible = "system-device-tree-v1,lop,modify"; > > > + modify = "/chosen:#address-cells:<1>"; > > > + }; > > > + lop_4 { > > > + compatible = "system-device-tree-v1,lop,modify"; > > > + modify = "/chosen:#size-cells:<1>"; > > > + }; > > > + }; > > > +}; > > > +EOF > > > +} > > > + > > > +write_lop_add_to_xen_cmdline() { > > > + EXTRA_XEN_BOOTARGS="$2" > > > + cat <<EOF >"$1" > > > +/dts-v1/; > > > +/ { > > > + compatible = "system-device-tree-v1"; > > > + lops { > > > + lop_1 { > > > + compatible = "system-device-tree-v1,lop,code-v1"; > > > + options = "extra_xen_bootargs:${EXTRA_XEN_BOOTARGS}"; > > > + code = " > > > + chosen = node.tree['/chosen'] > > > + xen_bootargs = str(chosen['xen,xen-bootargs'].value) > > > + chosen['xen,xen-bootargs'] = '%s %s' % \ > > > + (xen_bootargs, extra_xen_bootargs) > > > + return True > > > + "; > > > + }; > > > + }; > > > +}; > > > +EOF > > > +} > > > + > > > +write_lop_add_chosen_module() { > > > + ADDR="$2" > > > + SIZE="$3" > > > + MODULE_TYPE="$4" > > > + cat <<EOF >"$1" > > > +/dts-v1/; > > > +/ { > > > + compatible = "system-device-tree-v1"; > > > + lops { > > > + lop_1 { > > > + compatible = "system-device-tree-v1,lop,add"; > > > + node_src = "module@${ADDR}"; > > > + node_dest = "/chosen/module@${ADDR}"; > > > + > > > + module@${ADDR} { > > > + compatible = "multiboot,module", "${MODULE_TYPE}"; > > > + reg = <${ADDR} ${SIZE}>; > > > + }; > > > + }; > > > + }; > > > +}; > > > +EOF > > > +} > > > + > > > +generate_xen_qemuboot_dtb() { > > > + # First: invoke qemu to generate an initial device tree. > > > + # Parameters supplied here inspired by inspection of: > > > + # runqemu "${IMAGE_BASENAME}" nographic slirp \ > > > + # qemuparams='-dtb "" -machine > dumpdtb=${B}/qemu-dumped.dtb' > > > + ${QB_SYSTEM_NAME} \ > > > + -device qemu-xhci \ > > > + -device usb-tablet \ > > > + -device usb-kbd \ > > > + -machine virt,gic-version=3 \ > > > + -machine virtualization=true \ > > > + ${QB_CPU} \ > > > + ${QB_SMP} \ > > > + ${QB_MEM} \ > > > + -nographic \ > > > + -serial mon:stdio \ > > > + -machine "dumpdtb=${B}/qemu-dumped.dtb" > > > + > > > + # Lopper generates temporary files in cwd, so run it within ${B} > > > + cd "${B}" > > > + write_lops_xen_section "${B}/lop-insert-xen-section.dts" > > > + > > > + write_lop_add_to_xen_cmdline "${B}/lop-xen-cmdline.dts" \ > > > + "${QB_XEN_CMDLINE_EXTRA}" > > > + > > > + if [ -z "${QB_XEN_DOMAIN_MODULES}" ]; then > > > + bbwarn "No domain modules: please set QB_XEN_DOMAIN_MODULES" > > > + fi > > > + > > > + for DOMAIN_MODULE in ${QB_XEN_DOMAIN_MODULES} > > > + do > > > + MODULE_FILE="$(echo ${DOMAIN_MODULE} | cut -f1 -d:)" > > > + ADDR="$(echo ${DOMAIN_MODULE} | cut -f2 -d:)" > > > + MODULE_TYPE="$(echo ${DOMAIN_MODULE} | cut -f3 -d:)" > > > + RESOLVED_FILE="$(readlink -f ${MODULE_FILE})" > > > + SIZE=$(printf '0x%x\n' $(stat -c '%s' "${RESOLVED_FILE}")) > > > + [ "x${SIZE}" != "x0x0" ] || bbfatal No module: > "${MODULE_FILE}" > > > + write_lop_add_chosen_module "${B}/lop-add-module-${ADDR}.dts" > \ > > > + "${ADDR}" "${SIZE}" "${MODULE_TYPE}" > > > + LOP_MODULE_ARGS="${LOP_MODULE_ARGS} -i > ${B}/lop-add-module-${ADDR}.dts" > > > + done > > > + > > > + QEMUBOOT_DTB="${IMGDEPLOYDIR}/${QB_DTB}" > > > + QEMUBOOT_DTB_LINK="${IMGDEPLOYDIR}/${QB_DTB_LINK}" > > > + > > > + lopper.py --werror --verbose \ > > > + -i "${B}/lop-insert-xen-section.dts" \ > > > + -i "${B}/lop-xen-cmdline.dts" \ > > > + ${LOP_MODULE_ARGS} \ > > > + -f -o "${QEMUBOOT_DTB}" \ > > > + "${B}/qemu-dumped.dtb" > > > + > > > + # To assist debugging: > > > + dtc -I dtb -O dts -o "${B}/output.dts" "${QEMUBOOT_DTB}" > > > + > > > + if [ "${QEMUBOOT_DTB_LINK}" != "${QEMUBOOT_DTB}" ] ; then > > > + if [ -e "${QEMUBOOT_DTB_LINK}" ] ; then > > > + rm "${QEMUBOOT_DTB_LINK}" > > > + fi > > > + ln -s "${QB_DTB}" "${QEMUBOOT_DTB_LINK}" > > > + fi > > > +} > > > + > > > +do_write_xen_qemuboot_dtb() { > > > + # Not all architectures qemuboot with a device tree binary, so > check > > > + # to see if one is needed. This allows this bbclass file to be > used > > > + # in the same image recipe for multiple architectures. > > > + if [ -n "${QB_DTB}" ] ; then > > > + generate_xen_qemuboot_dtb > > > + fi > > > +} > > > + > > > +addtask do_write_xen_qemuboot_dtb after do_write_qemuboot_conf before > do_image > > > +# Task dependency: > > > +# An expected common case is that the kernel for at least one of the > initial > > > +# domains (eg. dom0) is deployed from the virtual/kernel recipe, so > > > +# add virtual/kernel:do_deploy as a task dependency here since the > kernel size > > > +# needs to be known for generating the device tree. > > > +# Dependencies are only introduced if a device tree will be generated. > > > +QB_XEN_DTB_TASK_DEPENDS ?= " \ > > > + ${@[ ' \ > > > + qemu-helper-native:do_populate_sysroot \ > > > + lopper-native:do_populate_sysroot \ > > > + dtc-native:do_populate_sysroot \ > > > + virtual/kernel:do_deploy \ > > > + ', ''][d.getVar('QB_DTB') == '' or d.getVar('QB_DTB') is None]} \ > > > + " > > > +do_write_xen_qemuboot_dtb[depends] = "${QB_XEN_DTB_TASK_DEPENDS}" > > > + > > > +def qemuboot_dtb_vars(d): > > > + build_vars = ['MACHINE', 'TUNE_ARCH', 'DEPLOY_DIR_IMAGE', > > > + 'KERNEL_IMAGETYPE', 'IMAGE_NAME', 'IMAGE_LINK_NAME', > > > + 'STAGING_DIR_NATIVE', 'STAGING_BINDIR_NATIVE', > > > + 'STAGING_DIR_HOST', 'SERIAL_CONSOLES'] > > > + return build_vars + [k for k in d.keys() if k.startswith('QB_')] > > > + > > > +do_write_qemuboot_dtb[vardeps] += "${@' '.join(qemuboot_dtb_vars(d))}" > > > +do_write_qemuboot_dtb[vardepsexclude] += "TOPDIR" > > > diff --git a/conf/distro/include/meta-virt-xen.inc > b/conf/distro/include/meta-virt-xen.inc > > > index 02a7810..2becba1 100644 > > > --- a/conf/distro/include/meta-virt-xen.inc > > > +++ b/conf/distro/include/meta-virt-xen.inc > > > @@ -10,3 +10,6 @@ XEN_RPI4_64_CONFIG_PATH ??= \ > > > > 'dynamic-layers/raspberrypi/conf/distro/include/xen-raspberrypi4-64.inc' > > > include ${@bb.utils.contains('MACHINE', 'raspberrypi4-64', \ > > > '${XEN_RPI4_64_CONFIG_PATH}', '', d)} > > > + > > > +# Set serial for working qemuboot console > > > +SERIAL_CONSOLES_qemuarm64 ?= "115200;ttyAMA0" > > > diff --git a/recipes-extended/images/xen-image-minimal.bb > b/recipes-extended/images/xen-image-minimal.bb > > > index 6733801..2175e63 100644 > > > --- a/recipes-extended/images/xen-image-minimal.bb > > > +++ b/recipes-extended/images/xen-image-minimal.bb > > > @@ -31,7 +31,7 @@ XEN_PCIBACK_MODULE_x86-64 = > "kernel-module-xen-pciback" > > > > > > LICENSE = "MIT" > > > > > > -inherit core-image > > > +inherit core-image qemuboot-xen-defaults qemuboot-xen-dtb > > > > > > do_check_xen_state() { > > > if [ "${@bb.utils.contains('DISTRO_FEATURES', 'xen', ' yes', > 'no', d)}" = "no" ]; then > > > @@ -74,10 +74,12 @@ build_syslinux_cfg () { > > > > > > # Enable runqemu. eg: runqemu xen-image-minimal nographic slirp > > > WKS_FILE_x86-64 = "directdisk-xen.wks" > > > -QB_MEM ?= "-m 400" > > > +QB_MEM ?= "-m 512" > > > > Since you are bumping the default to 512 here. Do we even need that > > qemuarm64 specific override in the bbclass ? > > ack, I don't think we do. > I think the following works OK: put this into the > qemuboot-xen-defaults.bbclass: > > +# Increase the default qemu memory allocation to allow for the hypervisor. > +# Use a weak assignment to allow for change of default and override > elsewhere. > +QB_MEM_VALUE ??= "512" > +QB_MEM = "-m ${QB_MEM_VALUE}" > > and then setting QB_MEM at all can be dropped from the > xen-image-minimal recipe (since the default is now fine). > > If an image does need to update the memory setting, it can set QB_MEM_VALUE > which has a weak default so is easily overridden. It's also > MACHINE-agnostic, which is preferable. > > Does that sound OK? > Yes, sounds good to me! Bruce > > Christopher > > > > > > Bruce > > > > > QB_DEFAULT_KERNEL ?= "none" > > > QB_DEFAULT_FSTYPE ?= "wic" > > > +QB_DEFAULT_FSTYPE_qemux86-64 = "wic" > > > QB_FSINFO ?= "wic:kernel-in-fs" > > > +QB_SERIAL_OPT = "-serial mon:stdio" > > > # qemux86-64 machine does not include 'wic' in IMAGE_FSTYPES, which > is needed > > > # to boot this image, so add it here: > > > IMAGE_FSTYPES_qemux86-64 += "wic" > > > -- > > > 2.25.1 > > > > > > > > > -- > > - Thou shalt not follow the NULL pointer, for chaos and madness await > > thee at its end > > - "Use the force Harry" - Gandalf, Star Trek II > -- - Thou shalt not follow the NULL pointer, for chaos and madness await thee at its end - "Use the force Harry" - Gandalf, Star Trek II
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#6680): https://lists.yoctoproject.org/g/meta-virtualization/message/6680 Mute This Topic: https://lists.yoctoproject.org/mt/84061644/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/meta-virtualization/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
