On 11/04/2025 9:32 pm, Marek Marczykowski-Górecki wrote:
> The prerequisite for this is building rootfs.cpio.gz, not only
> initrd.tar.gz. That's a change in the test-artifacts repository in the
> "Rootfs generation" series from Andrew.
>
> Having that, do not repack the whole initrd, but only pack modified
> files and rely on Linux handling of concatenated archives.
> This allows packing just test-related files (which includes the whole
> toolstack), instead of the whole initrd.
It's probably worth saying ", fixed subsequently)" for the toolstack, as
it's the single largest aspect.
> DomU initrd handling is a bit more complicated thing. It's sent to the
> target host as part of the dom0 initrd. For now include prepared domU
> initrd as a whole in dom0's rootfs "overlay", which means compressing it
> (again) each job (this takes 3s). This can be later improved by
> splitting domU initrd into two parts (base + overlay) and concatenate
> them in target dom0 already. For this to be useful, test-artifacts would
> need to provide pre-made double-cpio version too (rootfs.cpio.gz wrapper
> in yet another cpio.gz as boot/domU-initrd).
>
> Since full initrd is not unpacked now when preparing domU (and dom0)
> rootfs, a couple of minor changes are needed to how they are prepared.
> This includes creating whole etc/issue file, instead of modifying
> existing one, and a need to create a couple directories.
>
> Finally, move adding "rc_verbose=yes" to /etc/rc.conf to initrd
> building.
These two paragraphs are a little stale now. etc/issue has already been
committed, and there's no initd building now I've moved arm into test
artefacts.
However, they're going to need tweaking again if I can get the rootfs
generation working more nicely, so lets hold off for now.
>
> Signed-off-by: Marek Marczykowski-Górecki <marma...@invisiblethingslab.com>
> ---
> Changes in v3:
> - adjust for Andrew's version of cpio rootfs artifacts
> - don't require double-cpio archive for domU initrd, instead add it via
> dom0 initrd as plain cpio to avoid re-compressing the same thing
> New in v2
>
> Jobs xilinx-* are untested, as I don't have necessary access.
> ---
> automation/gitlab-ci/test.yaml | 6 +++--
> automation/scripts/qemu-alpine-x86_64.sh | 9 +++-----
> automation/scripts/qemu-smoke-dom0-arm64.sh | 7 ++----
> automation/scripts/qemu-smoke-dom0less-arm64.sh | 7 ++----
> automation/scripts/qubes-x86-64.sh | 18 +++++++---------
> automation/scripts/xilinx-smoke-dom0-x86_64.sh | 15 ++++++-------
> automation/scripts/xilinx-smoke-dom0less-arm64.sh | 14 +++++-------
> 7 files changed, 35 insertions(+), 41 deletions(-)
One thing that does matter now is the order of appending archives.
I've reworked this locally to be of the form:
# DomU rootfs
cp ../binaries/rootfs.cpio.gz ../binaries/domU-rootfs.cpio.gz
# test-local configuration
mkdir rootfs
...
find . | cpio -H newc -o | gzip >> ../binaries/domU-rootfs.cpio.gz
This then ends up looking rather more clean in followon patches, where
we end up with:
# Dom0 rootfs
cp ../binaries/ucode.cpio ../binaries/dom0-rootfs.cpio.gz
cat ../binaries/rootfs.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
cat ../binaries/toolstack.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
if [[ "${TEST}" == argo ]]; then
cat ../binaries/argo.cpio.gz >> ../binaries/dom0-rootfs.cpio.gz
fi
# test-local configuration
...
find . | cpio -H newc -o | gzip >> ../binaries/dom0-rootfs.cpio.gz
> diff --git a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> index 4db249530823..52a0e05e3f1b 100755
> --- a/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> +++ b/automation/scripts/xilinx-smoke-dom0-x86_64.sh
> @@ -80,6 +80,7 @@ argo-exec -l -p 28333 -- /bin/echo
> "
> copy_dom0_files ()
> {
> + mkdir -p root usr/local/lib
> cp "${WORKDIR}/binaries/xen-argo.ko" "root/"
> cp -ar "${WORKDIR}/binaries/lib/"* "usr/local/lib/"
> cp "${WORKDIR}/binaries/argo-exec" "usr/local/bin/"
> @@ -92,9 +93,8 @@ fi
> # Set up domU rootfs.
> mkdir -p rootfs
> cd rootfs
> -tar xzf ../binaries/initrd.tar.gz
> mkdir proc run srv sys
> -rm var/run
> +mkdir -p etc/local.d
> echo "#!/bin/sh
> set -x
> export LD_LIBRARY_PATH=/usr/local/lib
> @@ -102,23 +102,22 @@ PATH=/usr/local/bin:/usr/local/sbin:\$PATH
> ${DOMU_CMD}
> " > etc/local.d/xen.start
> chmod +x etc/local.d/xen.start
> -echo "rc_verbose=yes" >> etc/rc.conf
> echo "domU Welcome to Alpine Linux 3.18
> Kernel \r on an \m (\l)
>
> " > etc/issue
> copy_domU_files
> -find . | cpio -H newc -o | gzip > ../binaries/domU-rootfs.cpio.gz
> +cp ../binaries/initrd.cpio.gz ../binaries/domU-rootfs.cpio.gz
Stray initrd. Found while reworking.
I'll give the whole lot commit-by-commit full runs, when I'm done tinkering.
~Andrew
