Re: [OE-core] [PATCH v8 0/3] pkg-database and systemd-sysext image
Hoi Alexandre,
and thanks again for patiently pointing out the autobuilder problems :-)
"only" the test_image_gen_pkgdbfs_{ipk,deb,rpm} fail, the
test_image_use_pkgdbfs succeeds
picking apart the yocto-autobuilder-helper/config.json showed the difference:
basically all oe-selftest-* use
"MACHINE" : "qemux86-64"
**but** oe-selftest-armhost sets:
"MACHINE": "qemuarm64"
the test_image_gen_pkgdbfs_ checks to sanity check the packagamanager state
looked for a "kernel-image" package in the listing... but for quemuarm64
"packagegroup-core-boot" replaces that
v9 of the patchstack adapts that particular assert(s) to look for either the
kernel-image or core-boot depending on the set MACHINE
gruß
Johannes
From: Alexandre Belloni
Sent: Wednesday, May 15, 2024 21:09
To: SCHNEIDER Johannes
Cc: openembedded-core@lists.openembedded.org;
richard.pur...@linuxfoundation.org; alex.kana...@gmail.com
Subject: Re: [OE-core] [PATCH v8 0/3] pkg-database and systemd-sysext image
This email is not from Hexagon’s Office 365 instance. Please be careful while
clicking links, opening attachments, or replying to this email.
Hello,
It seems that this reliably fails oe-selftest-armhost. I didn't really
pay attention until now beause we had other issues but in this build,
those are the only failing tests:
https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/3322/steps/15/logs/stdio
On 12/05/2024 08:31:56+0200, Johannes Schneider via lists.openembedded.org
wrote:
> systemd-sysext allows to overlay another image (or multiple) ontop of
> a "base-image" = the current rootfs, via the use of overlayfs; to add
> tools and features meant for development purposes.
>
> To quote the documentation on systemd-sysext:
> " ...addition in order to make debugging/development easier). System
> extension images should not be misunderstood as a generic software
> packaging framework, ..."
>
> To build a lean image, that only holds packages that are not already
> part of the base-image, a snapshot of the package-database is taken
> after the installation of the base-rootfs is done, and picked up again
> when collecting the rootfs of such a extension image.
>
> with all this in place an example usage could look like this:
> some-core-image.bb
> inherit core-image
> IMAGE_GEN_PKGDBFS = "1"
>
> extending-image.bb
> inherit image-sysext
> IMAGE_FSTYPES = "squashfs"
> IMAGE_BASE_PKGDB = "some-core-image"
> # the above pointing at a package-db similar to:
> #
> build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz
>
> then on the device, running some-core-image, with the extension image placed
> at FN:
> $> ln -s "$FN" /run/extensions/$(basename $FN).raw
> $> systemd-sysext list
> $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge
>
> As long as the VERSION_ID of the extension image matches the os-release
> in the base image, the above commands return sucessfully;
> for details on the compativility check see the docs for systemd-sysext.
>
> =
>
> changes with v2:
> rebase from 'kirkstone' onto 'master'
>
> changes with v3:
> incorporate review suggestions for simplification
> add task dependency handling
> add oe-selftest for the pkgdb handling
> add variable documentation and
> some more comments, and examples in the commit-msg
>
> changes with v4:
> rebase onto 'master' => no functional changes
> fixed patchtest findings
>
> changes with v5:
> rebase onto 'master'
> add '.sysext' to the deployed symlink name
> sidenote on the tests and autobuilder failure: run locally they
> succeed, e.g.:
> #> oe-selftest --verbose -r
> imagefeatures.ImageFeatures.test_image_gen_pkgdbfs
>
> changes with v6:
> tests: restructure to call 'bitbake' only once in the testcase itself
> (in hopes of solving the autobuilder problem; local test runs succeed)
>
> changes with v7:
> tests: undo the restructuring of done in v6, in favour of explicitly
> forcing steps:
> '-c rootfs -f'. since concurrency of the autobuilder, and reusing
> artifacts/sstate-caching is probably the issue?
>
> changes with v8:
> tests: fixed missing feature=PACKAGE_CLASSES during
> 'test_image_gen_pkgdbfs'
> which is now split into three, one for each of the three available
> packagemanager: ipk, deb, rpm
>
>
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com/
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#199434):
https://lists.openembedded.org/g/openembedded-core/message/199434
Mute This Topic: https://lists.openembedded.org/mt/106051545/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v8 0/3] pkg-database and systemd-sysext image
Hello, It seems that this reliably fails oe-selftest-armhost. I didn't really pay attention until now beause we had other issues but in this build, those are the only failing tests: https://autobuilder.yoctoproject.org/typhoon/#/builders/127/builds/3322/steps/15/logs/stdio On 12/05/2024 08:31:56+0200, Johannes Schneider via lists.openembedded.org wrote: > systemd-sysext allows to overlay another image (or multiple) ontop of > a "base-image" = the current rootfs, via the use of overlayfs; to add > tools and features meant for development purposes. > > To quote the documentation on systemd-sysext: > " ...addition in order to make debugging/development easier). System > extension images should not be misunderstood as a generic software > packaging framework, ..." > > To build a lean image, that only holds packages that are not already > part of the base-image, a snapshot of the package-database is taken > after the installation of the base-rootfs is done, and picked up again > when collecting the rootfs of such a extension image. > > with all this in place an example usage could look like this: > some-core-image.bb > inherit core-image > IMAGE_GEN_PKGDBFS = "1" > > extending-image.bb > inherit image-sysext > IMAGE_FSTYPES = "squashfs" > IMAGE_BASE_PKGDB = "some-core-image" > # the above pointing at a package-db similar to: > # > build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz > > then on the device, running some-core-image, with the extension image placed > at FN: > $> ln -s "$FN" /run/extensions/$(basename $FN).raw > $> systemd-sysext list > $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge > > As long as the VERSION_ID of the extension image matches the os-release > in the base image, the above commands return sucessfully; > for details on the compativility check see the docs for systemd-sysext. > > = > > changes with v2: > rebase from 'kirkstone' onto 'master' > > changes with v3: > incorporate review suggestions for simplification > add task dependency handling > add oe-selftest for the pkgdb handling > add variable documentation and > some more comments, and examples in the commit-msg > > changes with v4: > rebase onto 'master' => no functional changes > fixed patchtest findings > > changes with v5: > rebase onto 'master' > add '.sysext' to the deployed symlink name > sidenote on the tests and autobuilder failure: run locally they > succeed, e.g.: > #> oe-selftest --verbose -r > imagefeatures.ImageFeatures.test_image_gen_pkgdbfs > > changes with v6: > tests: restructure to call 'bitbake' only once in the testcase itself > (in hopes of solving the autobuilder problem; local test runs succeed) > > changes with v7: > tests: undo the restructuring of done in v6, in favour of explicitly > forcing steps: > '-c rootfs -f'. since concurrency of the autobuilder, and reusing > artifacts/sstate-caching is probably the issue? > > changes with v8: > tests: fixed missing feature=PACKAGE_CLASSES during > 'test_image_gen_pkgdbfs' > which is now split into three, one for each of the three available > packagemanager: ipk, deb, rpm > > > -- Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#199429): https://lists.openembedded.org/g/openembedded-core/message/199429 Mute This Topic: https://lists.openembedded.org/mt/106051545/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v8 0/3] pkg-database and systemd-sysext image
systemd-sysext allows to overlay another image (or multiple) ontop of a "base-image" = the current rootfs, via the use of overlayfs; to add tools and features meant for development purposes. To quote the documentation on systemd-sysext: " ...addition in order to make debugging/development easier). System extension images should not be misunderstood as a generic software packaging framework, ..." To build a lean image, that only holds packages that are not already part of the base-image, a snapshot of the package-database is taken after the installation of the base-rootfs is done, and picked up again when collecting the rootfs of such a extension image. with all this in place an example usage could look like this: some-core-image.bb inherit core-image IMAGE_GEN_PKGDBFS = "1" extending-image.bb inherit image-sysext IMAGE_FSTYPES = "squashfs" IMAGE_BASE_PKGDB = "some-core-image" # the above pointing at a package-db similar to: # build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz then on the device, running some-core-image, with the extension image placed at FN: $> ln -s "$FN" /run/extensions/$(basename $FN).raw $> systemd-sysext list $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge As long as the VERSION_ID of the extension image matches the os-release in the base image, the above commands return sucessfully; for details on the compativility check see the docs for systemd-sysext. = changes with v2: rebase from 'kirkstone' onto 'master' changes with v3: incorporate review suggestions for simplification add task dependency handling add oe-selftest for the pkgdb handling add variable documentation and some more comments, and examples in the commit-msg changes with v4: rebase onto 'master' => no functional changes fixed patchtest findings changes with v5: rebase onto 'master' add '.sysext' to the deployed symlink name sidenote on the tests and autobuilder failure: run locally they succeed, e.g.: #> oe-selftest --verbose -r imagefeatures.ImageFeatures.test_image_gen_pkgdbfs changes with v6: tests: restructure to call 'bitbake' only once in the testcase itself (in hopes of solving the autobuilder problem; local test runs succeed) changes with v7: tests: undo the restructuring of done in v6, in favour of explicitly forcing steps: '-c rootfs -f'. since concurrency of the autobuilder, and reusing artifacts/sstate-caching is probably the issue? changes with v8: tests: fixed missing feature=PACKAGE_CLASSES during 'test_image_gen_pkgdbfs' which is now split into three, one for each of the three available packagemanager: ipk, deb, rpm -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#199209): https://lists.openembedded.org/g/openembedded-core/message/199209 Mute This Topic: https://lists.openembedded.org/mt/106051545/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
