Re: [OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image
Hoi, thanks for pointing the build failure out, Im trying to reproduce it locally but fail so far to see the problem :-S rebased on the latest master, when the two selftests added with the patches are run selecively, they pass: #> oe-selftest -r imagefeatures.ImageFeatures.test_image_gen_pkgdbfs imagefeatures.ImageFeatures.test_image_use_pkgdbfs but running the whole test-routine, same way the buildserver does in that log, currently fails me for other reasons (lack of resources? or i might be calling it the wrong way) any hints/ideas/pointers would be appreciated :-) gruß Johannes From: Alexandre Belloni Sent: Saturday, March 23, 2024 14:40 To: SCHNEIDER Johannes Cc: openembedded-core@lists.openembedded.org; richard.pur...@linuxfoundation.org Subject: Re: [OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image [You don't often get email from alexandre.bell...@bootlin.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] This email is not from Hexagon’s Office 365 instance. Please be careful while clicking links, opening attachments, or replying to this email. Hello, The series causes failures on the autobuilders: https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6548/steps/14/logs/stdio 2024-03-21 21:05:49,848 - oe-selftest - INFO - 6: 20/46 500/587 (137.73s) (0 failed) (imagefeatures.ImageFeatures.test_image_gen_pkgdbfs) 2024-03-21 21:05:49,848 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 327, in test_image_gen_pkgdbfs self.assertTrue(os.path.exists(os.path.join(img_vars['DEPLOY_DIR_IMAGE'], 'var/lib/opkg/status')), 'opkg\'s status file was not present in: %s' % tar_file) File "/usr/lib64/python3.9/unittest/case.py", line 688, in assertTrue raise self.failureException(msg) AssertionError: False is not true : opkg's status file was not present in: /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-3332915/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-pkgdb.tar.gz On 04/03/2024 07:15:50+0100, Johannes Schneider via lists.openembedded.org wrote: > systemd-sysext allows to overlay another image (or multiple) ontop of > a "base-image" = the current rootfs, via the use of overlayfs; to add > tools and features meant for development purposes. > > To quote the documentation on systemd-sysext: > " ...addition in order to make debugging/development easier). System > extension images should not be misunderstood as a generic software > packaging framework, ..." > > To build a lean image, that only holds packages that are not already > part of the base-image, a snapshot of the package-database is taken > after the installation of the base-rootfs is done, and picked up again > when collecting the rootfs of such a extension image. > > with all this in place an example usage could look like this: > some-core-image.bb > inherit core-image > IMAGE_GEN_PKGDBFS = "1" > > extending-image.bb > inherit image-sysext > IMAGE_FSTYPES = "squashfs" > IMAGE_BASE_PKGDB = "some-core-image" > # the above pointing at a package-db similar to: > # > build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz > > then on the device, running some-core-image, with the extension image placed > at FN: > $> ln -s "$FN" /run/extensions/$(basename $FN).raw > $> systemd-sysext list > $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge > > As long as the VERSION_ID of the extension image matches the os-release > in the base image, the above commands return sucessfully; > for details on the compativility check see the docs for systemd-sysext. > > = > > changes with v2: > rebase from 'kirkstone' onto 'master' > > changes with v3; > incorporate review suggestions for simplification > add task dependency handling > add oe-selftest for the pkgdb handling > add variable documentation and > some more comments, and examples in the commit-msg > > changes with v4; > rebase onto 'master' => no functional changes > fixed patchtest findings > > > -- Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com/ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#197485): https://lists.openembedded.org/g/openembedded-core/message/197485 Mute This Topic: https://lists.openembedded.org/mt/104717642/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image
Hello, The series causes failures on the autobuilders: https://autobuilder.yoctoproject.org/typhoon/#/builders/79/builds/6548/steps/14/logs/stdio 2024-03-21 21:05:49,848 - oe-selftest - INFO - 6: 20/46 500/587 (137.73s) (0 failed) (imagefeatures.ImageFeatures.test_image_gen_pkgdbfs) 2024-03-21 21:05:49,848 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last): File "/home/pokybuild/yocto-worker/oe-selftest-centos/build/meta/lib/oeqa/selftest/cases/imagefeatures.py", line 327, in test_image_gen_pkgdbfs self.assertTrue(os.path.exists(os.path.join(img_vars['DEPLOY_DIR_IMAGE'], 'var/lib/opkg/status')), 'opkg\'s status file was not present in: %s' % tar_file) File "/usr/lib64/python3.9/unittest/case.py", line 688, in assertTrue raise self.failureException(msg) AssertionError: False is not true : opkg's status file was not present in: /home/pokybuild/yocto-worker/oe-selftest-centos/build/build-st-3332915/tmp/deploy/images/qemux86-64/core-image-minimal-qemux86-64.rootfs-pkgdb.tar.gz On 04/03/2024 07:15:50+0100, Johannes Schneider via lists.openembedded.org wrote: > systemd-sysext allows to overlay another image (or multiple) ontop of > a "base-image" = the current rootfs, via the use of overlayfs; to add > tools and features meant for development purposes. > > To quote the documentation on systemd-sysext: > " ...addition in order to make debugging/development easier). System > extension images should not be misunderstood as a generic software > packaging framework, ..." > > To build a lean image, that only holds packages that are not already > part of the base-image, a snapshot of the package-database is taken > after the installation of the base-rootfs is done, and picked up again > when collecting the rootfs of such a extension image. > > with all this in place an example usage could look like this: > some-core-image.bb > inherit core-image > IMAGE_GEN_PKGDBFS = "1" > > extending-image.bb > inherit image-sysext > IMAGE_FSTYPES = "squashfs" > IMAGE_BASE_PKGDB = "some-core-image" > # the above pointing at a package-db similar to: > # > build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz > > then on the device, running some-core-image, with the extension image placed > at FN: > $> ln -s "$FN" /run/extensions/$(basename $FN).raw > $> systemd-sysext list > $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge > > As long as the VERSION_ID of the extension image matches the os-release > in the base image, the above commands return sucessfully; > for details on the compativility check see the docs for systemd-sysext. > > = > > changes with v2: > rebase from 'kirkstone' onto 'master' > > changes with v3; > incorporate review suggestions for simplification > add task dependency handling > add oe-selftest for the pkgdb handling > add variable documentation and > some more comments, and examples in the commit-msg > > changes with v4; > rebase onto 'master' => no functional changes > fixed patchtest findings > > > -- Alexandre Belloni, co-owner and COO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#197452): https://lists.openembedded.org/g/openembedded-core/message/197452 Mute This Topic: https://lists.openembedded.org/mt/104717642/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image
Hoi Alex, thanks for the heads up - i'll patiently wait then :-) gruß Johannes From: Alexander Kanavin Sent: Monday, March 4, 2024 10:25 To: SCHNEIDER Johannes Cc: openembedded-core@lists.openembedded.org; richard.pur...@linuxfoundation.org Subject: Re: [OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image [You don't often get email from alex.kana...@gmail.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] This email is not from Hexagon’s Office 365 instance. Please be careful while clicking links, opening attachments, or replying to this email. There is a feature freeze now, so this might have to wait for after that. Alex On Mon, 4 Mar 2024 at 07:16, Johannes Schneider via lists.openembedded.org wrote: > > systemd-sysext allows to overlay another image (or multiple) ontop of > a "base-image" = the current rootfs, via the use of overlayfs; to add > tools and features meant for development purposes. > > To quote the documentation on systemd-sysext: > " ...addition in order to make debugging/development easier). System > extension images should not be misunderstood as a generic software > packaging framework, ..." > > To build a lean image, that only holds packages that are not already > part of the base-image, a snapshot of the package-database is taken > after the installation of the base-rootfs is done, and picked up again > when collecting the rootfs of such a extension image. > > with all this in place an example usage could look like this: > some-core-image.bb > inherit core-image > IMAGE_GEN_PKGDBFS = "1" > > extending-image.bb > inherit image-sysext > IMAGE_FSTYPES = "squashfs" > IMAGE_BASE_PKGDB = "some-core-image" > # the above pointing at a package-db similar to: > # > build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz > > then on the device, running some-core-image, with the extension image placed > at FN: > $> ln -s "$FN" /run/extensions/$(basename $FN).raw > $> systemd-sysext list > $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge > > As long as the VERSION_ID of the extension image matches the os-release > in the base image, the above commands return sucessfully; > for details on the compativility check see the docs for systemd-sysext. > > = > > changes with v2: > rebase from 'kirkstone' onto 'master' > > changes with v3; > incorporate review suggestions for simplification > add task dependency handling > add oe-selftest for the pkgdb handling > add variable documentation and > some more comments, and examples in the commit-msg > > changes with v4; > rebase onto 'master' => no functional changes > fixed patchtest findings > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#196593): https://lists.openembedded.org/g/openembedded-core/message/196593 Mute This Topic: https://lists.openembedded.org/mt/104717642/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image
There is a feature freeze now, so this might have to wait for after that. Alex On Mon, 4 Mar 2024 at 07:16, Johannes Schneider via lists.openembedded.org wrote: > > systemd-sysext allows to overlay another image (or multiple) ontop of > a "base-image" = the current rootfs, via the use of overlayfs; to add > tools and features meant for development purposes. > > To quote the documentation on systemd-sysext: > " ...addition in order to make debugging/development easier). System > extension images should not be misunderstood as a generic software > packaging framework, ..." > > To build a lean image, that only holds packages that are not already > part of the base-image, a snapshot of the package-database is taken > after the installation of the base-rootfs is done, and picked up again > when collecting the rootfs of such a extension image. > > with all this in place an example usage could look like this: > some-core-image.bb > inherit core-image > IMAGE_GEN_PKGDBFS = "1" > > extending-image.bb > inherit image-sysext > IMAGE_FSTYPES = "squashfs" > IMAGE_BASE_PKGDB = "some-core-image" > # the above pointing at a package-db similar to: > # > build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz > > then on the device, running some-core-image, with the extension image placed > at FN: > $> ln -s "$FN" /run/extensions/$(basename $FN).raw > $> systemd-sysext list > $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge > > As long as the VERSION_ID of the extension image matches the os-release > in the base image, the above commands return sucessfully; > for details on the compativility check see the docs for systemd-sysext. > > = > > changes with v2: > rebase from 'kirkstone' onto 'master' > > changes with v3; > incorporate review suggestions for simplification > add task dependency handling > add oe-selftest for the pkgdb handling > add variable documentation and > some more comments, and examples in the commit-msg > > changes with v4; > rebase onto 'master' => no functional changes > fixed patchtest findings > > > -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#196592): https://lists.openembedded.org/g/openembedded-core/message/196592 Mute This Topic: https://lists.openembedded.org/mt/104717642/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v4 0/3] pkg-database and systemd-sysext image
systemd-sysext allows to overlay another image (or multiple) ontop of a "base-image" = the current rootfs, via the use of overlayfs; to add tools and features meant for development purposes. To quote the documentation on systemd-sysext: " ...addition in order to make debugging/development easier). System extension images should not be misunderstood as a generic software packaging framework, ..." To build a lean image, that only holds packages that are not already part of the base-image, a snapshot of the package-database is taken after the installation of the base-rootfs is done, and picked up again when collecting the rootfs of such a extension image. with all this in place an example usage could look like this: some-core-image.bb inherit core-image IMAGE_GEN_PKGDBFS = "1" extending-image.bb inherit image-sysext IMAGE_FSTYPES = "squashfs" IMAGE_BASE_PKGDB = "some-core-image" # the above pointing at a package-db similar to: # build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz then on the device, running some-core-image, with the extension image placed at FN: $> ln -s "$FN" /run/extensions/$(basename $FN).raw $> systemd-sysext list $> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge As long as the VERSION_ID of the extension image matches the os-release in the base image, the above commands return sucessfully; for details on the compativility check see the docs for systemd-sysext. = changes with v2: rebase from 'kirkstone' onto 'master' changes with v3; incorporate review suggestions for simplification add task dependency handling add oe-selftest for the pkgdb handling add variable documentation and some more comments, and examples in the commit-msg changes with v4; rebase onto 'master' => no functional changes fixed patchtest findings -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#196588): https://lists.openembedded.org/g/openembedded-core/message/196588 Mute This Topic: https://lists.openembedded.org/mt/104717642/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
