[OE-core][dunfell][PATCH] tzdata: Upgrade to 2024a

[Priyal Doshi via lists.openembedded.org]

From: Priyal Doshi 

Signed-off-by: Priyal Doshi 
---
 meta/recipes-extended/timezone/timezone.inc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/recipes-extended/timezone/timezone.inc 
b/meta/recipes-extended/timezone/timezone.inc
index 75f13cf..46bc1b7 100644
--- a/meta/recipes-extended/timezone/timezone.inc
+++ b/meta/recipes-extended/timezone/timezone.inc
@@ -6,7 +6,7 @@ SECTION = "base"
 LICENSE = "PD & BSD-3-Clause"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=c679c9d6b02bc2757b3eaf8f53c43fba"
 
-PV = "2023d"
+PV = "2024a"
 
 SRC_URI =" 
http://www.iana.org/time-zones/repository/releases/tzcode${PV}.tar.gz;name=tzcode
 \

http://www.iana.org/time-zones/repository/releases/tzdata${PV}.tar.gz;name=tzdata
 \
@@ -14,5 +14,5 @@ SRC_URI =" 
http://www.iana.org/time-zones/repository/releases/tzcode${PV}.tar.gz
 
 UPSTREAM_CHECK_URI = "http://www.iana.org/time-zones;
 
-SRC_URI[tzcode.sha256sum] = 
"e9a5f9e118886d2de92b62bb05510a28cc6c058d791c93bd6b84d3292c3c161e"
-SRC_URI[tzdata.sha256sum] = 
"dbca21970b0a8b8c0ceceec1d7b91fa903be0f6eca5ae732b5329672232a08f3"
+SRC_URI[tzcode.sha256sum] = 
"80072894adff5a458f1d143e16e4ca1d8b2a122c9c5399da482cb68cba6a1ff8"
+SRC_URI[tzdata.sha256sum] = 
"0d0434459acbd2059a7a8da1f3304a84a86591f6ed69c6248fffa502b6edffe3"
-- 
2.7.4


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196006): 
https://lists.openembedded.org/g/openembedded-core/message/196006
Mute This Topic: https://lists.openembedded.org/mt/104503456/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [poky] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Mark Hatle]

On 2/21/24 9:06 AM, Paul Barker wrote:

On 21/02/2024 10:57, Ross Burton wrote:

From: Ross Burton 

This is a new 64-bit "generic" Arm machine, that expects the hardware to
be SystemReady IR compatible. This is slightly forward-leaning as there's
not a _lot_ of SystemReady hardware in the wild, but most modern boards
are and the number will only grow.  Also, this is the only way to have a
'generic' machine as without standardised bootloaders and firmware it
would be impossible.

The base machine configuration isn't that exciting: it's a fully featured
machine that supports most things, booting via UEFI and an initramfs.

However, the kernel is more interesting.  This RFC uses the upstream defconfig
because unlike some other platforms, the arm64 defconfig is actively
maintained with the goal of being a 'boots on most hardware' configuration.
My argument is: why would we duplicate that effort?

The "linux-yocto way" is configuration fragments and after a week of
hair-pulling I do actually have fragments that boot on a BeaglePlay, but
to say this was a tiresome and frustrating exercise would be understating it.

So, a request for comments: is it acceptable to use the upstream defconfig in
a reference BSP?  Personally I'm torn: the Yocto way is fragments not monolithic
configs, but repeating the effort to fragmentise the configuration and then
also have it sufficiently modular that it can be used in pieces - instead of
just being a large file split up into smaller files - is a lot of effort for
what might end up being minimal gain.  My fear is we end up with a fragmented
configuration that can't be easily modified without breaking some platforms,
and badly copies what the defconfig already does.


I am in favour of this - I think the "genericarm64" machine should use
the in-tree defconfig so that it can support the widest array of
hardware. If someone wants to trim down the kernel for a particular
platform then they should probably create a specific MACHINE anyway.

If we take the other approach of building up the kernel config from
fragments, how would we know that all SystemReady IR capable systems
will be supported? Yocto Project doesn't have the resources to test
every platform.


I disagree here.  I think it would be MUCH better to have a 'SystemReady IR' 
hardware configuration.  So if SystemReady IR is desired, it is something that 
anyone can enable (starting with genericarm64).  Remember the defconfig is going 
to have more then hardware configs in it.  Will it have the right systemd 
configurations?  Will is have the magic filesystem a random user wants?  Will 
avoid having some other filesystem type that another user doesn't want?


Building up the kernel, and considering SystemReady IR as a 'hardware feature', 
and then add in the additional things that are needed for whatever reason is a 
much more reasonable way to do this and make it useful to otthers.



For the Renesas RZ SoCs I work on these days, the in-tree defconfig is
the configuration we test with the mainline kernel.


AMD does the same thing, for the kernel development it makes sense.  Kernel is 
built and tested standalone from userspace.


But with that said, I think it's the wrong way to do Yocto Project development. 
Yocto Project development needs further control and the separation of hardware 
and software configurations is pretty essential to having a system that can be 
customized appropriately.


The defconfig can be used as a guide to the other configurations, but separating 
hardware and software configs is a necessary first step in my opinion.


--Mark


Thanks,






-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196005): 
https://lists.openembedded.org/g/openembedded-core/message/196005
Mute This Topic: https://lists.openembedded.org/mt/104502773/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] Patchtest results for [PATCH v3 1/3] image.bbclass/rootfs: archive and deploy package database

[Patchtest]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch 
/home/patchtest/share/mboxes/v3-1-3-image.bbclass-rootfs-archive-and-deploy-package-database.patch

FAIL: test max line length: Patch line too long (current length 230, maximum is 
200) (test_metadata.TestMetadata.test_max_line_length)

PASS: pretest pylint (test_python_pylint.PyLint.pretest_pylint)
PASS: test Signed-off-by presence 
(test_mbox.TestMbox.test_signed_off_by_presence)
PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence 
(test_mbox.TestMbox.test_commit_message_presence)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test pylint (test_python_pylint.PyLint.test_pylint)
PASS: test shortlog format (test_mbox.TestMbox.test_shortlog_format)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)

SKIP: pretest src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes, skipping test 
(test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new CVE patches introduced 
(test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced 
(test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced 
(test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found 
(test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, 
skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_license_presence)
SKIP: test series merge on head: Merge test is disabled for now 
(test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing 
lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196004): 
https://lists.openembedded.org/g/openembedded-core/message/196004
Mute This Topic: https://lists.openembedded.org/mt/104500668/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 3/3] classes: add a systemd-sysext image class

[Johannes Schneider via lists.openembedded.org]

systemd-sysext can load a raw-image containing usr/ and opt/ folders
to mount them as RO overlay over the rootfs, to "extend" the systems.

This class provides the necessary changes/additions to the enclosed
file-system so that systemd-sysext accepts the extension for "merge"
into the rootfs.

With such an created image, placed into the correct folder (see [1]),
`systemd-sysext list` should be able to list the "extension" and
`systemd-sysext merge` should enable the overlay. On both commands a
preceding "SYSTEMD_LOG_LEVEL=debug" can aide in figuring out what is
amiss.

The strict name checking systemd-sysext does against the name of
extension-release.NAME file, is disabled, as there is only one such in
the resulting image. This is done to allow a user to freely rename the
resulting image file.
Note that for e.g. squashfs, the kernel needs CONFIG_SQUASHFS_XATTR=y

Link: 
https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html
Link: 
https://0pointer.net/blog/testing-my-system-code-in-usr-without-modifying-usr.html
Signed-off-by: Johannes Schneider 
---
 meta/classes-recipe/image-sysext.bbclass | 42 
 1 file changed, 42 insertions(+)
 create mode 100644 meta/classes-recipe/image-sysext.bbclass

diff --git a/meta/classes-recipe/image-sysext.bbclass 
b/meta/classes-recipe/image-sysext.bbclass
new file mode 100644
index 00..012db83353
--- /dev/null
+++ b/meta/classes-recipe/image-sysext.bbclass
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: MIT
+#
+# Copyright Leica Geosystems AG
+#
+
+# systemd-sysext [1] has a simple mechanism for version compatibility:
+# the extension to be loaded has to contain a
+# /usr/lib/extension-release.d/extension-release.NAME
+# with "NAME" *exactly* matching the filename of the extensions
+# raw-device filename/
+#
+# from the extension-release file the "ID" and "VERSION_ID" fields are
+# matched against the etc/os-release and the extension is only "merged"
+# if no mismatches between NAME, ID, and VERSION_ID.
+#
+# Link: 
https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html
+
+inherit image
+
+IMAGE_NAME_SUFFIX = ".sysext"
+EXTENSION_NAME = "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${IMAGE_FSTYPES}"
+
+DEPENDS += " os-release"
+
+sysext_image_mangle_rootfs() {
+R=${IMAGE_ROOTFS}
+
+# pull a copy of the rootfs version information, which systemd-sysext 
matches against
+cp -av ${RECIPE_SYSROOT}/${nonarch_libdir}/os-release 
${WORKDIR}/extension-release.base
+
+echo 'EXTENSION_RELOAD_MANAGER=1' >> ${WORKDIR}/extension-release.base
+
+install -d $R${nonarch_libdir}/extension-release.d
+install -m 0644 ${WORKDIR}/extension-release.base \
+
$R${nonarch_libdir}/extension-release.d/extension-release.${EXTENSION_NAME}
+
+# disable systemd-sysext's strict name checking, so that the image file 
can be renamed, while still being 'merge'-able
+setfattr -n user.extension-release.strict -v false \
+
$R${nonarch_libdir}/extension-release.d/extension-release.${EXTENSION_NAME}
+}
+
+ROOTFS_POSTPROCESS_COMMAND += " sysext_image_mangle_rootfs; "
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196003): 
https://lists.openembedded.org/g/openembedded-core/message/196003
Mute This Topic: https://lists.openembedded.org/mt/104500433/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH] classes: add a systemd-sysext image class

[Johannes Schneider via lists.openembedded.org]

systemd-sysext can load a raw-image containing usr/ and opt/ folders
to mount them as RO overlay over the rootfs, to "extend" the systems.

This class provides the necessary changes/additions to the enclosed
file-system so that systemd-sysext accepts the extension for "merge"
into the rootfs.

With such an created image, placed into the correct folder (see [1]),
`systemd-sysext list` should be able to list the "extension" and
`systemd-sysext merge` should enable the overlay. On both commands a
preceding "SYSTEMD_LOG_LEVEL=debug" can aide in figuring out what is
amiss.

The strict name checking systemd-sysext does against the name of
extension-release.NAME file, is disabled, as there is only one such in
the resulting image. This is done to allow a user to freely rename the
resulting image file.
Note that for e.g. squashfs, the kernel needs CONFIG_SQUASHFS_XATTR=y

Link: 
https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html
Link: 
https://0pointer.net/blog/testing-my-system-code-in-usr-without-modifying-usr.html
Signed-off-by: Johannes Schneider 
---
 meta/classes-recipe/image-sysext.bbclass | 42 
 1 file changed, 42 insertions(+)
 create mode 100644 meta/classes-recipe/image-sysext.bbclass

diff --git a/meta/classes-recipe/image-sysext.bbclass 
b/meta/classes-recipe/image-sysext.bbclass
new file mode 100644
index 00..012db83353
--- /dev/null
+++ b/meta/classes-recipe/image-sysext.bbclass
@@ -0,0 +1,42 @@
+# SPDX-License-Identifier: MIT
+#
+# Copyright Leica Geosystems AG
+#
+
+# systemd-sysext [1] has a simple mechanism for version compatibility:
+# the extension to be loaded has to contain a
+# /usr/lib/extension-release.d/extension-release.NAME
+# with "NAME" *exactly* matching the filename of the extensions
+# raw-device filename/
+#
+# from the extension-release file the "ID" and "VERSION_ID" fields are
+# matched against the etc/os-release and the extension is only "merged"
+# if no mismatches between NAME, ID, and VERSION_ID.
+#
+# Link: 
https://www.freedesktop.org/software/systemd/man/latest/systemd-sysext.html
+
+inherit image
+
+IMAGE_NAME_SUFFIX = ".sysext"
+EXTENSION_NAME = "${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${IMAGE_FSTYPES}"
+
+DEPENDS += " os-release"
+
+sysext_image_mangle_rootfs() {
+R=${IMAGE_ROOTFS}
+
+# pull a copy of the rootfs version information, which systemd-sysext 
matches against
+cp -av ${RECIPE_SYSROOT}/${nonarch_libdir}/os-release 
${WORKDIR}/extension-release.base
+
+echo 'EXTENSION_RELOAD_MANAGER=1' >> ${WORKDIR}/extension-release.base
+
+install -d $R${nonarch_libdir}/extension-release.d
+install -m 0644 ${WORKDIR}/extension-release.base \
+
$R${nonarch_libdir}/extension-release.d/extension-release.${EXTENSION_NAME}
+
+# disable systemd-sysext's strict name checking, so that the image file 
can be renamed, while still being 'merge'-able
+setfattr -n user.extension-release.strict -v false \
+
$R${nonarch_libdir}/extension-release.d/extension-release.${EXTENSION_NAME}
+}
+
+ROOTFS_POSTPROCESS_COMMAND += " sysext_image_mangle_rootfs; "
-- 
2.34.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#196002): 
https://lists.openembedded.org/g/openembedded-core/message/196002
Mute This Topic: https://lists.openembedded.org/mt/104500432/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 2/3] image.bbclass/rootfs: set and unpack package-database

[Johannes Schneider via lists.openembedded.org]

set the package-database of a "lower image" to unpack and build upon
when installing packages for the current image. This way a lean image
will be created, which only holds the packages that are not already
present in the lower image.

An image build such could then be used with overlayfs or systemd-
sysext to extend the "lower image" on demand; for development purposes
on a device running the "lower image" in RO mode for example.

A configuration could look as follows:
  some-core-image.bb
inherit image
IMAGE_GEN_PKGDBFS = "1"

  extending-image.bb
inherit image
IMAGE_BASE_PKGDB = "some-core-image"

Signed-off-by: Johannes Schneider 
---
 meta/classes-recipe/image.bbclass | 23 --
 meta/conf/documentation.conf  |  3 +-
 meta/lib/oe/package_manager/deb/rootfs.py |  2 +
 meta/lib/oe/package_manager/ipk/rootfs.py |  6 ++-
 meta/lib/oe/package_manager/rpm/rootfs.py |  7 ++-
 meta/lib/oe/rootfs.py | 18 
 meta/lib/oeqa/selftest/cases/imagefeatures.py | 46 +++
 7 files changed, 97 insertions(+), 8 deletions(-)

diff --git a/meta/classes-recipe/image.bbclass 
b/meta/classes-recipe/image.bbclass
index 3ccaaa17b8..c573c37cd8 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -42,8 +42,16 @@ IMAGE_FEATURES ?= ""
 IMAGE_FEATURES[type] = "list"
 IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs 
read-only-rootfs-delayed-postinsts stateless-rootfs empty-root-password 
allow-empty-password allow-root-login serial-autologin-root 
post-install-logging overlayfs-etc"
 
-# Generate snapshot of the package database?
+# Image layering:
+# a "base image" would create a snapshot of the package-database after the
+# installation of all packages into the rootfs is done. The next/other image
+# "layered on-top" of the former would then import that database and install
+# further packages; without reinstalling packages/dependencies that are already
+# installed in the layer below.
+# Set to '1' in a "base image" recipe, to preserve a snapshot of the package 
database.
 IMAGE_GEN_PKGDBFS ?= "0"
+# "PN" of a "base image", upon which the current image is to be built upon.
+IMAGE_BASE_PKGDB ?= ""
 
 # Generate companion debugfs?
 IMAGE_GEN_DEBUGFS ?= "0"
@@ -118,6 +126,15 @@ do_rootfs[depends] += " \
 "
 do_rootfs[recrdeptask] += "do_packagedata"
 
+python () {
+# make sure that the 'base image' has been queued in before this
+# image wants to unpack and build upon the formers pgkdb
+base_image = d.getVar('IMAGE_BASE_PKGDB')
+pn = d.getVar('PN')
+if base_image and base_image != pn:
+d.appendVarFlag("do_rootfs", 'depends', ' '+ base_image + 
':do_image_complete')
+}
+
 def rootfs_command_variables(d):
 return 
['ROOTFS_POSTPROCESS_COMMAND','ROOTFS_PREPROCESS_COMMAND','ROOTFS_POSTINSTALL_COMMAND','ROOTFS_POSTUNINSTALL_COMMAND','OPKG_PREPROCESS_COMMANDS','OPKG_POSTPROCESS_COMMANDS','IMAGE_POSTPROCESS_COMMAND',
 
'IMAGE_PREPROCESS_COMMAND','RPM_PREPROCESS_COMMANDS','RPM_POSTPROCESS_COMMANDS','DEB_PREPROCESS_COMMANDS','DEB_POSTPROCESS_COMMANDS']
@@ -134,8 +151,8 @@ def rootfs_variables(d):
  
'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS',
 'IMAGE_LINGUAS_COMPLEMENTARY', 'IMAGE_LOCALES_ARCHIVE',
  
'MULTILIBRE_ALLOW_REP','MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS',
  
'PACKAGE_ARCHS','PACKAGE_CLASSES','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','USE_DEVFS',
- 'CONVERSIONTYPES', 'IMAGE_GEN_PKGDBFS', 'IMAGE_GEN_DEBUGFS', 
'ROOTFS_RO_UNNEEDED', 'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 
'REPRODUCIBLE_TIMESTAMP_ROOTFS',
- 'IMAGE_INSTALL_DEBUGFS']
+ 'CONVERSIONTYPES', 'IMAGE_GEN_PKGDBFS', 'IMAGE_BASE_PKGDB', 
'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 'IMGDEPLOYDIR', 
'PACKAGE_EXCLUDE_COMPLEMENTARY',
+ 'REPRODUCIBLE_TIMESTAMP_ROOTFS', 'IMAGE_INSTALL_DEBUGFS']
 variables.extend(rootfs_command_variables(d))
 variables.extend(variable_depends(d))
 return " ".join(variables)
diff --git a/meta/conf/documentation.conf b/meta/conf/documentation.conf
index 7ec6f07dfc..9013556c04 100644
--- a/meta/conf/documentation.conf
+++ b/meta/conf/documentation.conf
@@ -208,6 +208,7 @@ ICECC_PATH[doc] = "The location of the icecc binary."
 ICECC_CLASS_DISABLE[doc] = "Identifies user classes that you do not want the 
Icecream distributed compile support to consider."
 ICECC_RECIPE_DISABLE[doc] = "Identifies user recipes that you do not want the 
Icecream distributed compile support to consider."
 ICECC_RECIPE_ENABLE[doc] = "Identifies user recipes that use 

[OE-core] [PATCH v3 0/3] pkg-database and systemd-sysext image

[Johannes Schneider via lists.openembedded.org]

systemd-sysext allows to overlay another image (or multiple) ontop of
a "base-image" = the current rootfs, via the use of overlayfs; to add
tools and features meant for development purposes.

To quote the documentation on systemd-sysext:
" ...addition in order to make debugging/development easier). System
extension images should not be misunderstood as a generic software
packaging framework, ..."

To build a lean image, that only holds packages that are not already
part of the base-image, a snapshot of the package-database is taken
after the installation of the base-rootfs is done, and picked up again
when collecting the rootfs of such a extension image.

with all this in place an example usage could look like this:
some-core-image.bb
  inherit core-image
  IMAGE_GEN_PKGDBFS = "1"

extending-image.bb
  inherit image-sysext
  IMAGE_FSTYPES = "squashfs"
  IMAGE_BASE_PKGDB = "some-core-image"
  # the above pointing at a package-db similar to:
  # 
build/deploy/images/$MACHINE/some-core-image-$MACHINE-20240210172305-pkgdb.rootfs.tar.gz

then on the device, running some-core-image, with the extension image placed at 
FN:
$> ln -s "$FN" /run/extensions/$(basename $FN).raw
$> systemd-sysext list
$> SYSTEMD_LOG_LEVEL=debug systemd-sysext merge

As long as the VERSION_ID of the extension image matches the os-release
in the base image, the above commands return sucessfully;
for details on the compativility check see the docs for systemd-sysext.

=

changes with v2:
rebase from 'kirkstone' onto 'master'

changes with v3;
incorporate review suggestions for simplification
add task dependency handling
add oe-selftest for the pkgdb handling
add variable documentation and
some more comments, and examples in the commit-msg

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195999): 
https://lists.openembedded.org/g/openembedded-core/message/195999
Mute This Topic: https://lists.openembedded.org/mt/104500428/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [PATCH v3 1/3] image.bbclass/rootfs: archive and deploy package database

[Johannes Schneider via lists.openembedded.org]

archive the package database after the rootfs has been put together as
*rootfs-pkdbfs.tar.gz, and put it into the deploy folder.

This creates a snapshot of the package mangers state at the point in
time when all dependencies have been resolved and installed; which
could be used by "extension images" to built upon.

Signed-off-by: Johannes Schneider 
---
 meta/classes-recipe/image.bbclass | 44 -
 meta/classes-recipe/image_types.bbclass   |  1 +
 meta/conf/documentation.conf  |  1 +
 meta/lib/oe/package_manager/deb/rootfs.py |  1 +
 meta/lib/oe/package_manager/ipk/rootfs.py |  1 +
 meta/lib/oe/package_manager/rpm/rootfs.py |  1 +
 meta/lib/oe/rootfs.py | 20 
 meta/lib/oeqa/selftest/cases/imagefeatures.py | 47 +++
 8 files changed, 115 insertions(+), 1 deletion(-)

diff --git a/meta/classes-recipe/image.bbclass 
b/meta/classes-recipe/image.bbclass
index 28be6c6362..3ccaaa17b8 100644
--- a/meta/classes-recipe/image.bbclass
+++ b/meta/classes-recipe/image.bbclass
@@ -42,6 +42,9 @@ IMAGE_FEATURES ?= ""
 IMAGE_FEATURES[type] = "list"
 IMAGE_FEATURES[validitems] += "debug-tweaks read-only-rootfs 
read-only-rootfs-delayed-postinsts stateless-rootfs empty-root-password 
allow-empty-password allow-root-login serial-autologin-root 
post-install-logging overlayfs-etc"
 
+# Generate snapshot of the package database?
+IMAGE_GEN_PKGDBFS ?= "0"
+
 # Generate companion debugfs?
 IMAGE_GEN_DEBUGFS ?= "0"
 
@@ -131,7 +134,8 @@ def rootfs_variables(d):
  
'IMAGE_ROOTFS_MAXSIZE','IMAGE_NAME','IMAGE_LINK_NAME','IMAGE_MANIFEST','DEPLOY_DIR_IMAGE','IMAGE_FSTYPES','IMAGE_INSTALL_COMPLEMENTARY','IMAGE_LINGUAS',
 'IMAGE_LINGUAS_COMPLEMENTARY', 'IMAGE_LOCALES_ARCHIVE',
  
'MULTILIBRE_ALLOW_REP','MULTILIB_TEMP_ROOTFS','MULTILIB_VARIANTS','MULTILIBS','ALL_MULTILIB_PACKAGE_ARCHS','MULTILIB_GLOBAL_VARIANTS','BAD_RECOMMENDATIONS','NO_RECOMMENDATIONS',
  
'PACKAGE_ARCHS','PACKAGE_CLASSES','TARGET_VENDOR','TARGET_ARCH','TARGET_OS','OVERRIDES','BBEXTENDVARIANT','FEED_DEPLOYDIR_BASE_URI','INTERCEPT_DIR','USE_DEVFS',
- 'CONVERSIONTYPES', 'IMAGE_GEN_DEBUGFS', 'ROOTFS_RO_UNNEEDED', 
'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 
'REPRODUCIBLE_TIMESTAMP_ROOTFS', 'IMAGE_INSTALL_DEBUGFS']
+ 'CONVERSIONTYPES', 'IMAGE_GEN_PKGDBFS', 'IMAGE_GEN_DEBUGFS', 
'ROOTFS_RO_UNNEEDED', 'IMGDEPLOYDIR', 'PACKAGE_EXCLUDE_COMPLEMENTARY', 
'REPRODUCIBLE_TIMESTAMP_ROOTFS',
+ 'IMAGE_INSTALL_DEBUGFS']
 variables.extend(rootfs_command_variables(d))
 variables.extend(variable_depends(d))
 return " ".join(variables)
@@ -337,6 +341,17 @@ python do_image_qa_setscene () {
 }
 addtask do_image_qa_setscene
 
+def setup_pkgdbfs_variables(d):
+d.appendVar('IMAGE_ROOTFS', '-pkgdb')
+if d.getVar('IMAGE_LINK_NAME'):
+d.appendVar('IMAGE_LINK_NAME', '-pkgdb')
+d.appendVar('IMAGE_NAME','-pkgdb')
+d.setVar('IMAGE_FSTYPES', 'tar.gz')
+
+python setup_pkgdbfs () {
+setup_pkgdbfs_variables(d)
+}
+
 def setup_debugfs_variables(d):
 d.appendVar('IMAGE_ROOTFS', '-dbg')
 if d.getVar('IMAGE_LINK_NAME'):
@@ -381,6 +396,11 @@ python () {
 alltypes = d.getVar('IMAGE_FSTYPES').split()
 typedeps = {}
 
+if d.getVar('IMAGE_GEN_PKGDBFS') == "1":
+pkgdbfs_fstypes = ['tar.gz']
+for t in pkgdbfs_fstypes:
+alltypes.append("pkgdbfs_" + t)
+
 if d.getVar('IMAGE_GEN_DEBUGFS') == "1":
 debugfs_fstypes = d.getVar('IMAGE_FSTYPES_DEBUGFS').split()
 for t in debugfs_fstypes:
@@ -393,6 +413,10 @@ python () {
 basetypes[baset]= []
 if t not in basetypes[baset]:
 basetypes[baset].append(t)
+pkgdb = ""
+if t.startswith("pkgdbfs_"):
+t = t[8:]
+pkgdb = "pkgdbfs_"
 debug = ""
 if t.startswith("debugfs_"):
 t = t[8:]
@@ -401,6 +425,13 @@ python () {
 vardeps.add('IMAGE_TYPEDEP:' + t)
 if baset not in typedeps:
 typedeps[baset] = set()
+deps = [pkgdb + dep for dep in deps]
+for dep in deps:
+if dep not in alltypes:
+alltypes.append(dep)
+_add_type(dep)
+basedep = _image_base_type(dep)
+typedeps[baset].add(basedep)
 deps = [debug + dep for dep in deps]
 for dep in deps:
 if dep not in alltypes:
@@ -419,6 +450,7 @@ python () {
 
 maskedtypes = (d.getVar('IMAGE_TYPES_MASKED') or "").split()
 maskedtypes = [dbg + t for t in maskedtypes for dbg in ("", "debugfs_")]
+maskedtypes = [pkgdb + t for t in maskedtypes for pkgdb in ("", 
"pkgdbfs_")]
 
 for t in basetypes:
 vardeps = set()
@@ -430,6 +462,11 @@ python () {
 continue
 
 localdata = bb.data.createCopy(d)
+pkgdb = ""
+if t.startswith("pkgdbfs_"):
+

Re: [OE-core] [PATCH] sanity.bbclass: raise_sanity_error if /tmp is noexec

[Randy MacLeod via lists.openembedded.org]

On 2024-02-21 5:08 a.m., Alexander Kanavin via lists.openembedded.org wrote:

On Wed, 21 Feb 2024 at 10:48, Ross Burton  wrote:

You _can_ export TMPDIR but that has to be done on a per-recipe/class basis 
very carefully as TMPDIR means something else to Bitbake.

The problem is recipes that use mktemp to write files and execute them (be it 
shell scripts, or as a place to write C and then compile in the same 
directory).  These will be in /tmp (again, we can’t set TMPDIR because for 
foolish historical reasons, TMPDIR is used by bitbake).

We first noticed this with Meson where noexec /tmp meant the configure tests 
failed. We worked around it at the time by assigning TMPDIR when calling Meson, 
but since them Meson writes to its own build tree now.  This has been seen 
before though, but luckily noexec /tmp is fairly unusual so I doubt this will 
break many builds.

I'm actually curious where noexec /tmp can be observed. It does seem
rare, because I think it's the first time someone came up with a
sanity check for it. Perhaps it should be treated as a bug in that
respective environment/OS/container?



We've been using noexec /tmp since 2019 with few if any problems

using:

meta-anaconda
meta-aws
meta-browser
meta-clang
meta-cloud-services
meta-dpdk
meta-imx
meta-intel
meta-intel-qat
meta-iot-cloud
meta-lat
meta-mingw
meta-openembedded
meta-qt6
meta-raspberrypi
meta-realtime
meta-secure-core
meta-security
meta-selinux
meta-tensorflow
meta-virtualization
meta-xilinx
meta-xilinx-tools
meta-yocto


Michal, what problem are you seeing?

../Randy




Alex





--
# Randy MacLeod
# Wind River Linux

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195998): 
https://lists.openembedded.org/g/openembedded-core/message/195998
Mute This Topic: https://lists.openembedded.org/mt/104258828/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 13/13] scripts: python 3.12 regex

[Adrian Freihofer]

From: Adrian Freihofer 

All the regexes throw a warning like this:

WARNING: scripts/lib/recipetool/create_buildsys.py:140:
  SyntaxWarning: invalid escape sequence '\s'
  proj_re = re.compile('project\s*\(([^)]*)\)', re.IGNORECASE)

Python 3 interprets string literals as Unicode strings, and therefore
\s is treated as an escaped Unicode character which is not correct.
Declaring the RegEx pattern as a raw string instead of unicode is
required for Python 3.

Signed-off-by: Adrian Freihofer 
Signed-off-by: Richard Purdie 

Backported from master: 24b0ba00d4f0b4d9834f7693ecb6032dfc534a80

Signed-off-by: Adrian Freihofer 
---
 scripts/combo-layer   |  2 +-
 scripts/contrib/bbvars.py |  6 ++--
 scripts/contrib/convert-overrides.py  |  8 ++---
 scripts/lib/checklayer/__init__.py|  4 +--
 scripts/lib/recipetool/create.py  | 12 +++
 scripts/lib/recipetool/create_buildsys.py | 38 +++
 scripts/oe-check-sstate   |  2 +-
 scripts/oe-pkgdata-util   |  2 +-
 scripts/opkg-query-helper.py  |  2 +-
 9 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/scripts/combo-layer b/scripts/combo-layer
index 7f2020fca7..19ad32660d 100755
--- a/scripts/combo-layer
+++ b/scripts/combo-layer
@@ -483,7 +483,7 @@ def check_repo_clean(repodir):
 exit if repo is dirty
 """
 output=runcmd("git status --porcelain", repodir)
-r = re.compile('\?\? patch-.*/')
+r = re.compile(r'\?\? patch-.*/')
 dirtyout = [item for item in output.splitlines() if not r.match(item)]
 if dirtyout:
 logger.error("git repo %s is dirty, please fix it first", repodir)
diff --git a/scripts/contrib/bbvars.py b/scripts/contrib/bbvars.py
index 090133600b..a9cdf082ab 100755
--- a/scripts/contrib/bbvars.py
+++ b/scripts/contrib/bbvars.py
@@ -36,8 +36,8 @@ def bbvar_is_documented(var, documented_vars):
 def collect_documented_vars(docfiles):
 ''' Walk the docfiles and collect the documented variables '''
 documented_vars = []
-prog = re.compile(".*($|[^A-Z_])')
+prog = re.compile(r".*($|[^A-Z_])')
 for d in docfiles:
 with open(d) as f:
 documented_vars += var_prog.findall(f.read())
@@ -45,7 +45,7 @@ def collect_documented_vars(docfiles):
 return documented_vars
 
 def bbvar_doctag(var, docconf):
-prog = re.compile('^%s\[doc\] *= *"(.*)"' % (var))
+prog = re.compile(r'^%s\[doc\] *= *"(.*)"' % (var))
 if docconf == "":
 return "?"
 
diff --git a/scripts/contrib/convert-overrides.py 
b/scripts/contrib/convert-overrides.py
index 1939757f1b..c69acb4095 100755
--- a/scripts/contrib/convert-overrides.py
+++ b/scripts/contrib/convert-overrides.py
@@ -81,19 +81,19 @@ skip_ext = [".html", ".patch", ".m4", ".diff"] + 
args.skip_ext
 
 vars_re = {}
 for exp in vars:
-vars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + 
exp), r"\1:" + exp)
+vars_re[exp] = (re.compile(r'((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + 
exp), r"\1:" + exp)
 
 shortvars_re = {}
 for exp in shortvars:
-shortvars_re[exp] = (re.compile('((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' 
+ exp + '([\(\'"\s:])'), r"\1:" + exp + r"\3")
+shortvars_re[exp] = 
(re.compile(r'((^|[#\'"\s\-\+])[A-Za-z0-9_\-:${}\.]+)_' + exp + 
r'([\(\'"\s:])'), r"\1:" + exp + r"\3")
 
 package_re = {}
 for exp in packagevars:
-package_re[exp] = (re.compile('(^|[#\'"\s\-\+]+)' + exp + '_' + 
'([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
+package_re[exp] = (re.compile(r'(^|[#\'"\s\-\+]+)' + exp + r'_' + 
r'([$a-z"\'\s%\[<{\\\*].)'), r"\1" + exp + r":\2")
 
 # Other substitutions to make
 subs = {
-'r = re.compile("([^:]+):\s*(.*)")' : 'r = re.compile("(^.+?):\s+(.*)")',
+'r = re.compile(r"([^:]+):\s*(.*)")' : 'r = re.compile(r"(^.+?):\s+(.*)")',
 "val = d.getVar('%s_%s' % (var, pkg))" : "val = d.getVar('%s:%s' % (var, 
pkg))",
 "f.write('%s_%s: %s\\n' % (var, pkg, encode(val)))" : "f.write('%s:%s: 
%s\\n' % (var, pkg, encode(val)))",
 "d.getVar('%s_%s' % (scriptlet_name, pkg))" : "d.getVar('%s:%s' % 
(scriptlet_name, pkg))",
diff --git a/scripts/lib/checklayer/__init__.py 
b/scripts/lib/checklayer/__init__.py
index 938805289e..53f99dce1e 100644
--- a/scripts/lib/checklayer/__init__.py
+++ b/scripts/lib/checklayer/__init__.py
@@ -324,8 +324,8 @@ def get_signatures(builddir, failsafe=False, machine=None, 
extravars=None):
 else:
 raise
 
-sig_regex = re.compile("^(?P.*:.*):(?P.*) .$")
-tune_regex = re.compile("(^|\s)SIGGEN_LOCKEDSIGS_t-(?P\S*)\s*=\s*")
+sig_regex = re.compile(r"^(?P.*:.*):(?P.*) .$")
+tune_regex = re.compile(r"(^|\s)SIGGEN_LOCKEDSIGS_t-(?P\S*)\s*=\s*")
 current_tune = None
 with open(sigs_file, 'r') as f:
 for line in f.readlines():
diff --git a/scripts/lib/recipetool/create.py b/scripts/lib/recipetool/create.py
index 67894fb4d0..7b4c501456 100644
--- 

[OE-core] [kirkstone][PATCH v2 12/13] meta/recipes: python 3.12 regex

[Adrian Freihofer]

From: Adrian Freihofer 

Python 3 interprets string literals as Unicode strings, and therefore
\s is treated as an escaped Unicode character which is not correct.
Declaring the RegEx pattern as a raw string instead of unicode is
required for Python 3.

Signed-off-by: Adrian Freihofer 
Signed-off-by: Richard Purdie 

Cherry-picked from master: f2d80817baea298b953d6e14daad65087b3b50c9

Signed-off-by: Adrian Freihofer 
---
 meta/recipes-kernel/perf/perf/sort-pmuevents.py | 8 
 meta/recipes-rt/rt-tests/files/rt_bmark.py  | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/recipes-kernel/perf/perf/sort-pmuevents.py 
b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
index 0362f2d8fa..0a87e553ab 100755
--- a/meta/recipes-kernel/perf/perf/sort-pmuevents.py
+++ b/meta/recipes-kernel/perf/perf/sort-pmuevents.py
@@ -36,10 +36,10 @@ with open(infile, 'r') as file:
 preamble_regex = re.compile( '^(.*?)^(struct|const struct|static struct|static 
const struct)', re.MULTILINE | re.DOTALL )
 
 preamble = re.search( preamble_regex, data )
-struct_block_regex = re.compile( '^(struct|const struct|static struct|static 
const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
-field_regex =  re.compile( '{.*?},', re.MULTILINE | re.DOTALL )
-cpuid_regex = re.compile( '\.cpuid = (.*?),', re.MULTILINE | re.DOTALL )
-name_regex = re.compile( '\.name = (.*?),', re.MULTILINE | re.DOTALL )
+struct_block_regex = re.compile(r'^(struct|const struct|static struct|static 
const struct).*?(\w+) (.*?)\[\] = {(.*?)^};', re.MULTILINE | re.DOTALL )
+field_regex =  re.compile(r'{.*?},', re.MULTILINE | re.DOTALL )
+cpuid_regex = re.compile(r'\.cpuid = (.*?),', re.MULTILINE | re.DOTALL )
+name_regex = re.compile(r'\.name = (.*?),', re.MULTILINE | re.DOTALL )
 
 # create a dictionary structure to store all the structs, their
 # types and then their fields.
diff --git a/meta/recipes-rt/rt-tests/files/rt_bmark.py 
b/meta/recipes-rt/rt-tests/files/rt_bmark.py
index 3b84447a0f..2a4eed412f 100755
--- a/meta/recipes-rt/rt-tests/files/rt_bmark.py
+++ b/meta/recipes-rt/rt-tests/files/rt_bmark.py
@@ -265,7 +265,7 @@ cmd = ("cyclictest",
"-d", str(interval_delta),
"-l", str(loop_count)
)
-rex = re.compile(b"C:\s*(\d+).*Min:\s*(\d+).*Avg:\s*(\d+).*Max:\s*(\d+)")
+rex = re.compile(r"C:\s*(\d+).*Min:\s*(\d+).*Avg:\s*(\d+).*Max:\s*(\d+)")
 
 def run_cyclictest_once():
 res = subprocess.check_output(cmd)
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195996): 
https://lists.openembedded.org/g/openembedded-core/message/195996
Mute This Topic: https://lists.openembedded.org/mt/104495347/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 11/13] meta/lib/oeqa: python 3.12 regex

[Adrian Freihofer]

From: Adrian Freihofer 

Python 3 interprets string literals as Unicode strings, and therefore
\s is treated as an escaped Unicode character which is not correct.
Declaring the RegEx pattern as a raw string instead of unicode is
required for Python 3.

Signed-off-by: Adrian Freihofer 
Signed-off-by: Richard Purdie 

Cherry-picked from master: 9002850f0c2e409d3bc629e36bb360b96326bb64

Signed-off-by: Adrian Freihofer 
---
 meta/lib/oeqa/oetest.py  | 2 +-
 meta/lib/oeqa/selftest/cases/bblayers.py | 2 +-
 meta/lib/oeqa/selftest/cases/fitimage.py | 6 +++---
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 9c84466dd0..b53c611062 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -256,7 +256,7 @@ class TestContext(object):
 
 modules = []
 for test in self.testslist:
-if re.search("\w+\.\w+\.test_\S+", test):
+if re.search(r"\w+\.\w+\.test_\S+", test):
 test = '.'.join(t.split('.')[:3])
 module = pkgutil.get_loader(test)
 modules.append(module)
diff --git a/meta/lib/oeqa/selftest/cases/bblayers.py 
b/meta/lib/oeqa/selftest/cases/bblayers.py
index 7d74833f61..0b9f16eeae 100644
--- a/meta/lib/oeqa/selftest/cases/bblayers.py
+++ b/meta/lib/oeqa/selftest/cases/bblayers.py
@@ -46,7 +46,7 @@ class BitbakeLayers(OESelftestTestCase):
 bb_file = os.path.join(testoutdir, recipe_path, recipe_file)
 self.assertTrue(os.path.isfile(bb_file), msg = "Cannot find 
xcursor-transparent-theme_0.1.1.bb in the test_bitbakelayers_flatten local 
dir.")
 contents = ftools.read_file(bb_file)
-find_in_contents = re.search("# bbappended from meta-selftest 
#\n(.*\n)*include test_recipe.inc", contents)
+find_in_contents = re.search(r"# bbappended from meta-selftest 
#\n(.*\n)*include test_recipe.inc", contents)
 self.assertTrue(find_in_contents, msg = "Flattening layers did not 
work. bitbake-layers flatten output: %s" % result.output)
 
 def test_bitbakelayers_add_remove(self):
diff --git a/meta/lib/oeqa/selftest/cases/fitimage.py 
b/meta/lib/oeqa/selftest/cases/fitimage.py
index d732a9020d..4d820faf92 100644
--- a/meta/lib/oeqa/selftest/cases/fitimage.py
+++ b/meta/lib/oeqa/selftest/cases/fitimage.py
@@ -202,7 +202,7 @@ UBOOT_MKIMAGE_SIGN_ARGS = "-c 'a smart comment'"
 signed_sections = {}
 for line in result.output.splitlines():
 if line.startswith((' Configuration', ' Image')):
-in_signed = re.search('\((.*)\)', line).groups()[0]
+in_signed = re.search(r'\((.*)\)', line).groups()[0]
 elif re.match('^ *', line) in (' ', ''):
 in_signed = None
 elif in_signed:
@@ -521,7 +521,7 @@ UBOOT_FIT_HASH_ALG = "sha256"
 signed_sections = {}
 for line in result.output.splitlines():
 if line.startswith((' Image')):
-in_signed = re.search('\((.*)\)', line).groups()[0]
+in_signed = re.search(r'\((.*)\)', line).groups()[0]
 elif re.match(' \w', line):
 in_signed = None
 elif in_signed:
@@ -675,7 +675,7 @@ FIT_SIGN_INDIVIDUAL = "1"
 signed_sections = {}
 for line in result.output.splitlines():
 if line.startswith((' Image')):
-in_signed = re.search('\((.*)\)', line).groups()[0]
+in_signed = re.search(r'\((.*)\)', line).groups()[0]
 elif re.match(' \w', line):
 in_signed = None
 elif in_signed:
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195995): 
https://lists.openembedded.org/g/openembedded-core/message/195995
Mute This Topic: https://lists.openembedded.org/mt/104495344/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 09/13] oeqa/selftest/oelib/buildhistory: git default branch

[Adrian Freihofer]

From: Adrian Freihofer 

On hosts with git defaulting to main branch the following exception
occures:

File .../buildhistory.py", line 99, in test_compare_dict_blobs_default
  blob1 = self.repo.heads.master.commit.tree.blobs[0]
  ^^
File "/usr/lib/python3.12/site-packages/git/util.py", line 1114, in __getattr__
  return list.__getattribute__(self, attr)
 ^
AttributeError: 'IterableList' object has no attribute 'master'

Support main and master branch for these test cases.

Note: setting the default branch with --initial-branch requires git
version 2.28 or later. Some of the still supported host distros do not
provide this feature yet.

Signed-off-by: Adrian Freihofer 
Signed-off-by: Richard Purdie 

Cherry-picked from master: 7df99843d8f31d8e0c2872ff625f4a5abf28f740

Signed-off-by: Adrian Freihofer 
---
 .../oeqa/selftest/cases/oelib/buildhistory.py  | 18 ++
 1 file changed, 14 insertions(+), 4 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/oelib/buildhistory.py 
b/meta/lib/oeqa/selftest/cases/oelib/buildhistory.py
index 33bd6df2f3..ae12aa0865 100644
--- a/meta/lib/oeqa/selftest/cases/oelib/buildhistory.py
+++ b/meta/lib/oeqa/selftest/cases/oelib/buildhistory.py
@@ -28,6 +28,16 @@ class TestBlobParsing(OESelftestTestCase):
 import shutil
 shutil.rmtree(self.repo_path)
 
+@property
+def heads_default(self):
+"""
+Support repos defaulting to master or to main branch
+"""
+try:
+return self.repo.heads.main
+except AttributeError:
+return self.repo.heads.master
+
 def commit_vars(self, to_add={}, to_remove = [], msg="A commit message"):
 if len(to_add) == 0 and len(to_remove) == 0:
 return
@@ -65,10 +75,10 @@ class TestBlobParsing(OESelftestTestCase):
 changesmap = { "foo-2" : ("2", "8"), "bar" : ("","4"), "bar-2" : 
("","5")}
 
 self.commit_vars(to_add = { "foo" : "1", "foo-2" : "2", "foo-3" : "3" 
})
-blob1 = self.repo.heads.master.commit.tree.blobs[0]
+blob1 = self.heads_default.commit.tree.blobs[0]
 
 self.commit_vars(to_add = { "foo-2" : "8", "bar" : "4", "bar-2" : "5" 
})
-blob2 = self.repo.heads.master.commit.tree.blobs[0]
+blob2 = self.heads_default.commit.tree.blobs[0]
 
 change_records = compare_dict_blobs(os.path.join(self.repo_path, 
self.test_file),
 blob1, blob2, False, False)
@@ -84,10 +94,10 @@ class TestBlobParsing(OESelftestTestCase):
 defaultmap = { x : ("default", "1")  for x in ["PKG", "PKGE", "PKGV", 
"PKGR"]}
 
 self.commit_vars(to_add = { "foo" : "1" })
-blob1 = self.repo.heads.master.commit.tree.blobs[0]
+blob1 = self.heads_default.commit.tree.blobs[0]
 
 self.commit_vars(to_add = { "PKG" : "1", "PKGE" : "1", "PKGV" : "1", 
"PKGR" : "1" })
-blob2 = self.repo.heads.master.commit.tree.blobs[0]
+blob2 = self.heads_default.commit.tree.blobs[0]
 
 change_records = compare_dict_blobs(os.path.join(self.repo_path, 
self.test_file),
 blob1, blob2, False, False)
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195993): 
https://lists.openembedded.org/g/openembedded-core/message/195993
Mute This Topic: https://lists.openembedded.org/mt/104495340/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 10/13] feature-microblaze-versions.inc: python 3.12 regex

[Adrian Freihofer]

From: Adrian Freihofer 

Python 3 interprets string literals as Unicode strings, and therefore
\s is treated as an escaped Unicode character which is not correct.
Declaring the RegEx pattern as a raw string instead of unicode is
required for Python 3.

Signed-off-by: Adrian Freihofer 

feature-microblaze-versions.inc#

Signed-off-by: Richard Purdie 

Cherry-picked from master: 662f52f1713c9f070550fc0c874eb62312218ea4

Signed-off-by: Adrian Freihofer 
---
 .../machine/include/microblaze/feature-microblaze-versions.inc  | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc 
b/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
index 5c37f49abb..658e87b8cd 100644
--- a/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
+++ b/meta/conf/machine/include/microblaze/feature-microblaze-versions.inc
@@ -16,7 +16,7 @@ def microblaze_current_version(d, gcc = False):
 # find the current version, and convert it to major/minor integers
 version = None
 for t in (d.getVar("TUNE_FEATURES") or "").split():
-m = re.search("^v(\d+)\.(\d+)", t)
+m = re.search(r"^v(\d+)\.(\d+)", t)
 if m:
 version = int(m.group(1)), int(m.group(2))
 break
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195994): 
https://lists.openembedded.org/g/openembedded-core/message/195994
Mute This Topic: https://lists.openembedded.org/mt/104495341/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 08/13] oeqa/selftest/recipetool: expect meson.bb

[Adrian Freihofer]

Latest recipetool from master branch generates a pyhton3-meson.bb recipe
while the older version from kirkstone generates a meson.bb. Change the
test to pass with meson.bb.

Signed-off-by: Adrian Freihofer 
---
 meta/lib/oeqa/selftest/cases/recipetool.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py 
b/meta/lib/oeqa/selftest/cases/recipetool.py
index c888770533..a2d8d292ad 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -444,7 +444,7 @@ class RecipetoolCreateTests(RecipetoolBase):
 # older release of Meson at present so we don't need a toml parser.
 temprecipe = os.path.join(self.tempdir, 'recipe')
 os.makedirs(temprecipe)
-recipefile = os.path.join(temprecipe, 'python3-meson_git.bb')
+recipefile = os.path.join(temprecipe, 'meson_git.bb')
 srcuri = 'https://github.com/mesonbuild/meson;rev=0.52.1'
 cmd = ['recipetool', 'create', '-o', temprecipe, srcuri]
 result = runCmd(cmd)
@@ -480,7 +480,7 @@ class RecipetoolCreateTests(RecipetoolBase):
 temprecipe = os.path.join(self.tempdir, 'recipe')
 os.makedirs(temprecipe)
 pv = '0.52.1'
-recipefile = os.path.join(temprecipe, 'python3-meson_%s.bb' % pv)
+recipefile = os.path.join(temprecipe, 'meson_%s.bb' % pv)
 srcuri = 
'https://github.com/mesonbuild/meson/releases/download/%s/meson-%s.tar.gz' % 
(pv, pv)
 result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
 self.assertTrue(os.path.isfile(recipefile))
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195992): 
https://lists.openembedded.org/g/openembedded-core/message/195992
Mute This Topic: https://lists.openembedded.org/mt/104495339/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 06/13] oeqa/selftest/recipetool: fix for python 3.12

[Adrian Freihofer]

From: Adrian Freihofer 

test_recipetool_create_github and test_recipetool_create_github_tarball
fail because the old meson version used by these tests cases does not
run on Python 3.12. The issue is in the dependencies.py which comes with
meson:
ERROR: build/tmp/work/recipetool-3z4osyl7/source/git/mesonbuild/
   dependencies.py:777: SyntaxWarning: invalid escape sequence '\.'

Use meson 1.3.1 (what is currently also used on master) as a reference
for these tests.

With this version of meson, recipetool creates recipes named
meson_git.bb or meson_1.3.1.bb. Since this looks more reasonable than
e.g. python3-meson_git.bb the test gets adapted.

Signed-off-by: Adrian Freihofer 
Signed-off-by: Richard Purdie 

Backported from master: 7374a8a2810a6cf027bfefefe87691a3529123ff

Signed-off-by: Adrian Freihofer 
---
 meta/lib/oeqa/selftest/cases/recipetool.py | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py 
b/meta/lib/oeqa/selftest/cases/recipetool.py
index db8790b57b..db21325155 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -444,13 +444,13 @@ class RecipetoolCreateTests(RecipetoolBase):
 temprecipe = os.path.join(self.tempdir, 'recipe')
 os.makedirs(temprecipe)
 recipefile = os.path.join(temprecipe, 'meson_git.bb')
-srcuri = 'https://github.com/mesonbuild/meson;rev=0.32.0'
+srcuri = 'https://github.com/mesonbuild/meson;rev=1.3.1'
 result = runCmd(['recipetool', 'create', '-o', temprecipe, srcuri])
 self.assertTrue(os.path.isfile(recipefile))
 checkvars = {}
-checkvars['LICENSE'] = set(['Apache-2.0'])
-checkvars['SRC_URI'] = 
'git://github.com/mesonbuild/meson;protocol=https;branch=master'
-inherits = ['setuptools3']
+checkvars['LICENSE'] = set(['Apache-2.0', 'Proprietary', 'Unknown'])
+checkvars['SRC_URI'] = 
'git://github.com/mesonbuild/meson;protocol=https;branch=1.3'
+inherits = ['python_setuptools_build_meta']
 self._test_recipe_contents(recipefile, checkvars, inherits)
 
 def test_recipetool_create_python3_setuptools(self):
@@ -476,15 +476,15 @@ class RecipetoolCreateTests(RecipetoolBase):
 # Basic test to ensure github URL mangling doesn't apply to release 
tarballs
 temprecipe = os.path.join(self.tempdir, 'recipe')
 os.makedirs(temprecipe)
-pv = '0.32.0'
+pv = '1.3.1'
 recipefile = os.path.join(temprecipe, 'meson_%s.bb' % pv)
 srcuri = 
'https://github.com/mesonbuild/meson/releases/download/%s/meson-%s.tar.gz' % 
(pv, pv)
 result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
 self.assertTrue(os.path.isfile(recipefile))
 checkvars = {}
-checkvars['LICENSE'] = set(['Apache-2.0'])
+checkvars['LICENSE'] = set(['Apache-2.0', 'Proprietary', 'Unknown'])
 checkvars['SRC_URI'] = 
'https://github.com/mesonbuild/meson/releases/download/${PV}/meson-${PV}.tar.gz'
-inherits = ['setuptools3']
+inherits = ['python_setuptools_build_meta']
 self._test_recipe_contents(recipefile, checkvars, inherits)
 
 def _test_recipetool_create_git(self, srcuri, branch=None):
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195990): 
https://lists.openembedded.org/g/openembedded-core/message/195990
Mute This Topic: https://lists.openembedded.org/mt/104495337/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 07/13] oeqa/selftest/recipetool: downgrade meson version to not use pyproject.toml

[Adrian Freihofer]

From: Ross Burton 

recipetool's pyproject.toml parsing needs tomllib (python 3.11+) or
tomli (not a hard dependency), so is prone to failing depending on the
host configuration.

Downgrade the Meson release used for the checks to 0.52.1, which was the
last release before moving to pyproject.toml.

Signed-off-by: Ross Burton 
Signed-off-by: Richard Purdie 

Backported from master: 6dfe573d83687e5431841f062442b54b9fa22ff3

Signed-off-by: Adrian Freihofer 
---
 meta/lib/oeqa/selftest/cases/recipetool.py | 29 --
 1 file changed, 16 insertions(+), 13 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/recipetool.py 
b/meta/lib/oeqa/selftest/cases/recipetool.py
index db21325155..c888770533 100644
--- a/meta/lib/oeqa/selftest/cases/recipetool.py
+++ b/meta/lib/oeqa/selftest/cases/recipetool.py
@@ -440,17 +440,19 @@ class RecipetoolCreateTests(RecipetoolBase):
 self._test_recipe_contents(recipefile, checkvars, inherits)
 
 def test_recipetool_create_github(self):
-# Basic test to see if github URL mangling works
+# Basic test to see if github URL mangling works. Deliberately use an
+# older release of Meson at present so we don't need a toml parser.
 temprecipe = os.path.join(self.tempdir, 'recipe')
 os.makedirs(temprecipe)
-recipefile = os.path.join(temprecipe, 'meson_git.bb')
-srcuri = 'https://github.com/mesonbuild/meson;rev=1.3.1'
-result = runCmd(['recipetool', 'create', '-o', temprecipe, srcuri])
-self.assertTrue(os.path.isfile(recipefile))
+recipefile = os.path.join(temprecipe, 'python3-meson_git.bb')
+srcuri = 'https://github.com/mesonbuild/meson;rev=0.52.1'
+cmd = ['recipetool', 'create', '-o', temprecipe, srcuri]
+result = runCmd(cmd)
+self.assertTrue(os.path.isfile(recipefile), msg="recipe %s not created 
for command %s, output %s" % (recipefile, " ".join(cmd), result.output))
 checkvars = {}
-checkvars['LICENSE'] = set(['Apache-2.0', 'Proprietary', 'Unknown'])
-checkvars['SRC_URI'] = 
'git://github.com/mesonbuild/meson;protocol=https;branch=1.3'
-inherits = ['python_setuptools_build_meta']
+checkvars['LICENSE'] = set(['Apache-2.0', "Unknown"])
+checkvars['SRC_URI'] = 
'git://github.com/mesonbuild/meson;protocol=https;branch=0.52'
+inherits = ['setuptools3']
 self._test_recipe_contents(recipefile, checkvars, inherits)
 
 def test_recipetool_create_python3_setuptools(self):
@@ -473,18 +475,19 @@ class RecipetoolCreateTests(RecipetoolBase):
 self._test_recipe_contents(recipefile, checkvars, inherits)
 
 def test_recipetool_create_github_tarball(self):
-# Basic test to ensure github URL mangling doesn't apply to release 
tarballs
+# Basic test to ensure github URL mangling doesn't apply to release 
tarballs.
+# Deliberately use an older release of Meson at present so we don't 
need a toml parser.
 temprecipe = os.path.join(self.tempdir, 'recipe')
 os.makedirs(temprecipe)
-pv = '1.3.1'
-recipefile = os.path.join(temprecipe, 'meson_%s.bb' % pv)
+pv = '0.52.1'
+recipefile = os.path.join(temprecipe, 'python3-meson_%s.bb' % pv)
 srcuri = 
'https://github.com/mesonbuild/meson/releases/download/%s/meson-%s.tar.gz' % 
(pv, pv)
 result = runCmd('recipetool create -o %s %s' % (temprecipe, srcuri))
 self.assertTrue(os.path.isfile(recipefile))
 checkvars = {}
-checkvars['LICENSE'] = set(['Apache-2.0', 'Proprietary', 'Unknown'])
+checkvars['LICENSE'] = set(['Apache-2.0'])
 checkvars['SRC_URI'] = 
'https://github.com/mesonbuild/meson/releases/download/${PV}/meson-${PV}.tar.gz'
-inherits = ['python_setuptools_build_meta']
+inherits = ['setuptools3']
 self._test_recipe_contents(recipefile, checkvars, inherits)
 
 def _test_recipetool_create_git(self, srcuri, branch=None):
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195991): 
https://lists.openembedded.org/g/openembedded-core/message/195991
Mute This Topic: https://lists.openembedded.org/mt/104495338/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 05/13] oeqa: replace deprecated assertEquals

[Adrian Freihofer]

From: Adrian Freihofer 

assertEquals is deprecated since Python 2.7:
https://docs.python.org/2/library/unittest.html#deprecated-aliases
It throws errors at least on Python 3.12. Replace it by assertEqual.

Signed-off-by: Adrian Freihofer 
Signed-off-by: Richard Purdie 

Backported from master: 68286d0b70cf09a0d2950b48945c9192fb8c8769

Signed-off-by: Adrian Freihofer 
---
 meta/lib/oeqa/sdk/buildtools-cases/sanity.py | 2 +-
 meta/lib/oeqa/selftest/cases/devtool.py  | 2 +-
 meta/lib/oeqa/selftest/cases/liboe.py| 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/lib/oeqa/sdk/buildtools-cases/sanity.py 
b/meta/lib/oeqa/sdk/buildtools-cases/sanity.py
index 64baaa8f84..68b19f4d47 100644
--- a/meta/lib/oeqa/sdk/buildtools-cases/sanity.py
+++ b/meta/lib/oeqa/sdk/buildtools-cases/sanity.py
@@ -19,4 +19,4 @@ class SanityTests(OESDKTestCase):
 # Canonicalise the location of this command
 tool_path = os.path.realpath(self._run("command -v %s" % 
command).strip())
 # Assert that the tool was found inside the SDK root
-self.assertEquals(os.path.commonprefix((sdk_base, tool_path)), 
sdk_base)
+self.assertEqual(os.path.commonprefix((sdk_base, tool_path)), 
sdk_base)
diff --git a/meta/lib/oeqa/selftest/cases/devtool.py 
b/meta/lib/oeqa/selftest/cases/devtool.py
index aea2ad6561..dc0fc35062 100644
--- a/meta/lib/oeqa/selftest/cases/devtool.py
+++ b/meta/lib/oeqa/selftest/cases/devtool.py
@@ -919,7 +919,7 @@ class DevtoolModifyTests(DevtoolBase):
 runCmd('git -C %s checkout %s' % (tempdir, branch))
 with open(source, "rt") as f:
 content = f.read()
-self.assertEquals(content, expected)
+self.assertEqual(content, expected)
 check('devtool', 'This is a test for something\n')
 check('devtool-no-overrides', 'This is a test for something\n')
 check('devtool-override-qemuarm', 'This is a test for qemuarm\n')
diff --git a/meta/lib/oeqa/selftest/cases/liboe.py 
b/meta/lib/oeqa/selftest/cases/liboe.py
index afe8f8809f..da88ff480e 100644
--- a/meta/lib/oeqa/selftest/cases/liboe.py
+++ b/meta/lib/oeqa/selftest/cases/liboe.py
@@ -97,6 +97,6 @@ class LibOE(OESelftestTestCase):
 
 dstcnt = len(os.listdir(dst))
 srccnt = len(os.listdir(src))
-self.assertEquals(dstcnt, len(testfiles), "Number of files in dst (%s) 
differs from number of files in src(%s)." % (dstcnt, srccnt))
+self.assertEqual(dstcnt, len(testfiles), "Number of files in dst (%s) 
differs from number of files in src(%s)." % (dstcnt, srccnt))
 
 oe.path.remove(testloc)
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195989): 
https://lists.openembedded.org/g/openembedded-core/message/195989
Mute This Topic: https://lists.openembedded.org/mt/104495336/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 03/13] populate_sdk_ext: use ConfigParser instead of SafeConfigParser

[Adrian Freihofer]

From: Ross Burton 

SafeConfigParser was renamed to ConfigParser in 3.2, and the
SafeConfigParser alias will be removed in 3.12.

Signed-off-by: Ross Burton 
Signed-off-by: Alexandre Belloni 

Cherry-picked from master: 71b3e7f71727137b4b996cc4160c9cc1581824b8

Signed-off-by: Adrian Freihofer 
---
 meta/classes/populate_sdk_ext.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/populate_sdk_ext.bbclass 
b/meta/classes/populate_sdk_ext.bbclass
index ca1b7753cb..bdd86863c6 100644
--- a/meta/classes/populate_sdk_ext.bbclass
+++ b/meta/classes/populate_sdk_ext.bbclass
@@ -227,7 +227,7 @@ python copy_buildsystem () {
 
 # Write out config file for devtool
 import configparser
-config = configparser.SafeConfigParser()
+config = configparser.ConfigParser()
 config.add_section('General')
 config.set('General', 'bitbake_subdir', conf_bbpath)
 config.set('General', 'init_path', conf_initpath)
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195987): 
https://lists.openembedded.org/g/openembedded-core/message/195987
Mute This Topic: https://lists.openembedded.org/mt/104495334/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 04/13] runqemu: add qmp socket support

[Adrian Freihofer]

From: Ross Burton 

Add support for qmp sockets and defaults to unix:qmp.sock if unspecified

Signed-off-by: Ross Burton 
Signed-off-by: Eilís 'pidge' Ní Fhlannagáin 
Signed-off-by: Richard Purdie 

Backported from master: 380631797f0d63124a8c21efa93ab672dbd79283
Qemu throws many warnings without qmp and many runtime tests fail
without this patch also on kirkstone.

Signed-off-by: Adrian Freihofer 
---
 scripts/runqemu | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/scripts/runqemu b/scripts/runqemu
index b7d7c7b4e7..b8c5adcbec 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -82,6 +82,7 @@ of the following environment variables (in any order):
 kvm-vhost - enable KVM with vhost when running x86/x86_64 (VT-capable CPU 
required)
 publicvnc - enable a VNC server open to all hosts
 audio - enable audio
+qmp= - create a QMP socket (defaults to unix:qmp.sock if unspecified)
 [*/]ovmf* - OVMF firmware file or base name for booting with UEFI
   tcpserial= - specify tcp serial port number
   qemuparams= - specify custom parameters to QEMU
@@ -216,6 +217,7 @@ class BaseConfig(object):
 self.cleaned = False
 # Files to cleanup after run
 self.cleanup_files = []
+self.qmp = None
 
 def acquire_taplock(self, error=True):
 logger.debug("Acquiring lockfile %s..." % self.taplock)
@@ -526,6 +528,10 @@ class BaseConfig(object):
 elif arg == 'publicvnc':
 self.publicvnc = True
 self.qemu_opt_script += ' -vnc :0'
+elif arg == "qmp":
+self.qmp = "unix:qmp.sock"
+elif arg.startswith("qmp="):
+self.qmp = arg[len('qmp='):]
 elif arg.startswith('tcpserial='):
 self.tcpserial_portnum = '%s' % arg[len('tcpserial='):]
 elif arg.startswith('qemuparams='):
@@ -1346,6 +1352,10 @@ class BaseConfig(object):
 raise RunQemuError("Failed to boot, QB_SYSTEM_NAME is NULL!")
 self.qemu_system = qemu_system
 
+def setup_qmp(self):
+if self.qmp:
+self.qemu_opt += " -qmp %s,server,nowait" % self.qmp
+
 def setup_vga(self):
 if self.nographic == True:
 if self.sdl == True:
@@ -1476,6 +1486,7 @@ class BaseConfig(object):
 if self.snapshot:
 self.qemu_opt += " -snapshot"
 
+self.setup_qmp()
 self.setup_serial()
 self.setup_vga()
 
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195988): 
https://lists.openembedded.org/g/openembedded-core/message/195988
Mute This Topic: https://lists.openembedded.org/mt/104495335/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 02/13] scripts/runqemu: fix regex escape sequences

[Adrian Freihofer]

From: Trevor Gamblin 

When invoking runqemu with Python 3.12, the following warning is
encountered:

|SyntaxWarning: invalid escape sequence '\.'

This is because the interpreter scans the string before it is processed
by the regex module, and it interprets the backslash as part of an
escape sequence, but not a standard one. This will be registered as an
error rather than a warning in future Python versions. To avoid the it,
simply add an extra backslash so that Python doesn't misinterpret the
string, while the regex parser still sees an escaped '.' character.

Signed-off-by: Trevor Gamblin 
Signed-off-by: Richard Purdie 

Backported from master: 0e8a4142bb90a92d175df6b2537d24a372356f98

Signed-off-by: Adrian Freihofer 
---
 scripts/runqemu | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/runqemu b/scripts/runqemu
index 729b067a9f..b7d7c7b4e7 100755
--- a/scripts/runqemu
+++ b/scripts/runqemu
@@ -362,7 +362,7 @@ class BaseConfig(object):
 if p.endswith('.qemuboot.conf'):
 self.qemuboot = p
 self.qbconfload = True
-elif re.search('\.bin$', p) or re.search('bzImage', p) or \
+elif re.search('\\.bin$', p) or re.search('bzImage', p) or \
  re.search('zImage', p) or re.search('vmlinux', p) or \
  re.search('fitImage', p) or re.search('uImage', p):
 self.kernel =  p
@@ -376,13 +376,13 @@ class BaseConfig(object):
 fst = t
 break
 if not fst:
-m = re.search('.*\.(.*)$', self.rootfs)
+m = re.search('.*\\.(.*)$', self.rootfs)
 if m:
 fst =  m.group(1)
 if fst:
 self.check_arg_fstype(fst)
-qb = re.sub('\.' + fst + "$", '', self.rootfs)
-qb = '%s%s' % (re.sub('\.rootfs$', '', qb), '.qemuboot.conf')
+qb = re.sub('\\.' + fst + "$", '', self.rootfs)
+qb = '%s%s' % (re.sub('\\.rootfs$', '', qb), '.qemuboot.conf')
 if os.path.exists(qb):
 self.qemuboot = qb
 self.qbconfload = True
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195986): 
https://lists.openembedded.org/g/openembedded-core/message/195986
Mute This Topic: https://lists.openembedded.org/mt/104495333/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 01/13] recipetool/create_buildsys_python: use importlib instead of imp

[Adrian Freihofer]

From: Chris Laplante 

'imp' was deprecated in Python 3.4 and removed in 3.12. The
piece of importlib we use has been around since 3.3.

Signed-off-by: Chris Laplante 
Signed-off-by: Alexandre Belloni 
Signed-off-by: Richard Purdie 

Cherry-picked from master: 457f0dad87b4e45a53865b5ad2c150215bd74019

Signed-off-by: Adrian Freihofer 
---
 scripts/lib/recipetool/create_buildsys_python.py | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/scripts/lib/recipetool/create_buildsys_python.py 
b/scripts/lib/recipetool/create_buildsys_python.py
index 5686a62d3f..a7eed3256f 100644
--- a/scripts/lib/recipetool/create_buildsys_python.py
+++ b/scripts/lib/recipetool/create_buildsys_python.py
@@ -10,7 +10,7 @@ import codecs
 import collections
 import setuptools.command.build_py
 import email
-import imp
+import importlib
 import glob
 import itertools
 import logging
@@ -561,7 +561,6 @@ class PythonRecipeHandler(RecipeHandler):
 return deps
 
 def parse_pkgdata_for_python_packages(self):
-suffixes = [t[0] for t in imp.get_suffixes()]
 pkgdata_dir = tinfoil.config_data.getVar('PKGDATA_DIR')
 
 ldata = tinfoil.config_data.createCopy()
@@ -585,7 +584,7 @@ class PythonRecipeHandler(RecipeHandler):
 continue
 
 for fn in files_info:
-for suffix in suffixes:
+for suffix in importlib.machinery.all_suffixes():
 if fn.endswith(suffix):
 break
 else:
-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195985): 
https://lists.openembedded.org/g/openembedded-core/message/195985
Mute This Topic: https://lists.openembedded.org/mt/104495332/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [kirkstone][PATCH v2 00/13] python 3.12 related fixes

[Adrian Freihofer]

Changes in comparison to v2:
Fix oe-selftest errors
- https://errors.yoctoproject.org/Errors/Details/753468/
- https://errors.yoctoproject.org/Errors/Details/753470/
by adding one more commit: "oeqa/selftest/recipetool: expect meson.bb"

oe-selftest -a
Required tests failed (successes=502, skipped=6, failures=7, errors=3)

eSDK.oeSDKExtSelfTest.test_image_generation_binary_feeds: ERROR
eSDK.oeSDKExtSelfTest.test_install_libraries_headers: ERROR
runcmd.RunCmdTests.test_result_assertion: ERROR (0.01s)
runcmd.RunCmdTests.test_result_exception: ERROR (0.02s)
distrodata.Distrodata.test_checkpkg: FAILED (132.52s)
incompatible_lic.NoGPL3InImagesTests.test_core_image_full_cmdline_weston: 
FAILED (215.33s)
runtime_test.TestImage.test_testimage_apt: FAILED (807.54s)
runtime_test.TestImage.test_testimage_dnf: FAILED (1032.04s)
runtime_test.TestImage.test_testimage_install: FAILED (110.76s)
runtime_test.TestImage.test_testimage_virgl_gtk_sdl: FAILED (724.11s)
runtime_test.TestImage.test_testimage_virgl_headless: FAILED (0.04s)

Note: Some opengl/weston tests are currently also failing on master
on my setup. Not sure why. Lets wait for the opinion of the official
Fedora 39 runner.

For what ever reason recipetool seams to work differently on my machine.
  recipetool create -o https://github.com/mesonbuild/meson;rev=0.52.1
creates a recipe named meson.bb, not a recipe pyhton3-meson.bb.
Not sure if another is a patch required which fixes two test cases:
  test_recipetool_create_github
  test_recipetool_create_github_tarball
or if this depends on the host machine.

Adrian Freihofer (8):
  oeqa: replace deprecated assertEquals
  oeqa/selftest/recipetool: fix for python 3.12
  oeqa/selftest/recipetool: expect meson.bb
  oeqa/selftest/oelib/buildhistory: git default branch
  feature-microblaze-versions.inc: python 3.12 regex
  meta/lib/oeqa: python 3.12 regex
  meta/recipes: python 3.12 regex
  scripts: python 3.12 regex

Chris Laplante (1):
  recipetool/create_buildsys_python: use importlib instead of imp

Ross Burton (3):
  populate_sdk_ext: use ConfigParser instead of SafeConfigParser
  runqemu: add qmp socket support
  oeqa/selftest/recipetool: downgrade meson version to not use
pyproject.toml

Trevor Gamblin (1):
  scripts/runqemu: fix regex escape sequences

 meta/classes/populate_sdk_ext.bbclass |  2 +-
 .../feature-microblaze-versions.inc   |  2 +-
 meta/lib/oeqa/oetest.py   |  2 +-
 meta/lib/oeqa/sdk/buildtools-cases/sanity.py  |  2 +-
 meta/lib/oeqa/selftest/cases/bblayers.py  |  2 +-
 meta/lib/oeqa/selftest/cases/devtool.py   |  2 +-
 meta/lib/oeqa/selftest/cases/fitimage.py  |  6 +--
 meta/lib/oeqa/selftest/cases/liboe.py |  2 +-
 .../oeqa/selftest/cases/oelib/buildhistory.py | 18 +++--
 meta/lib/oeqa/selftest/cases/recipetool.py| 19 ++
 .../perf/perf/sort-pmuevents.py   |  8 ++--
 meta/recipes-rt/rt-tests/files/rt_bmark.py|  2 +-
 scripts/combo-layer   |  2 +-
 scripts/contrib/bbvars.py |  6 +--
 scripts/contrib/convert-overrides.py  |  8 ++--
 scripts/lib/checklayer/__init__.py|  4 +-
 scripts/lib/recipetool/create.py  | 12 +++---
 scripts/lib/recipetool/create_buildsys.py | 38 +--
 .../lib/recipetool/create_buildsys_python.py  |  5 +--
 scripts/oe-check-sstate   |  2 +-
 scripts/oe-pkgdata-util   |  2 +-
 scripts/opkg-query-helper.py  |  2 +-
 scripts/runqemu   | 19 --
 23 files changed, 95 insertions(+), 72 deletions(-)

-- 
2.43.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195984): 
https://lists.openembedded.org/g/openembedded-core/message/195984
Mute This Topic: https://lists.openembedded.org/mt/104495331/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[paulg]

From: Paul Gortmaker 

[[RFC PATCH] Add genericarm64 MACHINE using upstream defconfig] On 21/02/2024 
(Wed 10:57) ross.bur...@arm.com wrote:

> From: Ross Burton 
> 
> This is a new 64-bit "generic" Arm machine, that expects the hardware to
> be SystemReady IR compatible. This is slightly forward-leaning as there's
> not a _lot_ of SystemReady hardware in the wild, but most modern boards
> are and the number will only grow.  Also, this is the only way to have a
> 'generic' machine as without standardised bootloaders and firmware it
> would be impossible.
> 
> The base machine configuration isn't that exciting: it's a fully featured
> machine that supports most things, booting via UEFI and an initramfs.
> 
> However, the kernel is more interesting.  This RFC uses the upstream defconfig
> because unlike some other platforms, the arm64 defconfig is actively
> maintained with the goal of being a 'boots on most hardware' configuration.
> My argument is: why would we duplicate that effort?

I have no problem with the idea of a genericarm64, but the defconfig
approach is not the way to go.

> The "linux-yocto way" is configuration fragments and after a week of
> hair-pulling I do actually have fragments that boot on a BeaglePlay, but
> to say this was a tiresome and frustrating exercise would be understating it.

Yes, if you start with a giant "defconfig" file and try and slowly chip
away at it, it can be frustrating and time consuming.  I never do that.

It is the same problem space as trying to turn core-image-full-cmdline
into core-image-minimal by removing one package at a time.  Always
easier to start with the bare minimum and then add stuff.

> So, a request for comments: is it acceptable to use the upstream defconfig in
> a reference BSP?  Personally I'm torn: the Yocto way is fragments not 
> monolithic

In a word, the answer is "no".

Let me insert a bit of history here - darn near 20y ago now - when we
were just starting out with linux, we had BSPs on different kernel
versions, and worse - the use of flat monolithic "defconfig" type files
on a per BSP basis.  The latter was to be blunt, just crap.  There was
no uniformity across BSPs in the same "family" - where family would be
something like CGL back in the 2005 era.

One BSP might have built iso9660 fs, another one not. Or one enabled
some debug option and another one didn't.  The inconsistency was just
horrible and unbearable.  BSPs were screwing with options they had no
business touching.

After a couple years of that mess, I believe several of us (which
included Bruce and myself) had just about enough.  What we came up with
was isolating the board config to a select subset of hardware/driver
Kconfig options and the "platform" would handle "non-harware" options.

That largely is still what exists in Yocto today, even though we were
not using Yocto waaay back then.  It immediately brought consistency
across platforms, and a stark clarity to exactly what the board required
for Kconfig settings - typically less than a screen full - choose the
CPU and the driver for video and disk/flash and USB chip etc.  Compare
that to the unreadable 10,000+ lines in a defconfig now.

So, yes I'll grant you that making that one screen full of core hardware
settings can be frustrating.  But at the same time, it has also served
as a valuable filter.  If a person can't distill a BSP down into the
core options it *really* needs, then they typically either don't
understand the hardware, or are just not invested enough to go that
extra mile.  I'm not saying either is the case here - just in general.

On top of all that - we've had similar discussions before along similar
lines - where Yocto/OE are not RedHat or Ubuntu - where we try and
deliver a distro with "one config works everywhere" kind of stuff.

I'm fine with genericarm64 - but please please don't encourage the chaos
and madness associated with random unmaintainable defconfigs!  Just
because it exists upstream doesn't mean it is good.

Thanks,
Paul.

> configs, but repeating the effort to fragmentise the configuration and then
> also have it sufficiently modular that it can be used in pieces - instead of
> just being a large file split up into smaller files - is a lot of effort for
> what might end up being minimal gain.  My fear is we end up with a fragmented
> configuration that can't be easily modified without breaking some platforms,
> and badly copies what the defconfig already does.
> 
> Ross
> ---
>  meta-yocto-bsp/README.hardware.md |  7 +
>  meta-yocto-bsp/conf/machine/genericarm64.conf | 26 +++
>  .../linux/linux-yocto_6.6.bbappend|  9 +++
>  meta-yocto-bsp/wic/genericarm64.wks.in| 11 
>  4 files changed, 53 insertions(+)
>  create mode 100644 meta-yocto-bsp/conf/machine/genericarm64.conf
>  create mode 100644 meta-yocto-bsp/wic/genericarm64.wks.in
> 
> diff --git a/meta-yocto-bsp/README.hardware.md 
> b/meta-yocto-bsp/README.hardware.md
> 

[OE-core] [PATCH] linux-yocto: Remove unused patch

[Khem Raj]

This patch remained after bumping from 6.1 to 6.6

Signed-off-by: Khem Raj 
---
 ...cpumap-Make-counter-as-unsigned-ints.patch | 69 ---
 1 file changed, 69 deletions(-)
 delete mode 100644 
meta/recipes-kernel/linux/files/0001-perf-cpumap-Make-counter-as-unsigned-ints.patch

diff --git 
a/meta/recipes-kernel/linux/files/0001-perf-cpumap-Make-counter-as-unsigned-ints.patch
 
b/meta/recipes-kernel/linux/files/0001-perf-cpumap-Make-counter-as-unsigned-ints.patch
deleted file mode 100644
index 2bfc40fe04f..000
--- 
a/meta/recipes-kernel/linux/files/0001-perf-cpumap-Make-counter-as-unsigned-ints.patch
+++ /dev/null
@@ -1,69 +0,0 @@
-From d14450f9e0f05ea7177c5404a7a9289352caab77 Mon Sep 17 00:00:00 2001
-From: Khem Raj 
-Date: Mon, 23 Jan 2023 13:04:10 -0800
-Subject: [PATCH] perf cpumap: Make counter as unsigned ints
-
-These are loop counters which is inherently unsigned. Therefore make
-them unsigned. Moreover it also fixes alloc-size-larger-than
-error with gcc-13, where malloc can be called with (-1) due to tmp_len
-being an int type.
-
-Fixes
-| cpumap.c:366:20: error: argument 1 range [18446744065119617024, 
18446744073709551612] exceeds maximum object size 9223372036854775807 
[-Werror=alloc-size-larger-than=]
-|   366 | tmp_cpus = malloc(tmp_len * sizeof(struct perf_cpu));
-|   |^
-
-Signed-off-by: Khem Raj 
-Cc: Peter Zijlstra 
-Cc: Ingo Molnar 
-Cc: Arnaldo Carvalho de Melo 
-Cc: Mark Rutland 
-Cc: Alexander Shishkin 
-Cc: Jiri Olsa 
-Cc: Namhyung Kim 
-
-Upstream-Status: Submitted 
[https://lore.kernel.org/linux-perf-users/20230123211310.127532-1-raj.k...@gmail.com/T/#u]

- tools/lib/perf/cpumap.c | 10 +-
- 1 file changed, 5 insertions(+), 5 deletions(-)
-
-diff --git a/tools/lib/perf/cpumap.c b/tools/lib/perf/cpumap.c
-index 6cd0be7c1bb4..d960880dd903 100644
 a/tools/lib/perf/cpumap.c
-+++ b/tools/lib/perf/cpumap.c
-@@ -351,8 +351,8 @@ struct perf_cpu_map *perf_cpu_map__merge(struct 
perf_cpu_map *orig,
-struct perf_cpu_map *other)
- {
-   struct perf_cpu *tmp_cpus;
--  int tmp_len;
--  int i, j, k;
-+  unsigned int tmp_len;
-+  unsigned int i, j, k;
-   struct perf_cpu_map *merged;
- 
-   if (perf_cpu_map__is_subset(orig, other))
-@@ -369,7 +369,7 @@ struct perf_cpu_map *perf_cpu_map__merge(struct 
perf_cpu_map *orig,
- 
-   /* Standard merge algorithm from wikipedia */
-   i = j = k = 0;
--  while (i < orig->nr && j < other->nr) {
-+  while (i < (unsigned int)orig->nr && j < (unsigned int)other->nr) {
-   if (orig->map[i].cpu <= other->map[j].cpu) {
-   if (orig->map[i].cpu == other->map[j].cpu)
-   j++;
-@@ -378,10 +378,10 @@ struct perf_cpu_map *perf_cpu_map__merge(struct 
perf_cpu_map *orig,
-   tmp_cpus[k++] = other->map[j++];
-   }
- 
--  while (i < orig->nr)
-+  while (i < (unsigned int)orig->nr)
-   tmp_cpus[k++] = orig->map[i++];
- 
--  while (j < other->nr)
-+  while (j < (unsigned int)other->nr)
-   tmp_cpus[k++] = other->map[j++];
-   assert(k <= tmp_len);
- 
--- 
-2.39.1
-
-- 
2.43.2


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195982): 
https://lists.openembedded.org/g/openembedded-core/message/195982
Mute This Topic: https://lists.openembedded.org/mt/104493713/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Richard Purdie]

On Wed, 2024-02-21 at 08:15 -0800, Anton Antonov wrote:
> On Wed, Feb 21, 2024 at 03:21 AM, Richard Purdie wrote:
> > I think it comes down to whether the fragments are usable and
> > testable.
> > We have a list of targets we want this new machine to run on so
> > lets
> > start with those, define genericarm64 as that set of fragments
> > combined
> > plus the generic pieces linux-yocto adds, then go from there. If
> > you
> > add a new machine to the test matrix, we add a new fragment. If
> > someone
> > wants to add new config, they need to show a machine using it.
> 
> Although Ross mentioned that there are not a lot of SystemReady IR
> compatible hardware in the wild, we're already talking about tens of
> them in existence. With this approach the genericarm64 machine will
> be compatible with only some of them unless we test all of the
> certified platforms and update kernel fragments accordingly. 
> The whole idea of SystemReady+defconfig is that it would allow to
> avoid future maintenance of kernel fragments in Yocto for existing
> and new SR certified platforms. If a platform is SystemReady
> certified (i.e. required drivers are up-streamed and mainline
> defconfig is updated) then the genericarm64 Yocto image would "just
> work".  On the last Yocto summit Bruce mentioned a tool which can
> automate defconfig -> kernel fragments conversion. Using this tool as
> a part of kernel versions updates in Yocto might solve the problem
> for genericarm64. But, I don't know how up to date and robust the
> tools is.
>
> Some additional information explaining requirements for genericarm64 
> - currently for SystemReady IR certification it is required that at
> least two of the main Linux distros (Fedora, Debian, Ubuntu,
> OpenSuse) generic arm64 images are bootable and functional. We would
> like to expand this list with Yocto and Openwrt as well. There is
> also a PR into Openwrt which adds a generic armsr target with the
> same defconfig approach to build the kernel.

I think the problem here is going to be defining which kinds of
configuration should come from where.

Let me pick for example, ZFS as a random kernel config option. I could
pick a USB Alcatel Speedtouch modem instead, the point is something
which doesn't really have hardware implications (but could have distro
ones).

I have no idea whether the "systemready defconfig" enables zfs or or
not. From a Yocto Project perspective that option is very much a
"distro" piece of configuration and not related to the "machine" or
hardware.

By using fragments, we can clearly keep something like that separate
and off limits. I don't know what policy this defconfig has to have
things enabled or disabled but if the policy is "supports anything and
everything", zfs and likely a lot of other things are probably enabled
that we wouldn't usually want in Yocto Project builds.

This is where the full defconfig becomes tricky since you get the bits
needed for the Systemready spec, but you potentially get all kinds of
other pieces too. You end turning on everything just in case.

Is there some policy which says whether zfs should or should not be
enabled in that defconfig?

There has to be some set of config options that systemready requires
and some that it doesn't care about and the real issue here is wanting
to know which are which.

Cheers,

Richard













-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195981): 
https://lists.openembedded.org/g/openembedded-core/message/195981
Mute This Topic: https://lists.openembedded.org/mt/104486073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Anton Antonov]

On Wed, Feb 21, 2024 at 03:21 AM, Richard Purdie wrote:

> 
> I think it comes down to whether the fragments are usable and testable.
> We have a list of targets we want this new machine to run on so lets
> start with those, define genericarm64 as that set of fragments combined
> plus the generic pieces linux-yocto adds, then go from there. If you
> add a new machine to the test matrix, we add a new fragment. If someone
> wants to add new config, they need to show a machine using it.

Although Ross mentioned that there are not a lot of SystemReady IR compatible 
hardware in the wild, we're already talking about tens of them in existence. 
With this approach the genericarm64 machine will be compatible with only some 
of them unless we test all of the certified platforms and update kernel 
fragments accordingly.

The whole idea of SystemReady+defconfig is that it would allow to avoid future 
maintenance of kernel fragments in Yocto for existing and new SR certified 
platforms. If a platform is SystemReady certified (i.e. required drivers are 
up-streamed and mainline defconfig is updated) then the genericarm64 Yocto 
image would "just work".  On the last Yocto summit Bruce mentioned a tool which 
can automate defconfig -> kernel fragments conversion. Using this tool as a 
part of kernel versions updates in Yocto might solve the problem for 
genericarm64. But, I don't know how up to date and robust the tools is.

Some additional information explaining requirements for genericarm64  - 
currently for SystemReady IR certification it is required that at least two of 
the main Linux distros (Fedora, Debian, Ubuntu, OpenSuse) generic arm64 images 
are bootable and functional. We would like to expand this list with Yocto and 
Openwrt as well. There is also a PR into Openwrt which adds a generic armsr 
target with the same defconfig approach to build the kernel.

Cheers,

Anton

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195980): 
https://lists.openembedded.org/g/openembedded-core/message/195980
Mute This Topic: https://lists.openembedded.org/mt/104486073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Paul Barker]

On 21/02/2024 10:57, Ross Burton wrote:
> From: Ross Burton 
> 
> This is a new 64-bit "generic" Arm machine, that expects the hardware to
> be SystemReady IR compatible. This is slightly forward-leaning as there's
> not a _lot_ of SystemReady hardware in the wild, but most modern boards
> are and the number will only grow.  Also, this is the only way to have a
> 'generic' machine as without standardised bootloaders and firmware it
> would be impossible.
> 
> The base machine configuration isn't that exciting: it's a fully featured
> machine that supports most things, booting via UEFI and an initramfs.
> 
> However, the kernel is more interesting.  This RFC uses the upstream defconfig
> because unlike some other platforms, the arm64 defconfig is actively
> maintained with the goal of being a 'boots on most hardware' configuration.
> My argument is: why would we duplicate that effort?
> 
> The "linux-yocto way" is configuration fragments and after a week of
> hair-pulling I do actually have fragments that boot on a BeaglePlay, but
> to say this was a tiresome and frustrating exercise would be understating it.
> 
> So, a request for comments: is it acceptable to use the upstream defconfig in
> a reference BSP?  Personally I'm torn: the Yocto way is fragments not 
> monolithic
> configs, but repeating the effort to fragmentise the configuration and then
> also have it sufficiently modular that it can be used in pieces - instead of
> just being a large file split up into smaller files - is a lot of effort for
> what might end up being minimal gain.  My fear is we end up with a fragmented
> configuration that can't be easily modified without breaking some platforms,
> and badly copies what the defconfig already does.

I am in favour of this - I think the "genericarm64" machine should use
the in-tree defconfig so that it can support the widest array of
hardware. If someone wants to trim down the kernel for a particular
platform then they should probably create a specific MACHINE anyway.

If we take the other approach of building up the kernel config from
fragments, how would we know that all SystemReady IR capable systems
will be supported? Yocto Project doesn't have the resources to test
every platform.

For the Renesas RZ SoCs I work on these days, the in-tree defconfig is
the configuration we test with the mainline kernel.

Thanks,

-- 
Paul Barker


OpenPGP_0x27F4B3459F002257.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195979): 
https://lists.openembedded.org/g/openembedded-core/message/195979
Mute This Topic: https://lists.openembedded.org/mt/104489784/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[Enguerrand de Ribaucourt]

On 21/02/2024 15:12, Enguerrand de Ribaucourt wrote:



On 20/02/2024 10:01, adrian.freiho...@gmail.com wrote:

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:

When launching the debug configuration, the source files from the
debug
rootfs were openened in the editor instead of the local workspace
files.
We add an exception to properly map them to the file being developed
and
compiled by the IDE integration. This also more closely matches what
the
user would expect compared to native development.

This is also true for the devtool fallback mode.


This looks still wrong to me. If files from the rootfs are openend for
a recipe which is in the workspace, the SDK is broken and we need to
understand and fix that. This patch does not solve that, but will make
it much harder to find the right solution for this issue.

Hi Adrian,

This is affecting essentially the devtool fallback mode. Since it's 
compiled through devtool, it's using the bitbake packaging information.


Steps to reproduce:
  - devtool modify powertop (autotools recipe, currently fallback)
  - devtool ide-sdk powertop [args...]
  - open the powertop sources workspace
  - run the debug launch configuration
  - initial breakpoint opens the rootfs-dbg main.cpp instead of the 
current workspace


This probably will get fixed for autotools with your specific class 
support, but will remain for the fallback mode.


Do we want to keep this source map, but only for the fallback mode? Like 
you say, it's kinda "cheating" the debug information, but since we are 
in fallback mode, the devtool packaging is supposed to be using the 
sources from the workspace, so it's more like "hinting" at the proper 
source.
It also affects more meson projects like lighttpd. The "hint" would 
still be valid given how it's built and deployed now. If you are still 
working on an alternative tinfoil deploy method, maybe it's not affected.


Adrian



Signed-off-by: Enguerrand de Ribaucourt

---
  scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
  scripts/lib/devtool/ide_sdk.py  | 1 +
  2 files changed, 2 insertions(+)

diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
b/scripts/lib/devtool/ide_plugins/ide_code.py
index b2193130d2e..c063b7d0590 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
  if gdb_cross_config.image_recipe.rootfs_dbg:
  launch_config['additionalSOLibSearchPath'] =
modified_recipe.solib_search_path_str(
  gdb_cross_config.image_recipe)
+    src_file_map[os.path.join("/usr/src/debug",
modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
  src_file_map["/usr/src/debug"] = os.path.join(
  gdb_cross_config.image_recipe.rootfs_dbg, "usr",
"src", "debug")
  else:
diff --git a/scripts/lib/devtool/ide_sdk.py
b/scripts/lib/devtool/ide_sdk.py
index 14679744807..f292edbe25c 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -356,6 +356,7 @@ class RecipeModified:
  'PACKAGE_DEBUG_SPLIT_STYLE')
  self.path = recipe_d.getVar('PATH')
  self.pn = recipe_d.getVar('PN')
+    self.pv = recipe_d.getVar('PV')
  self.recipe_sysroot = os.path.realpath(
  recipe_d.getVar('RECIPE_SYSROOT'))
  self.recipe_sysroot_native = os.path.realpath(



Enguerrand de Ribaucourt


Enguerrand de Ribaucourt

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195978): 
https://lists.openembedded.org/g/openembedded-core/message/195978
Mute This Topic: https://lists.openembedded.org/mt/104450241/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core][kirkstone 7/8] kernel: fix localversion in v6.3+

[Ryan Eatmon via lists.openembedded.org]

On 2/16/2024 7:51 AM, Andreas Helbech Kleist wrote:

On Fri, 2024-02-16 at 09:11 +0100, Andreas Helbech Kleist wrote:

On Thu, 2024-02-15 at 19:45 -0600, Ryan Eatmon via
lists.openembedded.org wrote:


On 2/15/2024 7:43 PM, Steve Sakoman wrote:

On Thu, Feb 15, 2024 at 3:20 PM Ryan Eatmon  wrote:



With this patch in place we are seeing a breakage on our kernel builds.
This patch was cherry picked from master, but the next patch on this
file in master removes the LOCALVERSION setting...

https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/kernel-arch.bbclass?h=master-next=b378eec156998eea55ba61e59103cb34fab0d07c


There is a disconnect here.  Why did we change kirkstone, a stable LTS
version with a partial patch series based on master?


The original kirkstone backport request for this patch was sent to the
list for review on February 9.

There were no comments or objections, so I added it to the patch test
queue.  No issues were encountered on the autobuilder, so I sent this
patch (along with the rest of the patch queue) to the list for a
second review opportunity on February 12.  Once again there were no
comments or objections, so it was merged today February 15.

I follow this process because I'm not smart enough, nor do I have time
enough to thoroughly research all implications of every patch.  I rely
on autobuilder testing and community review to minimize breakage.

This normally works quite well, but this time it didn't.  I'll revert
this patch and if someone would like to resubmit a proper series to
deal with the issue it is trying to fix I will consider it and run it
through the same process.


I can appreciate that.  My comment was more for the person who sent the
patch in the first place.  You are doing a great job.  I will try and
pay attention for a replacement patch coming in and review as well.


I'm sorry about that. I need to use a 6.6 kernel on kirkstone, and this
patch fixed my usecase. It looked like a simple bugfix, and I was not
aware that there was a later fix for this change.

Is there any way I can reproduce the error you saw Ryan?

And would it make sense to add an automated test for this usecase?

I'll test my usecase with both patches backported and re-submit.


I've now looked into it and can make it work in my setup by backporting
these three commits from master:

   kernel.bbclass: introduce KERNEL_LOCALVERSION
  
https://git.openembedded.org/openembedded-core/commit/?h=master-next=229435a52f36ddec5f85fb6d5ccd42044b688397


   kernel: fix localversion in v6.3+
  
https://git.openembedded.org/openembedded-core/commit/?h=master-next=765b13b7305c8d2f222cfc66d77c02e6a088c691


   kernel: make LOCALVERSION consistent between recipes
  
https://git.openembedded.org/openembedded-core/commit/?h=master-next=b378eec156998eea55ba61e59103cb34fab0d07c


The cherry-pick's apply cleanly without any conflicts.

Would they be appropriate for kirkstone? I'm wondering if there would
be any issue with backporting the KERNEL_LOCALVERSION variable
specifically.

If I hear nothing before the middle of next week, I'll submit the patch
series (CC'ing you Ryan).



Can you send me the patch series that you want to want to send in 
against kirkstone?  I'll apply them locally and run some tests on our 
setup to make sure they don't cause an issue.  I don't foresee them 
being a problem, but I'm more than happy to verify.




/Andreas


PS: This is my first submission to oe-core, so I'm just starting to
learn. I have no idea what or how the autobuilder tests, but I'm
willing to learn.





Steve



On 2/12/2024 7:54 AM, Steve Sakoman wrote:

From: Bruce Ashfield 

During testing of the v6.4 reference kernel, it was noticed that
on-target modules no longer matched the magic value of the running
kernel.

This was due to a different localversion in the cross built kernel
and the scripts / resources created on target.

This was due to changes in the setlocalversion script introduced
in the v6.3 series.

The .scmversion file is no longer used (or packaged) to inhibit
the addition of a "+" (through querying of the git status of the
kernel) or the setting of a local version.

We recently introduced the KERNEL_LOCALVERSION variable to allow
recipes to place a value in .scmversion, so we extend the use of
that variable to kernel-arch.bbclass and use it to set the
exported variable LOCALVERSION.

We must do it at the kernel-arch level, as the variable must be
exported in any kernel build to ensure that setlocalversion always
correctly sets the localversion.

Signed-off-by: Bruce Ashfield 
Signed-off-by: Richard Purdie 

Cherry-picked from master 765b13b7305c8d2f222cfc66d77c02e6a088c691

Signed-off-by: Andreas Helbech Kleist 
Signed-off-by: Steve Sakoman 
---
meta/classes/kernel-arch.bbclass |  7 +++
meta/classes/kernel.bbclass  | 10 --
2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel-arch.bbclass 

Re: [OE-core][kirkstone 7/8] kernel: fix localversion in v6.3+

[Ryan Eatmon via lists.openembedded.org]

On 2/16/2024 2:11 AM, Andreas Helbech Kleist wrote:

On Thu, 2024-02-15 at 19:45 -0600, Ryan Eatmon via
lists.openembedded.org wrote:


On 2/15/2024 7:43 PM, Steve Sakoman wrote:

On Thu, Feb 15, 2024 at 3:20 PM Ryan Eatmon  wrote:



With this patch in place we are seeing a breakage on our kernel builds.
This patch was cherry picked from master, but the next patch on this
file in master removes the LOCALVERSION setting...

https://git.openembedded.org/openembedded-core/commit/meta/classes-recipe/kernel-arch.bbclass?h=master-next=b378eec156998eea55ba61e59103cb34fab0d07c


There is a disconnect here.  Why did we change kirkstone, a stable LTS
version with a partial patch series based on master?


The original kirkstone backport request for this patch was sent to the
list for review on February 9.

There were no comments or objections, so I added it to the patch test
queue.  No issues were encountered on the autobuilder, so I sent this
patch (along with the rest of the patch queue) to the list for a
second review opportunity on February 12.  Once again there were no
comments or objections, so it was merged today February 15.

I follow this process because I'm not smart enough, nor do I have time
enough to thoroughly research all implications of every patch.  I rely
on autobuilder testing and community review to minimize breakage.

This normally works quite well, but this time it didn't.  I'll revert
this patch and if someone would like to resubmit a proper series to
deal with the issue it is trying to fix I will consider it and run it
through the same process.


I can appreciate that.  My comment was more for the person who sent the
patch in the first place.  You are doing a great job.  I will try and
pay attention for a replacement patch coming in and review as well.


I'm sorry about that. I need to use a 6.6 kernel on kirkstone, and this
patch fixed my usecase. It looked like a simple bugfix, and I was not
aware that there was a later fix for this change.

Is there any way I can reproduce the error you saw Ryan?

And would it make sense to add an automated test for this usecase?

I'll test my usecase with both patches backported and re-submit.

/Andreas

PS: This is my first submission to oe-core, so I'm just starting to
learn. I have no idea what or how the autobuilder tests, but I'm
willing to learn.


The meta-ti layer kernel recipes set KERNEL_LOCALVERSION.  With this 
change in place the modules directory was being name incorrectly which 
means that our boot testing failed.  For example, if we set:


KERNEL_LOCALVERSION=""

Then the directory was created:

/lib/modules--

One too many  in the name.

We never saw this issue on master in our testing because they patched 
the file and backed it out a few days later.







Steve



On 2/12/2024 7:54 AM, Steve Sakoman wrote:

From: Bruce Ashfield 

During testing of the v6.4 reference kernel, it was noticed that
on-target modules no longer matched the magic value of the running
kernel.

This was due to a different localversion in the cross built kernel
and the scripts / resources created on target.

This was due to changes in the setlocalversion script introduced
in the v6.3 series.

The .scmversion file is no longer used (or packaged) to inhibit
the addition of a "+" (through querying of the git status of the
kernel) or the setting of a local version.

We recently introduced the KERNEL_LOCALVERSION variable to allow
recipes to place a value in .scmversion, so we extend the use of
that variable to kernel-arch.bbclass and use it to set the
exported variable LOCALVERSION.

We must do it at the kernel-arch level, as the variable must be
exported in any kernel build to ensure that setlocalversion always
correctly sets the localversion.

Signed-off-by: Bruce Ashfield 
Signed-off-by: Richard Purdie 

Cherry-picked from master 765b13b7305c8d2f222cfc66d77c02e6a088c691

Signed-off-by: Andreas Helbech Kleist 
Signed-off-by: Steve Sakoman 
---
meta/classes/kernel-arch.bbclass |  7 +++
meta/classes/kernel.bbclass  | 10 --
2 files changed, 15 insertions(+), 2 deletions(-)

diff --git a/meta/classes/kernel-arch.bbclass b/meta/classes/kernel-arch.bbclass
index 4cd08b96fb..0a79dea0af 100644
--- a/meta/classes/kernel-arch.bbclass
+++ b/meta/classes/kernel-arch.bbclass
@@ -66,3 +66,10 @@ KERNEL_LD = "${CCACHE}${HOST_PREFIX}ld.bfd 
${HOST_LD_KERNEL_ARCH}"
KERNEL_AR = "${CCACHE}${HOST_PREFIX}ar ${HOST_AR_KERNEL_ARCH}"
TOOLCHAIN ?= "gcc"

+# 6.3+ requires the variable LOCALVERSION to be set to not get a "+" in
+# the local version. Having it empty means nothing will be added, and any
+# value will be appended to the local kernel version. This replaces the
+# use of .scmversion file for setting a localversion without using
+# the CONFIG_LOCALVERSION option.
+KERNEL_LOCALVERSION ??= ""
+export LOCALVERSION ?= "${KERNEL_LOCALVERSION}"
diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
index 

Re: [OE-core] [PATCH v2 2/5] devtool: code: Add source mapping for debug source files

[Enguerrand de Ribaucourt]

On 20/02/2024 10:01, adrian.freiho...@gmail.com wrote:

On Mon, 2024-02-19 at 17:55 +0100, Enguerrand de Ribaucourt wrote:

When launching the debug configuration, the source files from the
debug
rootfs were openened in the editor instead of the local workspace
files.
We add an exception to properly map them to the file being developed
and
compiled by the IDE integration. This also more closely matches what
the
user would expect compared to native development.

This is also true for the devtool fallback mode.


This looks still wrong to me. If files from the rootfs are openend for
a recipe which is in the workspace, the SDK is broken and we need to
understand and fix that. This patch does not solve that, but will make
it much harder to find the right solution for this issue.

Hi Adrian,

This is affecting essentially the devtool fallback mode. Since it's 
compiled through devtool, it's using the bitbake packaging information.


Steps to reproduce:
 - devtool modify powertop (autotools recipe, currently fallback)
 - devtool ide-sdk powertop [args...]
 - open the powertop sources workspace
 - run the debug launch configuration
 - initial breakpoint opens the rootfs-dbg main.cpp instead of the 
current workspace


This probably will get fixed for autotools with your specific class 
support, but will remain for the fallback mode.


Do we want to keep this source map, but only for the fallback mode? Like 
you say, it's kinda "cheating" the debug information, but since we are 
in fallback mode, the devtool packaging is supposed to be using the 
sources from the workspace, so it's more like "hinting" at the proper 
source.


Adrian



Signed-off-by: Enguerrand de Ribaucourt

---
  scripts/lib/devtool/ide_plugins/ide_code.py | 1 +
  scripts/lib/devtool/ide_sdk.py  | 1 +
  2 files changed, 2 insertions(+)

diff --git a/scripts/lib/devtool/ide_plugins/ide_code.py
b/scripts/lib/devtool/ide_plugins/ide_code.py
index b2193130d2e..c063b7d0590 100644
--- a/scripts/lib/devtool/ide_plugins/ide_code.py
+++ b/scripts/lib/devtool/ide_plugins/ide_code.py
@@ -234,6 +234,7 @@ class IdeVSCode(IdeBase):
  if gdb_cross_config.image_recipe.rootfs_dbg:
  launch_config['additionalSOLibSearchPath'] =
modified_recipe.solib_search_path_str(
  gdb_cross_config.image_recipe)
+    src_file_map[os.path.join("/usr/src/debug",
modified_recipe.pn, modified_recipe.pv)] = "${workspaceFolder}"
  src_file_map["/usr/src/debug"] = os.path.join(
  gdb_cross_config.image_recipe.rootfs_dbg, "usr",
"src", "debug")
  else:
diff --git a/scripts/lib/devtool/ide_sdk.py
b/scripts/lib/devtool/ide_sdk.py
index 14679744807..f292edbe25c 100755
--- a/scripts/lib/devtool/ide_sdk.py
+++ b/scripts/lib/devtool/ide_sdk.py
@@ -356,6 +356,7 @@ class RecipeModified:
  'PACKAGE_DEBUG_SPLIT_STYLE')
  self.path = recipe_d.getVar('PATH')
  self.pn = recipe_d.getVar('PN')
+    self.pv = recipe_d.getVar('PV')
  self.recipe_sysroot = os.path.realpath(
  recipe_d.getVar('RECIPE_SYSROOT'))
  self.recipe_sysroot_native = os.path.realpath(



Enguerrand de Ribaucourt


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195975): 
https://lists.openembedded.org/g/openembedded-core/message/195975
Mute This Topic: https://lists.openembedded.org/mt/104450241/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Mikko Rapeli]

Hi,

On Wed, Feb 21, 2024 at 03:23:48PM +0200, Mikko Rapeli via 
lists.openembedded.org wrote:
> FWIW, we have been using upstream kernel.org aarch64 defconfig plus
> few board specific fragments and few extra features for our testing needs.
> I have been very happy that several major kernel version updates have already
> been done this way and zero adaptations needed on our side for the
> ARM SystemReady boards and firmware which we support. A simple CI run to show
> passing test results was sufficient for a poky update with new kernel
> major version.
> 
> The implementation here looks pretty much like ours. Looks good, thanks Ross!
>
> More details of our setup:
> 
> https://gitlab.com/Linaro/trustedsubstrate/meta-ledge-secure/-/blob/main/meta-ledge-secure/recipes-kernel/linux/linux-ledge-common.inc?ref_type=heads
> https://gitlab.com/Linaro/trustedsubstrate/meta-ledge-secure/-/tree/main/meta-ledge-secure/recipes-kernel/linux/ledgearm64-kmeta?ref_type=heads
> https://trs.readthedocs.io/en/latest/

Our machine is based on poky qemuarm64 machine and thus uses KBRANCH:qemuarm64 
so
I don't see a need for new branch from linux-yocto recipe maintainer.
https://git.yoctoproject.org/linux-yocto/log/?h=v6.5/standard/qemuarm64

But maybe I don't see the full picture.

Cheers,

-Mikko

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195974): 
https://lists.openembedded.org/g/openembedded-core/message/195974
Mute This Topic: https://lists.openembedded.org/mt/104486073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Bruce Ashfield]

I should add that if just doing this for qemu is acceptable, I can take
over the task of creating the configuration fragments with what remains
for the week.

Are the MACHINE configs and everything else I need already in OE-core
or available somewhere that I can find ?

Bruce

On Wed, Feb 21, 2024 at 8:34 AM Bruce Ashfield via
lists.openembedded.org
 wrote:
>
> I'm quite simply a hard NACK on this.
>
> Everyone can feel free to overrule me here, but I won't be able to
> maintain this along with the other board that I test each -dev,
> variant and -stable bump with.
>
> There are tools, etc, that while gathering dust can help chop up a
> config, and that's where we can spend the time, along with making sure
> the fragments make sense and are usable.
>
> We all agreed to hold the reference boards to a higher standard, and
> now I see this. So as tired and frustrated as you say you are,
> multiply that by two with me.
>
> Cheers,
>
> Bruce
>
>
> On Wed, Feb 21, 2024 at 5:57 AM Ross Burton  wrote:
> >
> > From: Ross Burton 
> >
> > This is a new 64-bit "generic" Arm machine, that expects the hardware to
> > be SystemReady IR compatible. This is slightly forward-leaning as there's
> > not a _lot_ of SystemReady hardware in the wild, but most modern boards
> > are and the number will only grow.  Also, this is the only way to have a
> > 'generic' machine as without standardised bootloaders and firmware it
> > would be impossible.
> >
> > The base machine configuration isn't that exciting: it's a fully featured
> > machine that supports most things, booting via UEFI and an initramfs.
> >
> > However, the kernel is more interesting.  This RFC uses the upstream 
> > defconfig
> > because unlike some other platforms, the arm64 defconfig is actively
> > maintained with the goal of being a 'boots on most hardware' configuration.
> > My argument is: why would we duplicate that effort?
> >
> > The "linux-yocto way" is configuration fragments and after a week of
> > hair-pulling I do actually have fragments that boot on a BeaglePlay, but
> > to say this was a tiresome and frustrating exercise would be understating 
> > it.
> >
> > So, a request for comments: is it acceptable to use the upstream defconfig 
> > in
> > a reference BSP?  Personally I'm torn: the Yocto way is fragments not 
> > monolithic
> > configs, but repeating the effort to fragmentise the configuration and then
> > also have it sufficiently modular that it can be used in pieces - instead of
> > just being a large file split up into smaller files - is a lot of effort for
> > what might end up being minimal gain.  My fear is we end up with a 
> > fragmented
> > configuration that can't be easily modified without breaking some platforms,
> > and badly copies what the defconfig already does.
> >
> > Ross
> > ---
> >  meta-yocto-bsp/README.hardware.md |  7 +
> >  meta-yocto-bsp/conf/machine/genericarm64.conf | 26 +++
> >  .../linux/linux-yocto_6.6.bbappend|  9 +++
> >  meta-yocto-bsp/wic/genericarm64.wks.in| 11 
> >  4 files changed, 53 insertions(+)
> >  create mode 100644 meta-yocto-bsp/conf/machine/genericarm64.conf
> >  create mode 100644 meta-yocto-bsp/wic/genericarm64.wks.in
> >
> > diff --git a/meta-yocto-bsp/README.hardware.md 
> > b/meta-yocto-bsp/README.hardware.md
> > index a8f38cb21a6..58ebc328b56 100644
> > --- a/meta-yocto-bsp/README.hardware.md
> > +++ b/meta-yocto-bsp/README.hardware.md
> > @@ -29,6 +29,7 @@ The following boards are supported by the meta-yocto-bsp 
> > layer:
> >
> >* Texas Instruments Beaglebone (beaglebone-yocto)
> >* General IA platforms (genericx86 and genericx86-64)
> > +  * General 64-bit Arm SystemReady platforms (genericarm64)
> >
> >  For more information see the board's section below. The appropriate MACHINE
> >  variable value corresponding to the board is given in brackets.
> > @@ -126,6 +127,12 @@ USB Device:
> > dd command to write the image to a USB stick.
> >
> >
> > +SystemReady Arm Platforms
> > +=
> > +
> > +TODO
> > +
> > +
> >  Texas Instruments Beaglebone (beaglebone-yocto)
> >  ===
> >
> > diff --git a/meta-yocto-bsp/conf/machine/genericarm64.conf 
> > b/meta-yocto-bsp/conf/machine/genericarm64.conf
> > new file mode 100644
> > index 000..2ea270d8b06
> > --- /dev/null
> > +++ b/meta-yocto-bsp/conf/machine/genericarm64.conf
> > @@ -0,0 +1,26 @@
> > +#@TYPE: Machine
> > +#@NAME: genericarm64
> > +#@DESCRIPTION: Generic Arm64 machine for typical SystemReady platforms, 
> > which
> > +#have working firmware and boot via EFI.
> > +
> > +require conf/machine/include/arm/arch-armv8a.inc
> > +
> > +# Arm Base System Architecture says v8.0+ is allowed, but FEAT_CRC32 is 
> > required
> > +DEFAULTTUNE = "armv8a-crc"
> > +
> > +MACHINE_FEATURES = "acpi alsa bluetooth efi keyboard pci qemu-usermode rtc 
> > screen usbhost vfat wifi"
> > +
> > +# Install all the 

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Bruce Ashfield]

I'm quite simply a hard NACK on this.

Everyone can feel free to overrule me here, but I won't be able to
maintain this along with the other board that I test each -dev,
variant and -stable bump with.

There are tools, etc, that while gathering dust can help chop up a
config, and that's where we can spend the time, along with making sure
the fragments make sense and are usable.

We all agreed to hold the reference boards to a higher standard, and
now I see this. So as tired and frustrated as you say you are,
multiply that by two with me.

Cheers,

Bruce


On Wed, Feb 21, 2024 at 5:57 AM Ross Burton  wrote:
>
> From: Ross Burton 
>
> This is a new 64-bit "generic" Arm machine, that expects the hardware to
> be SystemReady IR compatible. This is slightly forward-leaning as there's
> not a _lot_ of SystemReady hardware in the wild, but most modern boards
> are and the number will only grow.  Also, this is the only way to have a
> 'generic' machine as without standardised bootloaders and firmware it
> would be impossible.
>
> The base machine configuration isn't that exciting: it's a fully featured
> machine that supports most things, booting via UEFI and an initramfs.
>
> However, the kernel is more interesting.  This RFC uses the upstream defconfig
> because unlike some other platforms, the arm64 defconfig is actively
> maintained with the goal of being a 'boots on most hardware' configuration.
> My argument is: why would we duplicate that effort?
>
> The "linux-yocto way" is configuration fragments and after a week of
> hair-pulling I do actually have fragments that boot on a BeaglePlay, but
> to say this was a tiresome and frustrating exercise would be understating it.
>
> So, a request for comments: is it acceptable to use the upstream defconfig in
> a reference BSP?  Personally I'm torn: the Yocto way is fragments not 
> monolithic
> configs, but repeating the effort to fragmentise the configuration and then
> also have it sufficiently modular that it can be used in pieces - instead of
> just being a large file split up into smaller files - is a lot of effort for
> what might end up being minimal gain.  My fear is we end up with a fragmented
> configuration that can't be easily modified without breaking some platforms,
> and badly copies what the defconfig already does.
>
> Ross
> ---
>  meta-yocto-bsp/README.hardware.md |  7 +
>  meta-yocto-bsp/conf/machine/genericarm64.conf | 26 +++
>  .../linux/linux-yocto_6.6.bbappend|  9 +++
>  meta-yocto-bsp/wic/genericarm64.wks.in| 11 
>  4 files changed, 53 insertions(+)
>  create mode 100644 meta-yocto-bsp/conf/machine/genericarm64.conf
>  create mode 100644 meta-yocto-bsp/wic/genericarm64.wks.in
>
> diff --git a/meta-yocto-bsp/README.hardware.md 
> b/meta-yocto-bsp/README.hardware.md
> index a8f38cb21a6..58ebc328b56 100644
> --- a/meta-yocto-bsp/README.hardware.md
> +++ b/meta-yocto-bsp/README.hardware.md
> @@ -29,6 +29,7 @@ The following boards are supported by the meta-yocto-bsp 
> layer:
>
>* Texas Instruments Beaglebone (beaglebone-yocto)
>* General IA platforms (genericx86 and genericx86-64)
> +  * General 64-bit Arm SystemReady platforms (genericarm64)
>
>  For more information see the board's section below. The appropriate MACHINE
>  variable value corresponding to the board is given in brackets.
> @@ -126,6 +127,12 @@ USB Device:
> dd command to write the image to a USB stick.
>
>
> +SystemReady Arm Platforms
> +=
> +
> +TODO
> +
> +
>  Texas Instruments Beaglebone (beaglebone-yocto)
>  ===
>
> diff --git a/meta-yocto-bsp/conf/machine/genericarm64.conf 
> b/meta-yocto-bsp/conf/machine/genericarm64.conf
> new file mode 100644
> index 000..2ea270d8b06
> --- /dev/null
> +++ b/meta-yocto-bsp/conf/machine/genericarm64.conf
> @@ -0,0 +1,26 @@
> +#@TYPE: Machine
> +#@NAME: genericarm64
> +#@DESCRIPTION: Generic Arm64 machine for typical SystemReady platforms, which
> +#have working firmware and boot via EFI.
> +
> +require conf/machine/include/arm/arch-armv8a.inc
> +
> +# Arm Base System Architecture says v8.0+ is allowed, but FEAT_CRC32 is 
> required
> +DEFAULTTUNE = "armv8a-crc"
> +
> +MACHINE_FEATURES = "acpi alsa bluetooth efi keyboard pci qemu-usermode rtc 
> screen usbhost vfat wifi"
> +
> +# Install all the kernel modules and all the firmware
> +MACHINE_EXTRA_RRECOMMENDS += "kernel-modules linux-firmware"
> +
> +KERNEL_IMAGETYPE = "Image"
> +PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
> +INITRAMFS_IMAGE ?= "core-image-initramfs-boot"
> +
> +IMAGE_FSTYPES ?= "wic"
> +WKS_FILE ?= "genericarm64.wks.in"
> +
> +EFI_PROVIDER ?= "${@bb.utils.contains("DISTRO_FEATURES", "systemd", 
> "systemd-boot", "grub-efi", d)}"
> +
> +# Try to bring up one physical serial console, or a virtualized serial 
> console
> +SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
> diff --git 

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Mikko Rapeli]

Hi,

On Wed, Feb 21, 2024 at 11:21:39AM +, Richard Purdie wrote:
> On Wed, 2024-02-21 at 10:57 +, Ross Burton wrote:
> > From: Ross Burton 
> > 
> > This is a new 64-bit "generic" Arm machine, that expects the hardware to
> > be SystemReady IR compatible. This is slightly forward-leaning as there's
> > not a _lot_ of SystemReady hardware in the wild, but most modern boards
> > are and the number will only grow.  Also, this is the only way to have a
> > 'generic' machine as without standardised bootloaders and firmware it
> > would be impossible.
> > 
> > The base machine configuration isn't that exciting: it's a fully featured
> > machine that supports most things, booting via UEFI and an initramfs.
> > 
> > However, the kernel is more interesting.  This RFC uses the upstream 
> > defconfig
> > because unlike some other platforms, the arm64 defconfig is actively
> > maintained with the goal of being a 'boots on most hardware' configuration.
> > My argument is: why would we duplicate that effort?
> 
> Can you point at the policy/process which decides how a config option
> makes it in there?
> 
> > The "linux-yocto way" is configuration fragments and after a week of
> > hair-pulling I do actually have fragments that boot on a BeaglePlay, but
> > to say this was a tiresome and frustrating exercise would be understating 
> > it.
> > 
> > So, a request for comments: is it acceptable to use the upstream defconfig 
> > in
> > a reference BSP?  Personally I'm torn: the Yocto way is fragments not 
> > monolithic
> > configs, but repeating the effort to fragmentise the configuration and then
> > also have it sufficiently modular that it can be used in pieces - instead of
> > just being a large file split up into smaller files - is a lot of effort for
> > what might end up being minimal gain.  My fear is we end up with a 
> > fragmented
> > configuration that can't be easily modified without breaking some platforms,
> > and badly copies what the defconfig already does.
> 
> Let me play devils advocate.
> 
> I do understand it is a pain, equally, once you do have it working, it
> is something you rarely have to touch again for a given board.
> 
> In the context of linux-yocto, I suspect we'd end up with a few board
> specific lists of config options (fragments) that board *really* needs.
> genericarm64 would end up as the sum of all those configs, plus any
> other bits which you really want in a generic machine, presumably the
> genericx86 machines have an idea of that.
> 
> The advantage here is that you give the users three options:
> 
> a) the kitchen sink upstream "systemready" defconfig
> b) the genericarm64 machine defconfig which works most places and 
>is 'guaranteed' on our target test platforms
> c) a machine targeted defconfig with only what a board needs
> 
> Developers/users end up in different places at different points in
> there lifecycles. 
> 
> For in depth development, you might like to be able to cut the kernel
> down so you'd not rebuilding tons of stuff you don't need/use/care
> about each time, nor transferring it to a device during updates.
> 
> For a specific product release, you again might want to trim it down to
> what it needs.
> 
> If you're shipping demo software, a full systemready image would be
> much more appropriate.
> 
> I think it comes down to whether the fragments are usable and testable.
> We have a list of targets we want this new machine to run on so lets
> start with those, define genericarm64 as that set of fragments combined
> plus the generic pieces linux-yocto adds, then go from there. If you
> add a new machine to the test matrix, we add a new fragment. If someone
> wants to add new config, they need to show a machine using it.

FWIW, we have been using upstream kernel.org aarch64 defconfig plus
few board specific fragments and few extra features for our testing needs.
I have been very happy that several major kernel version updates have already
been done this way and zero adaptations needed on our side for the
ARM SystemReady boards and firmware which we support. A simple CI run to show
passing test results was sufficient for a poky update with new kernel
major version.

The implementation here looks pretty much like ours. Looks good, thanks Ross!

More details of our setup:

https://gitlab.com/Linaro/trustedsubstrate/meta-ledge-secure/-/blob/main/meta-ledge-secure/recipes-kernel/linux/linux-ledge-common.inc?ref_type=heads
https://gitlab.com/Linaro/trustedsubstrate/meta-ledge-secure/-/tree/main/meta-ledge-secure/recipes-kernel/linux/ledgearm64-kmeta?ref_type=heads
https://trs.readthedocs.io/en/latest/

Cheers,

-Mikko

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195971): 
https://lists.openembedded.org/g/openembedded-core/message/195971
Mute This Topic: https://lists.openembedded.org/mt/104486073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: 

[OE-core] [PATCH] oeqa/selftest/rust: Simplify the rust testsuite output gathering/processing

[Richard Purdie]

The rust testsuite was redirecting command output to a file, which made it
hard to debug failure cases since the logs were not available to print to
the console.

Rework the code so it uses the existing popen logging and hence allows us
to improve the error logging situation and make debugging failures easier.

Signed-off-by: Richard Purdie 
---
 meta/lib/oeqa/selftest/cases/rust.py | 13 -
 1 file changed, 8 insertions(+), 5 deletions(-)

diff --git a/meta/lib/oeqa/selftest/cases/rust.py 
b/meta/lib/oeqa/selftest/cases/rust.py
index 120be6454fa..ad14189c6df 100644
--- a/meta/lib/oeqa/selftest/cases/rust.py
+++ b/meta/lib/oeqa/selftest/cases/rust.py
@@ -216,13 +216,16 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, 
OEPTestResultTestCase):
 cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % 
rustlibpath
 # Trigger testing.
 cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s 
--target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
-runCmd(cmd)
+cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s 
--target %s" % (builddir, testargs, targetsys)
+retval = runCmd(cmd)
 end_time = time.time()
 
+resultlog = rustlibpath + "/results-log.txt"
+with open(resultlog, "w") as f:
+f.write(retval.output)
+
 ptestsuite = "rust"
-self.ptest_section(ptestsuite, duration = int(end_time - 
start_time), logfile = builddir + "/summary.txt")
-filename = builddir + "/summary.txt"
-test_results = parse_results(filename)
+self.ptest_section(ptestsuite, duration = int(end_time - 
start_time), logfile=resultlog)
+test_results = parse_results(resultlog)
 for test in test_results:
 self.ptest_result(ptestsuite, test, test_results[test])
-- 
2.40.1


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195970): 
https://lists.openembedded.org/g/openembedded-core/message/195970
Mute This Topic: https://lists.openembedded.org/mt/104487861/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] rust: Oe-selftest error log on console when it fails.

[Richard Purdie]

On Wed, 2024-02-21 at 13:30 +0530, Yash Shinde wrote:
> 
>  The runCmd() returns the 'Result' object containing information about the 
> command execution. It has the following attributes:
>     result.command = command
>     result.status = cmd.status
>     result.output = cmd.output
>     result.error = cmd.error
>     result.pid = cmd.process.pid
>  
> https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n198
>   
>  I tried to capture the return object value (stderr i.e result.error) and 
> print it to the terminal, but that didn't work as expected.
>  Even I tried to print some debug statements in rust.py file and it also 
> didn't show up in the terminal or in the summary.txt file.
>  I assume there's something in oe-selftest framework that doesn't print 
> statements directly.
>   
>  Also, I see there's a "output_log" parameter in the runCmd function 
> parameters, which I understand is used to redirect stdout of the
>  command being executed. Currently, I am checking with different values by 
> referring to other oe-selftests and their corresponding behavior with it.
>  
> 
> 
> 
>  I am checking with some functions and procedures from unittest and 
> subprocess.Popen frameworks to get the error logs:
>  
> 
> 
> 
>  https://docs.python.org/3/library/unittest.html
>  https://docs.python.org/3/library/subprocess.html#subprocess.Popen
>  


I had a look at this and tried an experiment locally. This seemed to
work:

diff --git a/meta/lib/oeqa/selftest/cases/rust.py 
b/meta/lib/oeqa/selftest/cases/rust.py
index 120be6454fa..ad14189c6df 100644
--- a/meta/lib/oeqa/selftest/cases/rust.py
+++ b/meta/lib/oeqa/selftest/cases/rust.py
@@ -216,13 +216,16 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, 
OEPTestResultTestCase):
 cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % 
rustlibpath
 # Trigger testing.
 cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s 
--target %s > summary.txt 2>&1;" % (builddir, testargs, targetsys)
-runCmd(cmd)
+cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s 
--target %s" % (builddir, testargs, targetsys)
+retval = runCmd(cmd)
 end_time = time.time()
 
+resultlog = rustlibpath + "/results-log.txt"
+with open(resultlog, "w") as f:
+f.write(retval.output)
+
 ptestsuite = "rust"
-self.ptest_section(ptestsuite, duration = int(end_time - 
start_time), logfile = builddir + "/summary.txt")
-filename = builddir + "/summary.txt"
-test_results = parse_results(filename)
+self.ptest_section(ptestsuite, duration = int(end_time - 
start_time), logfile=resultlog)
+test_results = parse_results(resultlog)
 for test in test_results:
 self.ptest_result(ptestsuite, test, test_results[test])


I'm fairly sure we could improve it further but this does start that
process.

Cheers,

Richard


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195969): 
https://lists.openembedded.org/g/openembedded-core/message/195969
Mute This Topic: https://lists.openembedded.org/mt/104331501/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Richard Purdie]

On Wed, 2024-02-21 at 10:57 +, Ross Burton wrote:
> From: Ross Burton 
> 
> This is a new 64-bit "generic" Arm machine, that expects the hardware to
> be SystemReady IR compatible. This is slightly forward-leaning as there's
> not a _lot_ of SystemReady hardware in the wild, but most modern boards
> are and the number will only grow.  Also, this is the only way to have a
> 'generic' machine as without standardised bootloaders and firmware it
> would be impossible.
> 
> The base machine configuration isn't that exciting: it's a fully featured
> machine that supports most things, booting via UEFI and an initramfs.
> 
> However, the kernel is more interesting.  This RFC uses the upstream defconfig
> because unlike some other platforms, the arm64 defconfig is actively
> maintained with the goal of being a 'boots on most hardware' configuration.
> My argument is: why would we duplicate that effort?

Can you point at the policy/process which decides how a config option
makes it in there?

> The "linux-yocto way" is configuration fragments and after a week of
> hair-pulling I do actually have fragments that boot on a BeaglePlay, but
> to say this was a tiresome and frustrating exercise would be understating it.
> 
> So, a request for comments: is it acceptable to use the upstream defconfig in
> a reference BSP?  Personally I'm torn: the Yocto way is fragments not 
> monolithic
> configs, but repeating the effort to fragmentise the configuration and then
> also have it sufficiently modular that it can be used in pieces - instead of
> just being a large file split up into smaller files - is a lot of effort for
> what might end up being minimal gain.  My fear is we end up with a fragmented
> configuration that can't be easily modified without breaking some platforms,
> and badly copies what the defconfig already does.

Let me play devils advocate.

I do understand it is a pain, equally, once you do have it working, it
is something you rarely have to touch again for a given board.

In the context of linux-yocto, I suspect we'd end up with a few board
specific lists of config options (fragments) that board *really* needs.
genericarm64 would end up as the sum of all those configs, plus any
other bits which you really want in a generic machine, presumably the
genericx86 machines have an idea of that.

The advantage here is that you give the users three options:

a) the kitchen sink upstream "systemready" defconfig
b) the genericarm64 machine defconfig which works most places and 
   is 'guaranteed' on our target test platforms
c) a machine targeted defconfig with only what a board needs

Developers/users end up in different places at different points in
there lifecycles. 

For in depth development, you might like to be able to cut the kernel
down so you'd not rebuilding tons of stuff you don't need/use/care
about each time, nor transferring it to a device during updates.

For a specific product release, you again might want to trim it down to
what it needs.

If you're shipping demo software, a full systemready image would be
much more appropriate.

I think it comes down to whether the fragments are usable and testable.
We have a list of targets we want this new machine to run on so lets
start with those, define genericarm64 as that set of fragments combined
plus the generic pieces linux-yocto adds, then go from there. If you
add a new machine to the test matrix, we add a new fragment. If someone
wants to add new config, they need to show a machine using it.

Cheers,

Richard




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195968): 
https://lists.openembedded.org/g/openembedded-core/message/195968
Mute This Topic: https://lists.openembedded.org/mt/104486073/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] Patchtest results for [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Patchtest]

Thank you for your submission. Patchtest identified one
or more issues with the patch. Please see the log below for
more information:

---
Testing patch 
/home/patchtest/share/mboxes/RFC-Add-genericarm64-MACHINE-using-upstream-defconfig.patch

FAIL: test Signed-off-by presence: Mbox is missing Signed-off-by. Add it 
manually or with "git commit --amend -s" 
(test_mbox.TestMbox.test_signed_off_by_presence)
FAIL: test shortlog format: Commit shortlog (first line of commit message) 
should follow the format ": " 
(test_mbox.TestMbox.test_shortlog_format)

PASS: test author valid (test_mbox.TestMbox.test_author_valid)
PASS: test commit message presence 
(test_mbox.TestMbox.test_commit_message_presence)
PASS: test max line length (test_metadata.TestMetadata.test_max_line_length)
PASS: test mbox format (test_mbox.TestMbox.test_mbox_format)
PASS: test non-AUH upgrade (test_mbox.TestMbox.test_non_auh_upgrade)
PASS: test shortlog length (test_mbox.TestMbox.test_shortlog_length)

SKIP: pretest pylint: No python related patches, skipping test 
(test_python_pylint.PyLint.pretest_pylint)
SKIP: pretest src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.pretest_src_uri_left_files)
SKIP: test CVE check ignore: No modified recipes, skipping test 
(test_metadata.TestMetadata.test_cve_check_ignore)
SKIP: test CVE tag format: No new CVE patches introduced 
(test_patch.TestPatch.test_cve_tag_format)
SKIP: test Signed-off-by presence: No new CVE patches introduced 
(test_patch.TestPatch.test_signed_off_by_presence)
SKIP: test Upstream-Status presence: No new CVE patches introduced 
(test_patch.TestPatch.test_upstream_status_presence_format)
SKIP: test bugzilla entry format: No bug ID found 
(test_mbox.TestMbox.test_bugzilla_entry_format)
SKIP: test lic files chksum modified not mentioned: No modified recipes, 
skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_modified_not_mentioned)
SKIP: test lic files chksum presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_lic_files_chksum_presence)
SKIP: test license presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_license_presence)
SKIP: test pylint: No python related patches, skipping test 
(test_python_pylint.PyLint.test_pylint)
SKIP: test series merge on head: Merge test is disabled for now 
(test_mbox.TestMbox.test_series_merge_on_head)
SKIP: test src uri left files: No modified recipes, skipping pretest 
(test_metadata.TestMetadata.test_src_uri_left_files)
SKIP: test summary presence: No added recipes, skipping test 
(test_metadata.TestMetadata.test_summary_presence)
SKIP: test target mailing list: Series merged, no reason to check other mailing 
lists (test_mbox.TestMbox.test_target_mailing_list)

---

Please address the issues identified and
submit a new revision of the patch, or alternatively, reply to this
email with an explanation of why the patch should be accepted. If you
believe these results are due to an error in patchtest, please submit a
bug at https://bugzilla.yoctoproject.org/ (use the 'Patchtest' category
under 'Yocto Project Subprojects'). For more information on specific
failures, see: https://wiki.yoctoproject.org/wiki/Patchtest. Thank
you!

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195967): 
https://lists.openembedded.org/g/openembedded-core/message/195967
Mute This Topic: https://lists.openembedded.org/mt/104485875/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[OE-core] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig

[Ross Burton]

From: Ross Burton 

This is a new 64-bit "generic" Arm machine, that expects the hardware to
be SystemReady IR compatible. This is slightly forward-leaning as there's
not a _lot_ of SystemReady hardware in the wild, but most modern boards
are and the number will only grow.  Also, this is the only way to have a
'generic' machine as without standardised bootloaders and firmware it
would be impossible.

The base machine configuration isn't that exciting: it's a fully featured
machine that supports most things, booting via UEFI and an initramfs.

However, the kernel is more interesting.  This RFC uses the upstream defconfig
because unlike some other platforms, the arm64 defconfig is actively
maintained with the goal of being a 'boots on most hardware' configuration.
My argument is: why would we duplicate that effort?

The "linux-yocto way" is configuration fragments and after a week of
hair-pulling I do actually have fragments that boot on a BeaglePlay, but
to say this was a tiresome and frustrating exercise would be understating it.

So, a request for comments: is it acceptable to use the upstream defconfig in
a reference BSP?  Personally I'm torn: the Yocto way is fragments not monolithic
configs, but repeating the effort to fragmentise the configuration and then
also have it sufficiently modular that it can be used in pieces - instead of
just being a large file split up into smaller files - is a lot of effort for
what might end up being minimal gain.  My fear is we end up with a fragmented
configuration that can't be easily modified without breaking some platforms,
and badly copies what the defconfig already does.

Ross
---
 meta-yocto-bsp/README.hardware.md |  7 +
 meta-yocto-bsp/conf/machine/genericarm64.conf | 26 +++
 .../linux/linux-yocto_6.6.bbappend|  9 +++
 meta-yocto-bsp/wic/genericarm64.wks.in| 11 
 4 files changed, 53 insertions(+)
 create mode 100644 meta-yocto-bsp/conf/machine/genericarm64.conf
 create mode 100644 meta-yocto-bsp/wic/genericarm64.wks.in

diff --git a/meta-yocto-bsp/README.hardware.md 
b/meta-yocto-bsp/README.hardware.md
index a8f38cb21a6..58ebc328b56 100644
--- a/meta-yocto-bsp/README.hardware.md
+++ b/meta-yocto-bsp/README.hardware.md
@@ -29,6 +29,7 @@ The following boards are supported by the meta-yocto-bsp 
layer:
 
   * Texas Instruments Beaglebone (beaglebone-yocto)
   * General IA platforms (genericx86 and genericx86-64)
+  * General 64-bit Arm SystemReady platforms (genericarm64)
 
 For more information see the board's section below. The appropriate MACHINE
 variable value corresponding to the board is given in brackets.
@@ -126,6 +127,12 @@ USB Device:
dd command to write the image to a USB stick.
 
 
+SystemReady Arm Platforms
+=
+
+TODO
+
+
 Texas Instruments Beaglebone (beaglebone-yocto)
 ===
 
diff --git a/meta-yocto-bsp/conf/machine/genericarm64.conf 
b/meta-yocto-bsp/conf/machine/genericarm64.conf
new file mode 100644
index 000..2ea270d8b06
--- /dev/null
+++ b/meta-yocto-bsp/conf/machine/genericarm64.conf
@@ -0,0 +1,26 @@
+#@TYPE: Machine
+#@NAME: genericarm64
+#@DESCRIPTION: Generic Arm64 machine for typical SystemReady platforms, which
+#have working firmware and boot via EFI.
+
+require conf/machine/include/arm/arch-armv8a.inc
+
+# Arm Base System Architecture says v8.0+ is allowed, but FEAT_CRC32 is 
required
+DEFAULTTUNE = "armv8a-crc"
+
+MACHINE_FEATURES = "acpi alsa bluetooth efi keyboard pci qemu-usermode rtc 
screen usbhost vfat wifi"
+
+# Install all the kernel modules and all the firmware
+MACHINE_EXTRA_RRECOMMENDS += "kernel-modules linux-firmware"
+
+KERNEL_IMAGETYPE = "Image"
+PREFERRED_PROVIDER_virtual/kernel ?= "linux-yocto"
+INITRAMFS_IMAGE ?= "core-image-initramfs-boot"
+
+IMAGE_FSTYPES ?= "wic"
+WKS_FILE ?= "genericarm64.wks.in"
+
+EFI_PROVIDER ?= "${@bb.utils.contains("DISTRO_FEATURES", "systemd", 
"systemd-boot", "grub-efi", d)}"
+
+# Try to bring up one physical serial console, or a virtualized serial console
+SERIAL_CONSOLES ?= "115200;ttyAMA0 115200;hvc0"
diff --git a/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_6.6.bbappend 
b/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_6.6.bbappend
index 8e465c241e8..18f95de348f 100644
--- a/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_6.6.bbappend
+++ b/meta-yocto-bsp/recipes-kernel/linux/linux-yocto_6.6.bbappend
@@ -1,19 +1,28 @@
 KBRANCH:genericx86  = "v6.6/standard/base"
+KBRANCH:genericarm64  = "v6.6/standard/base"
 KBRANCH:genericx86-64  = "v6.6/standard/base"
 KBRANCH:beaglebone-yocto = "v6.6/standard/beaglebone"
 
+KMACHINE:genericarm64 ?= "genericarm64"
 KMACHINE:genericx86 ?= "common-pc"
 KMACHINE:genericx86-64 ?= "common-pc-64"
 KMACHINE:beaglebone-yocto ?= "beaglebone"
 
+SRCREV_machine:genericarm64 ?= "332d4668fcc32826907d4f3c4938845206006089"
 SRCREV_machine:genericx86 ?= "332d4668fcc32826907d4f3c4938845206006089"
 

Re: [OE-core] [PATCH] sanity.bbclass: raise_sanity_error if /tmp is noexec

[Alexander Kanavin]

On Wed, 21 Feb 2024 at 10:48, Ross Burton  wrote:
> You _can_ export TMPDIR but that has to be done on a per-recipe/class basis 
> very carefully as TMPDIR means something else to Bitbake.
>
> The problem is recipes that use mktemp to write files and execute them (be it 
> shell scripts, or as a place to write C and then compile in the same 
> directory).  These will be in /tmp (again, we can’t set TMPDIR because for 
> foolish historical reasons, TMPDIR is used by bitbake).
>
> We first noticed this with Meson where noexec /tmp meant the configure tests 
> failed. We worked around it at the time by assigning TMPDIR when calling 
> Meson, but since them Meson writes to its own build tree now.  This has been 
> seen before though, but luckily noexec /tmp is fairly unusual so I doubt this 
> will break many builds.

I'm actually curious where noexec /tmp can be observed. It does seem
rare, because I think it's the first time someone came up with a
sanity check for it. Perhaps it should be treated as a bug in that
respective environment/OS/container?

Alex

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195965): 
https://lists.openembedded.org/g/openembedded-core/message/195965
Mute This Topic: https://lists.openembedded.org/mt/104258828/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] sanity.bbclass: raise_sanity_error if /tmp is noexec

[Ross Burton]

On 21 Feb 2024, at 07:18, ChenQi  wrote:
> I just noticed the change. I can't find the V2 in my mailbox, so I'm going to 
> reply here.
> I'm a little concerned about forcing such requirement here. It does not seem 
> *necessary*.
> As far as I know, the whole oe-core does not need /tmp to be exec. The commit 
> message says 'old meson', this means the current version of meson works well, 
> right?
> Also, why is there 'no simple way to workaround'? Is the recipe hardcoding 
> '/tmp' instead of using API or command? Does exporting TMPDIR work?
> e.g.,
> export TMPDIR="${B}/tmp”

You _can_ export TMPDIR but that has to be done on a per-recipe/class basis 
very carefully as TMPDIR means something else to Bitbake.

The problem is recipes that use mktemp to write files and execute them (be it 
shell scripts, or as a place to write C and then compile in the same 
directory).  These will be in /tmp (again, we can’t set TMPDIR because for 
foolish historical reasons, TMPDIR is used by bitbake).

We first noticed this with Meson where noexec /tmp meant the configure tests 
failed. We worked around it at the time by assigning TMPDIR when calling Meson, 
but since them Meson writes to its own build tree now.  This has been seen 
before though, but luckily noexec /tmp is fairly unusual so I doubt this will 
break many builds.

Ross
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#195964): 
https://lists.openembedded.org/g/openembedded-core/message/195964
Mute This Topic: https://lists.openembedded.org/mt/104258828/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH] rust: Oe-selftest error log on console when it fails.

[Shinde, Yash via lists.openembedded.org]

On 13-02-2024 23:42, Randy MacLeod wrote:

On 2024-02-13 8:04 a.m., yash.shi...@windriver.com wrote:

From: Yash Shinde

The rust oe-selftest output error log doesn't show any information on console 
when it fails.
The following changes emit stderr logs in terminal along with re-directing stdout and 
stderr to "summary.txt" file.

Changes made::
- cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > summary.txt 
2>&1;" % (builddir, testargs, targetsys)
+ cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 
2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)


summary.txt:  Redirects the standard output (stdout) of the 
command to a file 'summary.txt'

2> >(tee summary.txt >&2):  Redirects stderr & stdout to summary.txt & 
writes stderr on terminal

The overall effect is that both stdout and stderr are captured in the 
summary.txt file, while stderr still being displayed in the terminal.

Signed-off-by: Yash Shinde
---
  meta/lib/oeqa/selftest/cases/rust.py | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/lib/oeqa/selftest/cases/rust.py 
b/meta/lib/oeqa/selftest/cases/rust.py
index 164ad11ecd..07f1b5706c 100644
--- a/meta/lib/oeqa/selftest/cases/rust.py
+++ b/meta/lib/oeqa/selftest/cases/rust.py
@@ -213,7 +213,7 @@ class RustSelfTestSystemEmulated(OESelftestTestCase, 
OEPTestResultTestCase):
  cmd = cmd + " export RUST_TARGET_PATH=%s/rust-targets;" % 
rustlibpath
  # Trigger testing.
  cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" % qemu.ip
-cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s > 
summary.txt 2>&1;" % (builddir, testargs, targetsys)
+cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py test %s --target %s  > 
summary.txt 2> >(tee summary.txt >&2);" % (builddir, testargs, targetsys)
  runCmd(cmd)
  end_time = time.time()
  


Hi Yash,

We talked about this change and decided that it's just too odd a shell 
pipeline to merge!


Also, it depends on the bash shell as you can tell by pasting:

   #!/bin/sh
   python3 src/bootstrap/bootstrap.py test %s --target %s  > 
summary.txt 2> >(tee summary.txt >&2);


into https://www.shellcheck.net/ . You'll see the following log and error:

$ shellcheck myscript

Line 2:
python3 src/bootstrap/bootstrap.py test %s --target %s  > summary.txt 
2> >(tee summary.txt >&2);
  ^-- SC2094 
 (info): Make sure not to read 
and write the same file in the same pipeline.
>>   ^-- SC3001  (warning): In POSIX 
sh, process substitution is undefined.
>> ^-- SC2094  (info): Make sure not 
to read and write the same file in the same pipeline.



The YP goal / requirement is to avoid bash dependencies since user may 
be using dash or some other shell for /bin/sh.



I think that the command is being run locally (not on target), and we 
are already in a python context so
it seems that we would have a simpler solution if we handle the IO 
redirection from python.


Perhaps Richard or Joshua can recommend some code that you can 
reference when making this change.


Did you have time to look at the log file handling in runCmd()

https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n168

Please do some analysis and reply here every day or so as  you learn 
things or have questions.



The runCmd() returns the 'Result' object containing information about 
the command execution. It has the following attributes:

   result.command = command
   result.status = cmd.status
   result.output = cmd.output
   result.error = cmd.error
   result.pid = cmd.process.pid
https://git.openembedded.org/openembedded-core/tree/meta/lib/oeqa/utils/commands.py#n198 



I tried to capture the return object value (stderr i.e result.error) and 
print it to the terminal, but that didn't work as expected.
Even I tried to print some debug statements in rust.py file and it also 
didn't show up in the terminal or in the summary.txt file.
I assume there's something in oe-selftest framework that doesn't print 
statements directly.


Also, I see there's a "output_log 
" 
parameter in the runCmd function parameters, which I understand is used 
to redirect stdout of the
command being executed. Currently, I am checking with different values 
by referring to other oe-selftests and their corresponding behavior