[OE-core][scarthgap][PATCH] create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc
From: Mark Hatle While debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 02e262c291c0b2066132b4cb2ca5fda8145284a9) Signed-off-by: Mark Hatle --- meta/classes/create-spdx-2.2.bbclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index d104668ffd..ade1a04be3 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -315,7 +315,8 @@ def add_package_sources_from_debug(d, package_doc, spdx_package, package, packag debugsrc_path = search / debugsrc.replace('/usr/src/kernel/', '') else: debugsrc_path = search / debugsrc.lstrip("/") -if not debugsrc_path.exists(): +# We can only hash files below, skip directories, links, etc. +if not os.path.isfile(debugsrc_path): continue file_sha256 = bb.utils.sha256_file(debugsrc_path) -- 2.39.5 (Apple Git-154) -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#204628): https://lists.openembedded.org/g/openembedded-core/message/204628 Mute This Topic: https://lists.openembedded.org/mt/108508543/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core][scarthgap][PATCH] gcc: Fix spurious '/' in GLIBC_DYNAMIC_LINKER on microblaze
From: Khem Raj Signed-off-by: Khem Raj Cc: Mark Hatle Signed-off-by: Richard Purdie Backport from master OE-Core rev: f0eac82b9a1e4549b7d918df768c369ed7ab5183 Signed-off-by: Mark Hatle --- ...fine-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch b/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch index b0b77dbfa0..9de883c2c7 100644 --- a/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch +++ b/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch @@ -1,4 +1,4 @@ -From aacfd6e14dd583b1fdc65691def61c5e1bc89708 Mon Sep 17 00:00:00 2001 +From 4067ae345f0ff1fbf37c0348f2af09257513b817 Mon Sep 17 00:00:00 2001 From: Khem Raj Date: Fri, 29 Mar 2013 09:24:50 +0400 Subject: [PATCH] Define GLIBC_DYNAMIC_LINKER and UCLIBC_DYNAMIC_LINKER @@ -185,7 +185,7 @@ index aecaa02a199..62f88f7f9a2 100644 #undef GNU_USER_TARGET_LINK_SPEC #define GNU_USER_TARGET_LINK_SPEC \ diff --git a/gcc/config/microblaze/linux.h b/gcc/config/microblaze/linux.h -index e2e2c421c52..6f26480e3b5 100644 +index 5ed8ee518be..299d1a62c81 100644 --- a/gcc/config/microblaze/linux.h +++ b/gcc/config/microblaze/linux.h @@ -28,7 +28,7 @@ @@ -193,7 +193,7 @@ index e2e2c421c52..6f26480e3b5 100644 #define TLS_NEEDS_GOT 1 -#define GLIBC_DYNAMIC_LINKER "/lib/ld.so.1" -+#define GLIBC_DYNAMIC_LINKER SYSTEMLIBS_DIR "/ld.so.1" ++#define GLIBC_DYNAMIC_LINKER SYSTEMLIBS_DIR "ld.so.1" #define UCLIBC_DYNAMIC_LINKER "/lib/ld-uClibc.so.0" #if TARGET_BIG_ENDIAN_DEFAULT == 0 /* LE */ -- 2.44.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#204487): https://lists.openembedded.org/g/openembedded-core/message/204487 Mute This Topic: https://lists.openembedded.org/mt/108435969/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] gcc: Fix spurious '/' in GLIBC_DYNAMIC_LINKER on microblaze
I'm not sure if I'll have time to do Kirkstone, but I will get this backported to scarthgap and will send it up "soon". --Mark On 9/6/24 8:40 AM, Steve Sakoman wrote: On Thu, Sep 5, 2024 at 11:55 PM Khem Raj wrote: This needs backporting to kirkstone and scarthgap as well. This patch understandably doesn't cherry-pick to the earlier versions of gcc in kirkstone and scarthgap. If someone would like to prepare branch appropriate versions I'd be happy to add them to my test queues. Thanks, Steve On Tue, Sep 3, 2024 at 10:25 PM Khem Raj wrote: Signed-off-by: Khem Raj Cc: Mark Hatle --- ...fine-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch b/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch index 6ff16e38e19..4ed55015519 100644 --- a/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch +++ b/meta/recipes-devtools/gcc/gcc/0007-Define-GLIBC_DYNAMIC_LINKER-and-UCLIBC_DYNAMIC_LINKE.patch @@ -1,4 +1,4 @@ -From b597a2bc85ffdcf780731a44de02799ebc3d8f61 Mon Sep 17 00:00:00 2001 +From 4067ae345f0ff1fbf37c0348f2af09257513b817 Mon Sep 17 00:00:00 2001 From: Khem Raj Date: Fri, 29 Mar 2013 09:24:50 +0400 Subject: [PATCH] Define GLIBC_DYNAMIC_LINKER and UCLIBC_DYNAMIC_LINKER @@ -184,7 +184,7 @@ index e5d84e180e3..98cb49c189f 100644 #undef GNU_USER_TARGET_LINK_SPEC #define GNU_USER_TARGET_LINK_SPEC \ diff --git a/gcc/config/microblaze/linux.h b/gcc/config/microblaze/linux.h -index 5ed8ee518be..5553e7cac21 100644 +index 5ed8ee518be..299d1a62c81 100644 --- a/gcc/config/microblaze/linux.h +++ b/gcc/config/microblaze/linux.h @@ -28,7 +28,7 @@ @@ -192,7 +192,7 @@ index 5ed8ee518be..5553e7cac21 100644 #define TLS_NEEDS_GOT 1 -#define GLIBC_DYNAMIC_LINKER "/lib/ld.so.1" -+#define GLIBC_DYNAMIC_LINKER SYSTEMLIBS_DIR "/ld.so.1" ++#define GLIBC_DYNAMIC_LINKER SYSTEMLIBS_DIR "ld.so.1" #define UCLIBC_DYNAMIC_LINKER "/lib/ld-uClibc.so.0" #if TARGET_BIG_ENDIAN_DEFAULT == 0 /* LE */ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#204278): https://lists.openembedded.org/g/openembedded-core/message/204278 Mute This Topic: https://lists.openembedded.org/mt/108260331/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [scarthgap][PATCH 1/8] package.py: Fix static debuginfo split
From: Mark Hatle Fix: NameError: name 'shutil' is not defined Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 13bdd750ae54d57a5f459e4b7d8636c864978241) Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 1511ba4..ffe5a21 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -14,6 +14,7 @@ import glob import stat import mmap import subprocess +import shutil import oe.cachedpath -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202550): https://lists.openembedded.org/g/openembedded-core/message/202550 Mute This Topic: https://lists.openembedded.org/mt/107564320/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [scarthgap][PATCH 8/8] pseudo: Update to include open symlink handling bugfix
From: Richard Purdie Update to a new revision which includes "Bugfix for Linux open(O_CREAT|O_EXCL)" Signed-off-by: Richard Purdie (cherry picked from commit 92a9710ec88c8729fa3d83baa2e63dd74d95cdf8) Signed-off-by: Mark Hatle --- meta/recipes-devtools/pseudo/pseudo_git.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-devtools/pseudo/pseudo_git.bb b/meta/recipes-devtools/pseudo/pseudo_git.bb index 5f32b37..7d8f71f 100644 --- a/meta/recipes-devtools/pseudo/pseudo_git.bb +++ b/meta/recipes-devtools/pseudo/pseudo_git.bb @@ -14,7 +14,7 @@ SRC_URI:append:class-nativesdk = " \ file://older-glibc-symbols.patch" SRC_URI[prebuilt.sha256sum] = "ed9f456856e9d86359f169f46a70ad7be4190d6040282b84c8d97b99072485aa" -SRCREV = "e11ae91da7d0711f5e33ea9dfbf1875dde3c1734" +SRCREV = "374089f2ed83da4d0d4e58df067142ff99c7eb12" S = "${WORKDIR}/git" PV = "1.9.0+git" -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202557): https://lists.openembedded.org/g/openembedded-core/message/202557 Mute This Topic: https://lists.openembedded.org/mt/107564327/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [scarthgap][PATCH 4/8] create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS
From: Mark Hatle When a create-spdx-* classes is processing documents, it needs to find the document in a path that is related to the SSTATE_ARCH when a packge is generated. The SSTATE_ARCH can be affected by multilib configurations, resulting is something like armv8a-mlib. When the image (or SDK) is being generated and the components are collected, the system has no knowledge of the multilib arch and will fail to find it, such as: ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found for package libilp32-libgcc-dbg, False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12: sstate:libilp32-libgcc::14.1.0:r0::12: Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full set of SSTATE_ARCHS including ones that contain the multilib extension which will allow create-spdx-* to correctly find the document it is looking for. This would also be valuable to any other function doing a similar search through SSTATE_ARCH that may have been extended with multilib configurations. Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit f1499c36c1054fc90f7b7268cc95285f2eca72f7) spdx-3.0 items are not application and were removed. spdx-common.bbclass item was moved into create-sdpx-2.2.bbclass. Signed-off-by: Mark Hatle --- meta/classes-recipe/populate_sdk_base.bbclass | 4 meta/classes/create-spdx-2.2.bbclass | 14 -- 2 files changed, 12 insertions(+), 6 deletions(-) diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass index 81896d8..6cb43ad 100644 --- a/meta/classes-recipe/populate_sdk_base.bbclass +++ b/meta/classes-recipe/populate_sdk_base.bbclass @@ -6,6 +6,10 @@ PACKAGES = "" +# This exists as an optimization for SPDX processing to only run in image and +# SDK processing context. This class happens to be common to these usages. +SPDX_MULTILIB_SSTATE_ARCHS = "${@all_multilib_tune_values(d, 'SSTATE_ARCHS')}" + inherit image-postinst-intercepts image-artifact-names # Wildcards specifying complementary packages to install for every package that has been explicitly diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 4ea91f6..d104668 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -35,6 +35,8 @@ SPDX_LICENSES ??= "${COREBASE}/meta/files/spdx-licenses.json" SPDX_CUSTOM_ANNOTATION_VARS ??= "" +SPDX_MULTILIB_SSTATE_ARCHS ??= "${SSTATE_ARCHS}" + SPDX_ORG ??= "OpenEmbedded ()" SPDX_SUPPLIER ??= "Organization: ${SPDX_ORG}" SPDX_SUPPLIER[doc] = "The SPDX PackageSupplier field for SPDX packages created from \ @@ -349,7 +351,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) spdx_deps_file = Path(d.getVar("SPDXDEPS")) -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() dep_recipes = [] @@ -389,7 +391,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): return dep_recipes -collect_dep_recipes[vardepsexclude] = "SSTATE_ARCHS" +collect_dep_recipes[vardepsexclude] = "SPDX_MULTILIB_SSTATE_ARCHS" def collect_dep_sources(d, dep_recipes): import oe.sbom @@ -763,7 +765,7 @@ python do_create_runtime_spdx() { providers = collect_package_providers(d) pkg_arch = d.getVar("SSTATE_PKGARCH") -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() if not is_native: @@ -869,7 +871,7 @@ python do_create_runtime_spdx() { oe.sbom.write_doc(d, runtime_doc, pkg_arch, "runtime", spdx_deploy, indent=get_json_indent(d)) } -do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS" +do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS" addtask do_create_runtime_spdx after do_create_spdx before do_build do_rm_work SSTATETASKS += "do_create_runtime_spdx" @@ -1004,7 +1006,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx import bb.compress.zstd providers = collect_package_providers(d) -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") @@ -1155,4 +1157,4 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx tar.addfile(info, fileobj=index_str) -combine_spdx[vardepsexclude] += "BB_NUMBER_THRE
[OE-core] [scarthgap][PATCH 7/8] pseudo: Fix to work with glibc 2.40
From: Richard Purdie glibc 2.40 renames some internal header variables. Update our hack to work with the new version. These kinds of problems illustrate we need to address the issue properly. Signed-off-by: Richard Purdie (cherry picked from commit 35021d650de3eecc3f42000181b39a5db5a8eaa0) Signed-off-by: Mark Hatle --- meta/recipes-devtools/pseudo/files/glibc238.patch | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/meta/recipes-devtools/pseudo/files/glibc238.patch b/meta/recipes-devtools/pseudo/files/glibc238.patch index da4b8ca..dfb5c28 100644 --- a/meta/recipes-devtools/pseudo/files/glibc238.patch +++ b/meta/recipes-devtools/pseudo/files/glibc238.patch @@ -9,7 +9,7 @@ Index: git/pseudo_wrappers.c === --- git.orig/pseudo_wrappers.c +++ git/pseudo_wrappers.c -@@ -6,6 +6,15 @@ +@@ -6,6 +6,18 @@ * SPDX-License-Identifier: LGPL-2.1-only * */ @@ -21,6 +21,9 @@ Index: git/pseudo_wrappers.c +#undef __GLIBC_USE_ISOC2X +#undef __GLIBC_USE_C2X_STRTOL +#define __GLIBC_USE_C2X_STRTOL 0 ++#undef __GLIBC_USE_ISOC23 ++#undef __GLIBC_USE_C23_STRTOL ++#define __GLIBC_USE_C23_STRTOL 0 + #include #include @@ -29,7 +32,7 @@ Index: git/pseudo_util.c === --- git.orig/pseudo_util.c +++ git/pseudo_util.c -@@ -8,6 +8,14 @@ +@@ -8,6 +8,17 @@ */ /* we need access to RTLD_NEXT for a horrible workaround */ #define _GNU_SOURCE @@ -41,6 +44,9 @@ Index: git/pseudo_util.c +#undef __GLIBC_USE_ISOC2X +#undef __GLIBC_USE_C2X_STRTOL +#define __GLIBC_USE_C2X_STRTOL 0 ++#undef __GLIBC_USE_ISOC23 ++#undef __GLIBC_USE_C23_STRTOL ++#define __GLIBC_USE_C23_STRTOL 0 #include #include -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202556): https://lists.openembedded.org/g/openembedded-core/message/202556 Mute This Topic: https://lists.openembedded.org/mt/107564326/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [scarthgap][PATCH 2/8] package.py: Fix static library processing
From: Mark Hatle When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to hardlinks. This could trigger a race condition during stripping of static libraries where multiple strips (through hardlinks) could run at the same time triggering a truncated or modified file error. The hardlink breaking code is based on the existing code for elf files, but due to the nature of the symlinks needed to be done in a separate block of code. Add support for static-library debugfs hardlinking through the existing inode processing code. Print a note to the logs if the link target can't be found. This isn't strictly an error, but may be useful for debugging an issue where a file isn't present. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit ff371d69f60a1529ed456acb7d8e9305242e74bd) Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 56 ++ 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index ffe5a21..af0923a 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -1065,6 +1065,7 @@ def process_split_and_strip_files(d): d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'): checkelf = {} checkelflinks = {} +checkstatic = {} for root, dirs, files in cpath.walk(dvar): for f in files: file = os.path.join(root, f) @@ -1078,10 +1079,6 @@ def process_split_and_strip_files(d): if file in skipfiles: continue -if oe.package.is_static_lib(file): -staticlibs.append(file) -continue - try: ltarget = cpath.realpath(file, dvar, False) s = cpath.lstat(ltarget) @@ -1093,6 +1090,13 @@ def process_split_and_strip_files(d): continue if not s: continue + +if oe.package.is_static_lib(file): +# Use a reference of device ID and inode number to identify files +file_reference = "%d_%d" % (s.st_dev, s.st_ino) +checkstatic[file] = (file, file_reference) +continue + # Check its an executable if (s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) \ or (s[stat.ST_MODE] & stat.S_IXOTH) \ @@ -1157,6 +1161,27 @@ def process_split_and_strip_files(d): # Modified the file so clear the cache cpath.updatecache(file) +# Do the same hardlink processing as above, but for static libraries +results = list(checkstatic.keys()) + +# As above, sort the results. +results.sort(key=lambda x: x[0]) + +for file in results: +# Use a reference of device ID and inode number to identify files +file_reference = checkstatic[file][1] +if file_reference in inodes: +os.unlink(file) +os.link(inodes[file_reference][0], file) +inodes[file_reference].append(file) +else: +inodes[file_reference] = [file] +# break hardlink +bb.utils.break_hardlinks(file) +staticlibs.append(file) +# Modified the file so clear the cache +cpath.updatecache(file) + def strip_pkgd_prefix(f): nonlocal dvar @@ -1195,11 +1220,24 @@ def process_split_and_strip_files(d): dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] fpath = dvar + dest ftarget = dvar + dv["libdir"] + os.path.dirname(target) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] -bb.utils.mkdirhier(os.path.dirname(fpath)) -# Only one hardlink of separated debug info file in each directory -if not os.access(fpath, os.R_OK): -#bb.note("Link %s -> %s" % (fpath, ftarget)) -os.link(ftarget, fpath) +if os.access(ftarget, os.R_OK): +bb.utils.mkdirhier(os.path.dirname(fpath)) +# Only one hardlink of separated debug info file in each directory +if not os.access(fpath, os.R_OK): +#bb.note("Link %s -> %s" % (fpath, ftarget)) +os.link(ftarget, fpath) +elif (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'): +deststatic = dv["staticlibdir"] + os.path.dirname(src) + dv["s
[OE-core] [scarthgap][PATCH 3/8] selftest-hardlink: Add additional test cases
From: Mark Hatle Additional test cases for debug symlink generation both binaries and static libraries. This also has the side effect of testing for race conditions in the hardlink debug generation and stripping. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit 7171f41c07a39a7543bb64f075d38b8e74563089) Signed-off-by: Mark Hatle --- .../selftest-hardlink/selftest-hardlink.bb | 13 +++ meta/lib/oeqa/selftest/cases/package.py| 26 ++ 2 files changed, 39 insertions(+) diff --git a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb index be346b8..052bf0c 100644 --- a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb +++ b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb @@ -10,6 +10,9 @@ S = "${WORKDIR}" do_compile () { ${CC} hello.c -o hello1 ${CFLAGS} ${LDFLAGS} + + ${CC} hello.c -c -o hello.o ${CFLAGS} + ${AR} rcs libhello.a hello.o } do_install () { @@ -22,9 +25,19 @@ do_install () { ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello3 ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello4 + # We need so many hardlink copies to look for specific race conditions + install -d ${D}${libdir} + install -m 0644 libhello.a ${D}${libdir} + for num in `seq 1 100` ; do + ln ${D}${libdir}/libhello.a ${D}${libdir}/libhello-${num}.a + done + dd if=/dev/zero of=${D}${bindir}/sparsetest bs=1 count=0 seek=1M } RDEPENDS:${PN}-gdb += "gdb" PACKAGES =+ "${PN}-gdb" FILES:${PN}-gdb = "${bindir}/gdb.sh" + +PACKAGE_STRIP_STATIC = "1" +PACKAGE_DEBUG_STATIC_SPLIT = "1" diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py index 1aa6c03..38ed717 100644 --- a/meta/lib/oeqa/selftest/cases/package.py +++ b/meta/lib/oeqa/selftest/cases/package.py @@ -103,11 +103,37 @@ class PackageTests(OESelftestTestCase): dest = get_bb_var('PKGDEST', 'selftest-hardlink') bindir = get_bb_var('bindir', 'selftest-hardlink') +libdir = get_bb_var('libdir', 'selftest-hardlink') +libexecdir = get_bb_var('libexecdir', 'selftest-hardlink') def checkfiles(): # Recipe creates 4 hardlinked files, there is a copy in package/ and a copy in packages-split/ # so expect 8 in total. self.assertEqual(os.stat(dest + "/selftest-hardlink" + bindir + "/hello1").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink" + libexecdir + "/hello3").st_nlink, 8) + +# Check dbg version +# 2 items, a copy in both package/packages-split so 4 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + bindir + "/.debug/hello1").st_nlink, 4) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello1").st_nlink, 4) + +# Even though the libexecdir name is 'hello3' or 'hello4', that isn't the debug target name +self.assertEqual(os.path.exists(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello3"), False) +self.assertEqual(os.path.exists(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello4"), False) + +# Check the staticdev libraries +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-75.a").st_nlink, 202) + +# Check static dbg +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + &quo
[OE-core] [scarthgap][PATCH 6/8] oeqa sdk cases: Skip SDK test cases when TCLIBC is newlib
From: Mark Hatle Newlib generally requires additional components to function. Skip the cases where newlib is known to not work. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie (cherry picked from commit b993474e40d9980b90c3d541f4c702203561) Signed-off-by: Mark Hatle --- meta/lib/oeqa/sdk/cases/assimp.py | 4 meta/lib/oeqa/sdk/cases/buildcpio.py | 5 + meta/lib/oeqa/sdk/cases/buildepoxy.py | 4 meta/lib/oeqa/sdk/cases/buildgalculator.py | 4 meta/lib/oeqa/sdk/cases/buildlzip.py | 5 + meta/lib/oeqa/sdk/cases/gcc.py | 4 6 files changed, 26 insertions(+) diff --git a/meta/lib/oeqa/sdk/cases/assimp.py b/meta/lib/oeqa/sdk/cases/assimp.py index d990b1e..4cc30f2 100644 --- a/meta/lib/oeqa/sdk/cases/assimp.py +++ b/meta/lib/oeqa/sdk/cases/assimp.py @@ -19,6 +19,10 @@ class BuildAssimp(OESDKTestCase): """ def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("CMakeTest class: SDK doesn't contain a supported C library") + if not (self.tc.hasHostPackage("nativesdk-cmake") or self.tc.hasHostPackage("cmake-native")): raise unittest.SkipTest("Needs cmake") diff --git a/meta/lib/oeqa/sdk/cases/buildcpio.py b/meta/lib/oeqa/sdk/cases/buildcpio.py index 51003b1..ab8fc41 100644 --- a/meta/lib/oeqa/sdk/cases/buildcpio.py +++ b/meta/lib/oeqa/sdk/cases/buildcpio.py @@ -17,6 +17,11 @@ class BuildCpioTest(OESDKTestCase): """ Check that autotools will cross-compile correctly. """ +def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("AutotoolsTest class: SDK doesn't contain a supported C library") + def test_cpio(self): with tempfile.TemporaryDirectory(prefix="cpio-", dir=self.tc.sdk_dir) as testdir: tarball = self.fetch(testdir, self.td["DL_DIR"], "https://ftp.gnu.org/gnu/cpio/cpio-2.15.tar.gz";) diff --git a/meta/lib/oeqa/sdk/cases/buildepoxy.py b/meta/lib/oeqa/sdk/cases/buildepoxy.py index 147ee3e..5b9c36f 100644 --- a/meta/lib/oeqa/sdk/cases/buildepoxy.py +++ b/meta/lib/oeqa/sdk/cases/buildepoxy.py @@ -18,6 +18,10 @@ class EpoxyTest(OESDKTestCase): Test that Meson builds correctly. """ def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("MesonTest class: SDK doesn't contain a supported C library") + if not (self.tc.hasHostPackage("nativesdk-meson") or self.tc.hasHostPackage("meson-native")): raise unittest.SkipTest("EpoxyTest class: SDK doesn't contain Meson") diff --git a/meta/lib/oeqa/sdk/cases/buildgalculator.py b/meta/lib/oeqa/sdk/cases/buildgalculator.py index 178f074..2818743 100644 --- a/meta/lib/oeqa/sdk/cases/buildgalculator.py +++ b/meta/lib/oeqa/sdk/cases/buildgalculator.py @@ -18,6 +18,10 @@ class GalculatorTest(OESDKTestCase): Test that autotools and GTK+ 3 compiles correctly. """ def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("GTK3Test class: SDK doesn't contain a supported C library") + if not (self.tc.hasTargetPackage("gtk+3", multilib=True) or \ self.tc.hasTargetPackage("libgtk-3.0", multilib=True)): raise unittest.SkipTest("GalculatorTest class: SDK don't support gtk+3") diff --git a/meta/lib/oeqa/sdk/cases/buildlzip.py b/meta/lib/oeqa/sdk/cases/buildlzip.py index b4b7d85..afedc25 100644 --- a/meta/lib/oeqa/sdk/cases/buildlzip.py +++ b/meta/lib/oeqa/sdk/cases/buildlzip.py @@ -13,6 +13,11 @@ class BuildLzipTest(OESDKTestCase): """ Test that "plain" compilation works, using just $CC $CFLAGS etc. """ +def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("MakefileTest class: SDK doesn't contain a supported C library") + def test_lzip(self): with tempfile.TemporaryDirectory(prefix="lzip", dir=self.tc.sdk_dir) as testdir: tarball = self.fetch(testdir, self.td["DL_DIR"], "http://downloads.yoctoproject.org/mirror/sources/lzip-1.19.tar.gz";) diff --git a/meta/lib/oeqa/sdk/cases/gcc.py b/meta/lib/oeqa/sdk/cases/gcc.py index fc28b9c..e810d2c 100644 --- a/meta/lib/oeqa/sdk/cases/gcc.py +++ b/meta/lib/oeqa/sdk/cases/
[OE-core] [scarthgap][PATCH 5/8] create-spdx-3.0/populate_sdk_base: Add SDK_CLASSES inherit mechanism to fix tarball SPDX manifests
From: Richard Purdie Currently, "tarball" sdk based recipes don't generate SPDX manifests as they don't include the rootfs generation classes. Split the SPDX 3.0 image class into two so the SDK components can be included where needed. To do this, introduce an SDK_CLASSES variable similar to IMAGE_CLASSES which the SDK code can use. Migrate testsdk usage to this. Also move the image/sdk spdx classes to classes-recipe rather than the general classes directory since they'd never be included on a global level. For buildtools-tarball, it has its own testsdk functions so disable the class there as a deferred inherit would overwrite it. Signed-off-by: Richard Purdie (cherry picked from commit 662396533177b72cc1d83e95841b27f7e42dcb20) Eliminate spdx-3.0 items, not applicable to Scarthgap. Signed-off-by: Mark Hatle --- meta/classes-recipe/populate_sdk_base.bbclass | 3 +++ meta/classes-recipe/testimage.bbclass | 2 -- meta/recipes-core/meta/buildtools-tarball.bb | 3 +++ 3 files changed, 6 insertions(+), 2 deletions(-) diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass index 6cb43ad..a103e7b 100644 --- a/meta/classes-recipe/populate_sdk_base.bbclass +++ b/meta/classes-recipe/populate_sdk_base.bbclass @@ -4,6 +4,9 @@ # SPDX-License-Identifier: MIT # +SDK_CLASSES += "${@bb.utils.contains("IMAGE_CLASSES", "testimage", "testsdk", "", d)}" +inherit_defer ${SDK_CLASSES} + PACKAGES = "" # This exists as an optimization for SPDX processing to only run in image and diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass index ed0d87b..2f68f83 100644 --- a/meta/classes-recipe/testimage.bbclass +++ b/meta/classes-recipe/testimage.bbclass @@ -483,5 +483,3 @@ python () { if oe.types.boolean(d.getVar("TESTIMAGE_AUTO") or "False"): bb.build.addtask("testimage", "do_build", "do_image_complete", d) } - -inherit testsdk diff --git a/meta/recipes-core/meta/buildtools-tarball.bb b/meta/recipes-core/meta/buildtools-tarball.bb index 92fbda3..e2ce5b3 100644 --- a/meta/recipes-core/meta/buildtools-tarball.bb +++ b/meta/recipes-core/meta/buildtools-tarball.bb @@ -112,6 +112,9 @@ INHIBIT_DEFAULT_DEPS = "1" # Directory in testsdk that contains testcases TESTSDK_CASES = "buildtools-cases" +# We have our own code, avoid deferred inherit +SDK_CLASSES:remove = "testsdk" + python do_testsdk() { import oeqa.sdk.testsdk testsdk = oeqa.sdk.testsdk.TestSDK() -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202554): https://lists.openembedded.org/g/openembedded-core/message/202554 Mute This Topic: https://lists.openembedded.org/mt/107564324/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] oeqa sdk cases: Skip SDK test cases when TCLIBC is newlib
From: Mark Hatle Newlib generally requires additional components to function. Skip the cases where newlib is known to not work. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- I'm not entirely sure the patch is right, but it provides a mechanism to exclude specific TCLIBCs from certain sdktests. I only added newlib, since that is all I'm using. meta/lib/oeqa/sdk/cases/autotools.py | 6 ++ meta/lib/oeqa/sdk/cases/cmake.py | 4 meta/lib/oeqa/sdk/cases/gcc.py | 4 meta/lib/oeqa/sdk/cases/gtk3.py | 4 meta/lib/oeqa/sdk/cases/makefile.py | 6 ++ meta/lib/oeqa/sdk/cases/meson.py | 4 6 files changed, 28 insertions(+) diff --git a/meta/lib/oeqa/sdk/cases/autotools.py b/meta/lib/oeqa/sdk/cases/autotools.py index 848e9392ec..4bac28f04d 100644 --- a/meta/lib/oeqa/sdk/cases/autotools.py +++ b/meta/lib/oeqa/sdk/cases/autotools.py @@ -7,6 +7,7 @@ import os import tempfile import subprocess +import unittest from oeqa.sdk.case import OESDKTestCase from oeqa.utils.subprocesstweak import errors_have_output @@ -16,6 +17,11 @@ class AutotoolsTest(OESDKTestCase): """ Check that autotools will cross-compile correctly. """ +def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("AutotoolsTest class: SDK doesn't contain a supported C library") + def test_cpio(self): with tempfile.TemporaryDirectory(prefix="cpio-", dir=self.tc.sdk_dir) as testdir: tarball = self.fetch(testdir, self.td["DL_DIR"], "https://ftp.gnu.org/gnu/cpio/cpio-2.15.tar.gz";) diff --git a/meta/lib/oeqa/sdk/cases/cmake.py b/meta/lib/oeqa/sdk/cases/cmake.py index db7d826a38..cb0944ee99 100644 --- a/meta/lib/oeqa/sdk/cases/cmake.py +++ b/meta/lib/oeqa/sdk/cases/cmake.py @@ -19,6 +19,10 @@ class CMakeTest(OESDKTestCase): """ def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("CMakeTest class: SDK doesn't contain a supported C library") + if not (self.tc.hasHostPackage("nativesdk-cmake") or self.tc.hasHostPackage("cmake-native")): raise unittest.SkipTest("CMakeTest: needs cmake") diff --git a/meta/lib/oeqa/sdk/cases/gcc.py b/meta/lib/oeqa/sdk/cases/gcc.py index fc28b9c3d4..e810d2c42b 100644 --- a/meta/lib/oeqa/sdk/cases/gcc.py +++ b/meta/lib/oeqa/sdk/cases/gcc.py @@ -26,6 +26,10 @@ class GccCompileTest(OESDKTestCase): os.path.join(self.tc.sdk_dir, f)) def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("GccCompileTest class: SDK doesn't contain a supported C library") + machine = self.td.get("MACHINE") if not (self.tc.hasHostPackage("packagegroup-cross-canadian-%s" % machine) or self.tc.hasHostPackage("^gcc-", regex=True)): diff --git a/meta/lib/oeqa/sdk/cases/gtk3.py b/meta/lib/oeqa/sdk/cases/gtk3.py index c329c4bb86..8f60d5e7da 100644 --- a/meta/lib/oeqa/sdk/cases/gtk3.py +++ b/meta/lib/oeqa/sdk/cases/gtk3.py @@ -18,6 +18,10 @@ class GTK3Test(OESDKTestCase): Test that autotools and GTK+ 3 compiles correctly. """ def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("GTK3Test class: SDK doesn't contain a supported C library") + if not (self.tc.hasTargetPackage("gtk+3", multilib=True) or \ self.tc.hasTargetPackage("libgtk-3.0", multilib=True)): raise unittest.SkipTest("GalculatorTest class: SDK don't support gtk+3") diff --git a/meta/lib/oeqa/sdk/cases/makefile.py b/meta/lib/oeqa/sdk/cases/makefile.py index 2ff54ce25f..e1e2484820 100644 --- a/meta/lib/oeqa/sdk/cases/makefile.py +++ b/meta/lib/oeqa/sdk/cases/makefile.py @@ -5,6 +5,7 @@ # import os, tempfile, subprocess +import unittest from oeqa.sdk.case import OESDKTestCase from oeqa.utils.subprocesstweak import errors_have_output errors_have_output() @@ -13,6 +14,11 @@ class MakefileTest(OESDKTestCase): """ Test that "plain" compilation works, using just $CC $CFLAGS etc. """ +def setUp(self): +libc = self.td.get("TCLIBC") +if libc in [ 'newlib' ]: +raise unittest.SkipTest("MakefileTest class: SDK doesn't contain a supported C library") + def test_lzip(self): with tempfile.TemporaryDirectory(prefix="lzip", dir=self.tc.sdk_dir) as testdir: ta
[OE-core] [PATCH 2/2] create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS
From: Mark Hatle When a create-spdx-* classes is processing documents, it needs to find the document in a path that is related to the SSTATE_ARCH when a packge is generated. The SSTATE_ARCH can be affected by multilib configurations, resulting is something like armv8a-mlib. When the image (or SDK) is being generated and the components are collected, the system has no knowledge of the multilib arch and will fail to find it, such as: ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found for package libilp32-libgcc-dbg, False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12: sstate:libilp32-libgcc::14.1.0:r0::12: Adding in the new SPDX_MULTILIB_SSTATE_ARCHS will provide a full set of SSTATE_ARCHS including ones that contain the multilib extension which will allow create-spdx-* to correctly find the document it is looking for. This would also be valuable to any other function doing a similar search through SSTATE_ARCH that may have been extended with multilib configurations. Signed-off-by: Mark Hatle --- meta/classes-recipe/populate_sdk_base.bbclass | 4 meta/classes/create-spdx-2.2.bbclass | 12 ++-- meta/classes/create-spdx-3.0.bbclass | 4 ++-- meta/classes/spdx-common.bbclass | 2 ++ meta/lib/oe/sbom30.py | 2 +- 5 files changed, 15 insertions(+), 9 deletions(-) diff --git a/meta/classes-recipe/populate_sdk_base.bbclass b/meta/classes-recipe/populate_sdk_base.bbclass index 8d79d88ebe..61a8b5e300 100644 --- a/meta/classes-recipe/populate_sdk_base.bbclass +++ b/meta/classes-recipe/populate_sdk_base.bbclass @@ -6,6 +6,10 @@ PACKAGES = "" +# This exists as an optimization for SPDX processing to only run in image and +# SDK processing context. This class happens to be common to these usages. +SPDX_MULTILIB_SSTATE_ARCHS = "${@all_multilib_tune_values(d, 'SSTATE_ARCHS')}" + inherit image-postinst-intercepts image-artifact-names # Wildcards specifying complementary packages to install for every package that has been explicitly diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 12b78cb3f6..509d3b58b6 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -267,7 +267,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): import oe.spdx_common deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() dep_recipes = [] @@ -312,7 +312,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): return dep_recipes -collect_dep_recipes[vardepsexclude] = "SSTATE_ARCHS" +collect_dep_recipes[vardepsexclude] = "SPDX_MULTILIB_SSTATE_ARCHS" def collect_dep_sources(d, dep_recipes): import oe.sbom @@ -610,7 +610,7 @@ python do_create_runtime_spdx() { providers = oe.spdx_common.collect_package_providers(d) pkg_arch = d.getVar("SSTATE_PKGARCH") -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() if not is_native: @@ -716,7 +716,7 @@ python do_create_runtime_spdx() { oe.sbom.write_doc(d, runtime_doc, pkg_arch, "runtime", spdx_deploy, indent=get_json_indent(d)) } -do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS" +do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SPDX_MULTILIB_SSTATE_ARCHS" addtask do_create_runtime_spdx after do_create_spdx before do_build do_rm_work SSTATETASKS += "do_create_runtime_spdx" @@ -798,7 +798,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx import bb.compress.zstd providers = oe.spdx_common.collect_package_providers(d) -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("SPDX_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") @@ -950,4 +950,4 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx tar.addfile(info, fileobj=index_str) -combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SSTATE_ARCHS" +combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SPDX_MULTILIB_SSTATE_ARCHS" diff --git a/meta/classes/create-spdx-3.0.bbclass b/meta/classes/create-spdx-3.0.bbclass index 41840d9d1a..2eb44ba808 100644 --- a/meta/classes/create-spdx-3.0.bbclass +++ b/meta/classes/create-spdx-3.0.bbclass @@ -118,7 +118,7 @@ IMAGE_CLASSES:append = " create-spdx-image-3.0" oe.spdx30_tasks.set_timestamp_now[vardepsexclude] = "SPDX_
[OE-core] [PATCH 0/2] SDKs w/ multilibs and spdx30 'isfile'
From: Mark Hatle This resolves problems when building an SDK that includes multilibs. It also adds the isfile change that was previou put in for spdx 2.2. Introduce a new multilib SSTATE_ARCHs, as an optimization this just defaults to SSTATE_ARCHs in most cases. If building an SDK we do expand it with the multilibs. Additionally fixes the issue where were a debug component may refer to something that is not a file. This change was already put into the spdx 2.2 class. See: commit a798d00d54a1424f9972de43ff05a0ca8950d7de Author: Mark Hatle Date: Mon Jul 15 14:56:06 2024 -0500 create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc While debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents. (From OE-Core rev: 02e262c291c0b2066132b4cb2ca5fda8145284a9) Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie Mark Hatle (2): spdx30_tasks.py: switch from exists to isfile checking debugsrc create-spdx-*: Support multilibs via SPDX_MULTILIB_SSTATE_ARCHS meta/classes-recipe/populate_sdk_base.bbclass | 4 meta/classes/create-spdx-2.2.bbclass | 12 ++-- meta/classes/create-spdx-3.0.bbclass | 4 ++-- meta/classes/spdx-common.bbclass | 2 ++ meta/lib/oe/sbom30.py | 2 +- meta/lib/oe/spdx30_tasks.py | 3 ++- 6 files changed, 17 insertions(+), 10 deletions(-) -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202492): https://lists.openembedded.org/g/openembedded-core/message/202492 Mute This Topic: https://lists.openembedded.org/mt/107533770/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] spdx30_tasks.py: switch from exists to isfile checking debugsrc
From: Mark Hatle Same change as previously made to the create-spdx-2.2.bbclass, while debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents. Signed-off-by: Mark Hatle --- meta/lib/oe/spdx30_tasks.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/lib/oe/spdx30_tasks.py b/meta/lib/oe/spdx30_tasks.py index 7baa6be70e..9d5bbadc0f 100644 --- a/meta/lib/oe/spdx30_tasks.py +++ b/meta/lib/oe/spdx30_tasks.py @@ -238,7 +238,8 @@ def get_package_sources_from_debug( if file_sha256 is None: continue else: -if not debugsrc_path.exists(): +# We can only hash files below, skip directories, links, etc. +if not debugsrc_path.isfile(): source_hash_cache[debugsrc_path] = None continue -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202491): https://lists.openembedded.org/g/openembedded-core/message/202491 Mute This Topic: https://lists.openembedded.org/mt/107533769/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [scarthgap] package.py and selftest-hardlink backport request
I'm requesting the following changes be backported to scarthgap. The commits should apply directly. Any questions, let me know. commit def276f0c4c43fa4b95b7b38a944de7cc7e9286f Author: Mark Hatle Date: Fri Jul 19 13:58:16 2024 -0500 package.py: Fix static debuginfo split Fix: NameError: name 'shutil' is not defined (From OE-Core rev: 13bdd750ae54d57a5f459e4b7d8636c864978241) Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie commit fa894486a99237b56469354c24ef913a96753aaf Author: Mark Hatle Date: Fri Jul 19 13:58:17 2024 -0500 package.py: Fix static library processing When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to hardlinks. This could trigger a race condition during stripping of static libraries where multiple strips (through hardlinks) could run at the same time triggering a truncated or modified file error. The hardlink breaking code is based on the existing code for elf files, but due to the nature of the symlinks needed to be done in a separate block of code. Add support for static-library debugfs hardlinking through the existing inode processing code. Print a note to the logs if the link target can't be found. This isn't strictly an error, but may be useful for debugging an issue where a file isn't present. (From OE-Core rev: ff371d69f60a1529ed456acb7d8e9305242e74bd) Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie commit 4462724cab96ed3990fdf9b30edcea2adc9095ee Author: Mark Hatle Date: Fri Jul 19 13:58:18 2024 -0500 selftest-hardlink: Add additional test cases Additional test cases for debug symlink generation both binaries and static libraries. This also has the side effect of testing for race conditions in the hardlink debug generation and stripping. (From OE-Core rev: 7171f41c07a39a7543bb64f075d38b8e74563089) Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle Signed-off-by: Richard Purdie -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202413): https://lists.openembedded.org/g/openembedded-core/message/202413 Mute This Topic: https://lists.openembedded.org/mt/107508544/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v3 0/3] Fix static-library related issues
Fixes and test case for static-library related items. v3: Drop debug hard link change. My understanding of what was supposed to happen was incorrect. I've updated the self-tests to hopefully make it clearer that the additional links should NOT happen. Note, the hardlinks for static libraries SHOULD happen though! v2: The first three patches were combined into a single patch originally. This has been split apart to make it more clear the three individual changes. Mark Hatle (3): package.py: Fix static debuginfo split package.py: Fix static library processing selftest-hardlink: Add additional test cases .../selftest-hardlink/selftest-hardlink.bb | 13 + meta/lib/oe/package.py | 57 ++ meta/lib/oeqa/selftest/cases/package.py| 26 ++ 3 files changed, 87 insertions(+), 9 deletions(-) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202273): https://lists.openembedded.org/g/openembedded-core/message/202273 Mute This Topic: https://lists.openembedded.org/mt/107440533/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v3 3/3] selftest-hardlink: Add additional test cases
From: Mark Hatle Additional test cases for debug symlink generation both binaries and static libraries. This also has the side effect of testing for race conditions in the hardlink debug generation and stripping. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- .../selftest-hardlink/selftest-hardlink.bb | 13 +++ meta/lib/oeqa/selftest/cases/package.py| 26 ++ 2 files changed, 39 insertions(+) diff --git a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb index 5632bda..64fea8e 100644 --- a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb +++ b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb @@ -11,6 +11,9 @@ UNPACKDIR = "${S}" do_compile () { ${CC} hello.c -o hello1 ${CFLAGS} ${LDFLAGS} + + ${CC} hello.c -c -o hello.o ${CFLAGS} + ${AR} rcs libhello.a hello.o } do_install () { @@ -23,9 +26,19 @@ do_install () { ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello3 ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello4 + # We need so many hardlink copies to look for specific race conditions + install -d ${D}${libdir} + install -m 0644 libhello.a ${D}${libdir} + for num in `seq 1 100` ; do + ln ${D}${libdir}/libhello.a ${D}${libdir}/libhello-${num}.a + done + dd if=/dev/zero of=${D}${bindir}/sparsetest bs=1 count=0 seek=1M } RDEPENDS:${PN}-gdb += "gdb" PACKAGES =+ "${PN}-gdb" FILES:${PN}-gdb = "${bindir}/gdb.sh" + +PACKAGE_STRIP_STATIC = "1" +PACKAGE_DEBUG_STATIC_SPLIT = "1" diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py index 1aa6c03..c1b2e23 100644 --- a/meta/lib/oeqa/selftest/cases/package.py +++ b/meta/lib/oeqa/selftest/cases/package.py @@ -103,11 +103,37 @@ class PackageTests(OESelftestTestCase): dest = get_bb_var('PKGDEST', 'selftest-hardlink') bindir = get_bb_var('bindir', 'selftest-hardlink') +libdir = get_bb_var('libdir', 'selftest-hardlink') +libexecdir = get_bb_var('libexecdir', 'selftest-hardlink') def checkfiles(): # Recipe creates 4 hardlinked files, there is a copy in package/ and a copy in packages-split/ # so expect 8 in total. self.assertEqual(os.stat(dest + "/selftest-hardlink" + bindir + "/hello1").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink" + libexecdir + "/hello3").st_nlink, 8) + +# Check dbg version +# 2 items, a copy in both package/packages-split so 4 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + bindir + "/.debug/hello1").st_nlink, 4) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello1").st_nlink, 4) + +# Even though the libexecdir name is 'hello3' or 'hello4', that isn't the debug target name +self.assertEqual(os.path.exists(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello3"), False) +self.assertEqual(os.path.exists(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello4"), False) + +# Check the staticdev libraries +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-75.a").st_nlink, 202) + +# Check static dbg +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-75.a").st_nlink, 202) # Test a sparse file remains sparse sparses
[OE-core] [PATCH v3 2/3] package.py: Fix static library processing
From: Mark Hatle When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to hardlinks. This could trigger a race condition during stripping of static libraries where multiple strips (through hardlinks) could run at the same time triggering a truncated or modified file error. The hardlink breaking code is based on the existing code for elf files, but due to the nature of the symlinks needed to be done in a separate block of code. Add support for static-library debugfs hardlinking through the existing inode processing code. Print a note to the logs if the link target can't be found. This isn't strictly an error, but may be useful for debugging an issue where a file isn't present. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 56 ++ 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 235f2d6..c213a9a 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -1079,6 +1079,7 @@ def process_split_and_strip_files(d): d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'): checkelf = {} checkelflinks = {} +checkstatic = {} for root, dirs, files in cpath.walk(dvar): for f in files: file = os.path.join(root, f) @@ -1092,10 +1093,6 @@ def process_split_and_strip_files(d): if file in skipfiles: continue -if oe.package.is_static_lib(file): -staticlibs.append(file) -continue - try: ltarget = cpath.realpath(file, dvar, False) s = cpath.lstat(ltarget) @@ -1107,6 +1104,13 @@ def process_split_and_strip_files(d): continue if not s: continue + +if oe.package.is_static_lib(file): +# Use a reference of device ID and inode number to identify files +file_reference = "%d_%d" % (s.st_dev, s.st_ino) +checkstatic[file] = (file, file_reference) +continue + # Check its an executable if (s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) \ or (s[stat.ST_MODE] & stat.S_IXOTH) \ @@ -1171,6 +1175,27 @@ def process_split_and_strip_files(d): # Modified the file so clear the cache cpath.updatecache(file) +# Do the same hardlink processing as above, but for static libraries +results = list(checkstatic.keys()) + +# As above, sort the results. +results.sort(key=lambda x: x[0]) + +for file in results: +# Use a reference of device ID and inode number to identify files +file_reference = checkstatic[file][1] +if file_reference in inodes: +os.unlink(file) +os.link(inodes[file_reference][0], file) +inodes[file_reference].append(file) +else: +inodes[file_reference] = [file] +# break hardlink +bb.utils.break_hardlinks(file) +staticlibs.append(file) +# Modified the file so clear the cache +cpath.updatecache(file) + def strip_pkgd_prefix(f): nonlocal dvar @@ -1209,11 +1234,24 @@ def process_split_and_strip_files(d): dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] fpath = dvar + dest ftarget = dvar + dv["libdir"] + os.path.dirname(target) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] -bb.utils.mkdirhier(os.path.dirname(fpath)) -# Only one hardlink of separated debug info file in each directory -if not os.access(fpath, os.R_OK): -#bb.note("Link %s -> %s" % (fpath, ftarget)) -os.link(ftarget, fpath) +if os.access(ftarget, os.R_OK): +bb.utils.mkdirhier(os.path.dirname(fpath)) +# Only one hardlink of separated debug info file in each directory +if not os.access(fpath, os.R_OK): +#bb.note("Link %s -> %s" % (fpath, ftarget)) +os.link(ftarget, fpath) +elif (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'): +deststatic = dv["staticlibdir"] + os.path.dirname(src) + dv["staticdir"] + "/" + os.path.basename(file) + dv["staticappend"] +fpath = dv
[OE-core] [PATCH v3 1/3] package.py: Fix static debuginfo split
From: Mark Hatle Fix: NameError: name 'shutil' is not defined Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 1 + 1 file changed, 1 insertion(+) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index e6b46a0..235f2d6 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -14,6 +14,7 @@ import glob import stat import mmap import subprocess +import shutil import oe.cachedpath -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202272): https://lists.openembedded.org/g/openembedded-core/message/202272 Mute This Topic: https://lists.openembedded.org/mt/107440532/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 0/4] Fix static-library related issues
Fixes and test case for static-library related items. v2: The first three patches were combined into a single patch originally. This has been split apart to make it more clear the three individual changes. Mark Hatle (4): package.py: Fix static debuginfo split package.py: Fix inode link target package.py: Fix static library processing selftest-hardlink: Add additional test cases .../selftest-hardlink/selftest-hardlink.bb | 13 + meta/lib/oe/package.py | 60 ++ meta/lib/oeqa/selftest/cases/package.py| 24 + 3 files changed, 87 insertions(+), 10 deletions(-) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202242): https://lists.openembedded.org/g/openembedded-core/message/202242 Mute This Topic: https://lists.openembedded.org/mt/107426719/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 4/4] selftest-hardlink: Add additional test cases
From: Mark Hatle Additional test cases for debug symlink generation both binaries and static libraries. This also has the side effect of testing for race conditions in the hardlink debug generation and stripping. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- .../selftest-hardlink/selftest-hardlink.bb | 13 meta/lib/oeqa/selftest/cases/package.py| 24 ++ 2 files changed, 37 insertions(+) diff --git a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb index 5632bda..64fea8e 100644 --- a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb +++ b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb @@ -11,6 +11,9 @@ UNPACKDIR = "${S}" do_compile () { ${CC} hello.c -o hello1 ${CFLAGS} ${LDFLAGS} + + ${CC} hello.c -c -o hello.o ${CFLAGS} + ${AR} rcs libhello.a hello.o } do_install () { @@ -23,9 +26,19 @@ do_install () { ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello3 ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello4 + # We need so many hardlink copies to look for specific race conditions + install -d ${D}${libdir} + install -m 0644 libhello.a ${D}${libdir} + for num in `seq 1 100` ; do + ln ${D}${libdir}/libhello.a ${D}${libdir}/libhello-${num}.a + done + dd if=/dev/zero of=${D}${bindir}/sparsetest bs=1 count=0 seek=1M } RDEPENDS:${PN}-gdb += "gdb" PACKAGES =+ "${PN}-gdb" FILES:${PN}-gdb = "${bindir}/gdb.sh" + +PACKAGE_STRIP_STATIC = "1" +PACKAGE_DEBUG_STATIC_SPLIT = "1" diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py index 1aa6c03..71231f5 100644 --- a/meta/lib/oeqa/selftest/cases/package.py +++ b/meta/lib/oeqa/selftest/cases/package.py @@ -103,11 +103,35 @@ class PackageTests(OESelftestTestCase): dest = get_bb_var('PKGDEST', 'selftest-hardlink') bindir = get_bb_var('bindir', 'selftest-hardlink') +libdir = get_bb_var('libdir', 'selftest-hardlink') +libexecdir = get_bb_var('libexecdir', 'selftest-hardlink') def checkfiles(): # Recipe creates 4 hardlinked files, there is a copy in package/ and a copy in packages-split/ # so expect 8 in total. self.assertEqual(os.stat(dest + "/selftest-hardlink" + bindir + "/hello1").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink" + libexecdir + "/hello3").st_nlink, 8) + +# Check dbg version +# 4 items, a copy in both package/packages-split so 8 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + bindir + "/.debug/hello1").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + bindir + "/.debug/hello2").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello3").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello4").st_nlink, 8) + +# Check the staticdev libraries +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-75.a").st_nlink, 202) + +# Check static dbg +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-75.a").st_nlink, 202) # Test a sparse file remains sparse sparsestat = os.stat(dest + "/selftest-hardlink" + bindir + "/sparsetest") -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202241): https://lists.openembedded.org/g/openembedded-core/message/202241 Mute This Topic: https://lists.openembedded.org/mt/107426718/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 3/4] package.py: Fix static library processing
From: Mark Hatle When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to hardlinks. This could trigger a race condition during stripping of static libraries where multiple strips (through hardlinks) could run at the same time triggering a truncated or modified file error. The hardlink breaking code is based on the existing code for elf files, but due to the nature of the symlinks needed to be done in a separate block of code. Add support for static-library debugfs hardlinking through the existing inode processing code. Print a note to the logs if the link target can't be found. This isn't strictly an error, but may be useful for debugging an issue where a file isn't present. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 56 ++ 1 file changed, 47 insertions(+), 9 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index ce599f1..dbbda9f 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -1080,6 +1080,7 @@ def process_split_and_strip_files(d): d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'): checkelf = {} checkelflinks = {} +checkstatic = {} for root, dirs, files in cpath.walk(dvar): for f in files: file = os.path.join(root, f) @@ -1093,10 +1094,6 @@ def process_split_and_strip_files(d): if file in skipfiles: continue -if oe.package.is_static_lib(file): -staticlibs.append(file) -continue - try: ltarget = cpath.realpath(file, dvar, False) s = cpath.lstat(ltarget) @@ -1108,6 +1105,13 @@ def process_split_and_strip_files(d): continue if not s: continue + +if oe.package.is_static_lib(file): +# Use a reference of device ID and inode number to identify files +file_reference = "%d_%d" % (s.st_dev, s.st_ino) +checkstatic[file] = (file, file_reference) +continue + # Check its an executable if (s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) \ or (s[stat.ST_MODE] & stat.S_IXOTH) \ @@ -1172,6 +1176,27 @@ def process_split_and_strip_files(d): # Modified the file so clear the cache cpath.updatecache(file) +# Do the same hardlink processing as above, but for static libraries +results = list(checkstatic.keys()) + +# As above, sort the results. +results.sort(key=lambda x: x[0]) + +for file in results: +# Use a reference of device ID and inode number to identify files +file_reference = checkstatic[file][1] +if file_reference in inodes: +os.unlink(file) +os.link(inodes[file_reference][0], file) +inodes[file_reference].append(file) +else: +inodes[file_reference] = [file] +# break hardlink +bb.utils.break_hardlinks(file) +staticlibs.append(file) +# Modified the file so clear the cache +cpath.updatecache(file) + def strip_pkgd_prefix(f): nonlocal dvar @@ -1210,11 +1235,24 @@ def process_split_and_strip_files(d): dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(file) + dv["append"] fpath = dvar + dest ftarget = dvar + dv["libdir"] + os.path.dirname(target) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] -bb.utils.mkdirhier(os.path.dirname(fpath)) -# Only one hardlink of separated debug info file in each directory -if not os.access(fpath, os.R_OK): -#bb.note("Link %s -> %s" % (fpath, ftarget)) -os.link(ftarget, fpath) +if os.access(ftarget, os.R_OK): +bb.utils.mkdirhier(os.path.dirname(fpath)) +# Only one hardlink of separated debug info file in each directory +if not os.access(fpath, os.R_OK): +#bb.note("Link %s -> %s" % (fpath, ftarget)) +os.link(ftarget, fpath) +elif (d.getVar('PACKAGE_DEBUG_STATIC_SPLIT') == '1'): +deststatic = dv["staticlibdir"] + os.path.dirname(src) + dv["staticdir"] + "/" + os.path.basename(file) + dv["staticappend"] +fpath = dv
[OE-core] [PATCH v2 2/4] package.py: Fix inode link target
From: Mark Hatle The process_split_and_strip_files attempts to reconstruct hardlinks through the .debug directory components. Unfortunately there was an error in the 'dest' calculation that cause the SRC and DEST to be the same when they both exist in the same path. This did not trigger an error due to the os.access(fpath) check in the original code. Fix the dest calculation by using 'file' instead of target, as this will ensure we get the correct target filename at all times. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index 8a64e13..ce599f1 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -1207,7 +1207,7 @@ def process_split_and_strip_files(d): target = inodes[ref][0][len(dvar):] for file in inodes[ref][1:]: src = file[len(dvar):] -dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] +dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(file) + dv["append"] fpath = dvar + dest ftarget = dvar + dv["libdir"] + os.path.dirname(target) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] bb.utils.mkdirhier(os.path.dirname(fpath)) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202245): https://lists.openembedded.org/g/openembedded-core/message/202245 Mute This Topic: https://lists.openembedded.org/mt/107426722/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH v2 1/4] package.py: Fix static debuginfo split
From: Mark Hatle Fix: NameError: name 'shutil' is not defined Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index e6b46a0..8a64e13 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -838,6 +838,8 @@ def splitdebuginfo(file, dvar, dv, d): return (file, sources) def splitstaticdebuginfo(file, dvar, dv, d): +import shutil + # Unlike the function above, there is no way to split a static library # two components. So to get similar results we will copy the unmodified # static library (containing the debug symbols) into a new directory. -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202243): https://lists.openembedded.org/g/openembedded-core/message/202243 Mute This Topic: https://lists.openembedded.org/mt/107426720/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] package.py: Fix static library processing
From: Mark Hatle When PACKAGE_DEBUG_STATIC_SPLIT is enabled, the copy frunction fails since shutil was not availble in this python module. Fix this with an import. When PACKAGE_STRIP_STATIC is enabled the system did not pay attention to hardlinks. This could trigger a race condition during stripping of static libraries where multiple strips (through hardlinks) could run at the same time triggering a truncated or modified file error. The hardlink breaking code is based on the existing code for elf files, but due to the nature of the symlinks needed to be done in a separate block of code. The process_split_and_strip_files attempts to reconstruct hardlinks through the .debug directory components. Unfortunately there was an error in the 'dest' calculation that cause the SRC and DEST to be the same with they box exist in the same path. This did not trigger an error due to the os.access(fpath) check in the original code. Fix the dest calculation by using 'file' instead of target, as this will ensure we get the correct target filename at all times. Add support for static-library debugfs hardlinking as well, using the changes above as a foundation for the work. Print a note to the logs if the link target can't be found. This isn't strictly an error, but may be useful for debugging an issue where a file isn't present. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oe/package.py | 60 +- 1 file changed, 50 insertions(+), 10 deletions(-) diff --git a/meta/lib/oe/package.py b/meta/lib/oe/package.py index e6b46a0..dbbda9f 100644 --- a/meta/lib/oe/package.py +++ b/meta/lib/oe/package.py @@ -838,6 +838,8 @@ def splitdebuginfo(file, dvar, dv, d): return (file, sources) def splitstaticdebuginfo(file, dvar, dv, d): +import shutil + # Unlike the function above, there is no way to split a static library # two components. So to get similar results we will copy the unmodified # static library (containing the debug symbols) into a new directory. @@ -1078,6 +1080,7 @@ def process_split_and_strip_files(d): d.getVar('INHIBIT_PACKAGE_DEBUG_SPLIT') != '1'): checkelf = {} checkelflinks = {} +checkstatic = {} for root, dirs, files in cpath.walk(dvar): for f in files: file = os.path.join(root, f) @@ -1091,10 +1094,6 @@ def process_split_and_strip_files(d): if file in skipfiles: continue -if oe.package.is_static_lib(file): -staticlibs.append(file) -continue - try: ltarget = cpath.realpath(file, dvar, False) s = cpath.lstat(ltarget) @@ -1106,6 +1105,13 @@ def process_split_and_strip_files(d): continue if not s: continue + +if oe.package.is_static_lib(file): +# Use a reference of device ID and inode number to identify files +file_reference = "%d_%d" % (s.st_dev, s.st_ino) +checkstatic[file] = (file, file_reference) +continue + # Check its an executable if (s[stat.ST_MODE] & stat.S_IXUSR) or (s[stat.ST_MODE] & stat.S_IXGRP) \ or (s[stat.ST_MODE] & stat.S_IXOTH) \ @@ -1170,6 +1176,27 @@ def process_split_and_strip_files(d): # Modified the file so clear the cache cpath.updatecache(file) +# Do the same hardlink processing as above, but for static libraries +results = list(checkstatic.keys()) + +# As above, sort the results. +results.sort(key=lambda x: x[0]) + +for file in results: +# Use a reference of device ID and inode number to identify files +file_reference = checkstatic[file][1] +if file_reference in inodes: +os.unlink(file) +os.link(inodes[file_reference][0], file) +inodes[file_reference].append(file) +else: +inodes[file_reference] = [file] +# break hardlink +bb.utils.break_hardlinks(file) +staticlibs.append(file) +# Modified the file so clear the cache +cpath.updatecache(file) + def strip_pkgd_prefix(f): nonlocal dvar @@ -1205,14 +1232,27 @@ def process_split_and_strip_files(d): target = inodes[ref][0][len(dvar):] for file in inodes[ref][1:]: src = file[len(dvar):] -dest = dv["libdir"] + os.path.dirname(src) + dv["dir"] + "/" + os.path.basename(target) + dv["append"] +dest = dv["libdir"] + os.path.dirname(src)
[OE-core] [PATCH 2/2] selftest-hardlink: Add additional test cases
From: Mark Hatle Additional test cases for debug symlink generation both binaries and static libraries. This also has the side effect of testing for race conditions in the hardlink debug generation and stripping. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- .../selftest-hardlink/selftest-hardlink.bb | 13 meta/lib/oeqa/selftest/cases/package.py| 24 ++ 2 files changed, 37 insertions(+) diff --git a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb index 5632bda..64fea8e 100644 --- a/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb +++ b/meta-selftest/recipes-test/selftest-hardlink/selftest-hardlink.bb @@ -11,6 +11,9 @@ UNPACKDIR = "${S}" do_compile () { ${CC} hello.c -o hello1 ${CFLAGS} ${LDFLAGS} + + ${CC} hello.c -c -o hello.o ${CFLAGS} + ${AR} rcs libhello.a hello.o } do_install () { @@ -23,9 +26,19 @@ do_install () { ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello3 ln ${D}${bindir}/hello1 ${D}${libexecdir}/hello4 + # We need so many hardlink copies to look for specific race conditions + install -d ${D}${libdir} + install -m 0644 libhello.a ${D}${libdir} + for num in `seq 1 100` ; do + ln ${D}${libdir}/libhello.a ${D}${libdir}/libhello-${num}.a + done + dd if=/dev/zero of=${D}${bindir}/sparsetest bs=1 count=0 seek=1M } RDEPENDS:${PN}-gdb += "gdb" PACKAGES =+ "${PN}-gdb" FILES:${PN}-gdb = "${bindir}/gdb.sh" + +PACKAGE_STRIP_STATIC = "1" +PACKAGE_DEBUG_STATIC_SPLIT = "1" diff --git a/meta/lib/oeqa/selftest/cases/package.py b/meta/lib/oeqa/selftest/cases/package.py index 1aa6c03..71231f5 100644 --- a/meta/lib/oeqa/selftest/cases/package.py +++ b/meta/lib/oeqa/selftest/cases/package.py @@ -103,11 +103,35 @@ class PackageTests(OESelftestTestCase): dest = get_bb_var('PKGDEST', 'selftest-hardlink') bindir = get_bb_var('bindir', 'selftest-hardlink') +libdir = get_bb_var('libdir', 'selftest-hardlink') +libexecdir = get_bb_var('libexecdir', 'selftest-hardlink') def checkfiles(): # Recipe creates 4 hardlinked files, there is a copy in package/ and a copy in packages-split/ # so expect 8 in total. self.assertEqual(os.stat(dest + "/selftest-hardlink" + bindir + "/hello1").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink" + libexecdir + "/hello3").st_nlink, 8) + +# Check dbg version +# 4 items, a copy in both package/packages-split so 8 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + bindir + "/.debug/hello1").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + bindir + "/.debug/hello2").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello3").st_nlink, 8) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libexecdir + "/.debug/hello4").st_nlink, 8) + +# Check the staticdev libraries +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-staticdev" + libdir + "/libhello-75.a").st_nlink, 202) + +# Check static dbg +# 101 items, a copy in both package/packages-split so 202 +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-25.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-50.a").st_nlink, 202) +self.assertEqual(os.stat(dest + "/selftest-hardlink-dbg" + libdir + "/.debug-static/libhello-75.a").st_nlink, 202) # Test a sparse file remains sparse sparsestat = os.stat(dest + "/selftest-hardlink" + bindir + "/sparsetest") -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202237): https://lists.openembedded.org/g/openembedded-core/message/202237 Mute This Topic: https://lists.openembedded.org/mt/107424312/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 0/2] Fix SPDX processing with AMD toolchain building
AMD (Xilinx) toolchain building for our 'vitis' product line seems to trigger two issues with the SPDX processing. Specifically: *) Baremetal toolchain elements may have directories listed in them, which triggers a hashing failure *) The toolchains are built of multilib packages, which triggers a failure reported as 'No SPDX file found for package ' I think the first change is pretty obvious what needs to happen if directory names can end up in the debugsrc list. (Why they end up there, I have no idea -- but this is all built from existing YP binutils, gcc, and newlib.) The second patch it was tracked down that when multilibs are activated, the system won't use the correct 'sstate_arch' to search for the files. This was resolved following a similar approach to the PACKAGE_ARCHs, but does make the assumption that 'package.bbclass' is available in all instances. Reproducing the build environment: git clone https://git.yoctoproject.org/poky -b master git clone https://github.com/Xilinx/meta-xilinx -b master-next . ./poky oe-init-build-env bitbake-layers add-layer ../meta-xilinx/meta-xilinx-core \ ../meta-xilinx/meta-microblaze ../meta-xilinx/meta-xilinx-standalone \ ../meta-xilinx/meta-vitis-tc Build for each of the toolchains (really only aarch64-tc is needed to demonstrate the issues) MACHINE= DISTRO=xilinx-standalone bitbake meta-xilinx-toolchain Where machine is one of: aarch32-tc aarch64-tc arm-rm-tc microblaze-tc riscv-tc (aarch64-tc has by far the fewest multilibs so is the fastest to parse and build.) Any questions, let me know! (Also these changes were made available to https://git.yoctoproject.org/poky-contrib mgh/spdx-fixes) Mark Hatle (2): create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc package.bbclass: Add ALL_MULTILIB_SSTATE_ARCHS meta/classes-global/package.bbclass | 1 + meta/classes/create-spdx-2.2.bbclass | 15 --- 2 files changed, 9 insertions(+), 7 deletions(-) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202067): https://lists.openembedded.org/g/openembedded-core/message/202067 Mute This Topic: https://lists.openembedded.org/mt/107239145/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/2] create-sdpx-2.2.bbclass: Switch from exists to isfile checking debugsrc
From: Mark Hatle While debugsrc is almost always a file (or link), there are apparently cases where a directory could be returned from the dwarfsrcfiles processing. When this happens, the hashing fails and an error results when building the SPDX documents. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/classes/create-spdx-2.2.bbclass | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 239a95d..13d2aa1 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -223,7 +223,8 @@ def add_package_sources_from_debug(d, package_doc, spdx_package, package, packag debugsrc_path = search / debugsrc.replace('/usr/src/kernel/', '') else: debugsrc_path = search / debugsrc.lstrip("/") -if not debugsrc_path.exists(): +# We can only hash files below, skip directories, links, etc. +if not os.path.isfile(debugsrc_path): continue file_sha256 = bb.utils.sha256_file(debugsrc_path) -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202066): https://lists.openembedded.org/g/openembedded-core/message/202066 Mute This Topic: https://lists.openembedded.org/mt/107239144/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/2] package.bbclass: Add ALL_MULTILIB_SSTATE_ARCHS
From: Mark Hatle When create-spdx-2.2 class is processing documents, it needs to find the document in a path that is related to the SSTATE_ARCH when a packge is generated. The SSTATE_ARCH can be affected by multilib configurations, resulting is something like armv8a-mlib. When the image (or SDK) is being generated and the components are collected, the system has no knowledge of the multilib arch and will fail to find it, such as: ERROR: meta-toolchain-1.0-r0 do_populate_sdk: No SPDX file found for package libilp32-libgcc-dbg, False sstate:libilp32-libgcc:armv8a-ilp32-mllibilp32-elf:14.1.0:r0:armv8a-ilp32:12: sstate:libilp32-libgcc::14.1.0:r0::12: Adding in the new ALL_MULTILIB_SSTATE_ARCHS will provide a full set of SSTATE_ARCHS including ones that contain the multilib extension which will allow create-spdx-2.2 to correctly find the document it is looking for. This would also be valuable to any other function doing a similar search through SSTATE_ARCH that may have been extended with multilib configurations. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/classes-global/package.bbclass | 1 + meta/classes/create-spdx-2.2.bbclass | 12 ++-- 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/meta/classes-global/package.bbclass b/meta/classes-global/package.bbclass index 6cd8c01..5869fe9 100644 --- a/meta/classes-global/package.bbclass +++ b/meta/classes-global/package.bbclass @@ -55,6 +55,7 @@ PKGDEST = "${WORKDIR}/packages-split" LOCALE_SECTION ?= '' ALL_MULTILIB_PACKAGE_ARCHS = "${@all_multilib_tune_values(d, 'PACKAGE_ARCHS')}" +ALL_MULTILIB_SSTATE_ARCHS = "${@all_multilib_tune_values(d, 'SSTATE_ARCHS')}" # rpm is used for the per-file dependency identification # dwarfsrcfiles is used to determine the list of debug source files diff --git a/meta/classes/create-spdx-2.2.bbclass b/meta/classes/create-spdx-2.2.bbclass index 13d2aa1..d2cbec8 100644 --- a/meta/classes/create-spdx-2.2.bbclass +++ b/meta/classes/create-spdx-2.2.bbclass @@ -259,7 +259,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): import oe.spdx deploy_dir_spdx = Path(d.getVar("DEPLOY_DIR_SPDX")) -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("ALL_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() dep_recipes = [] @@ -304,7 +304,7 @@ def collect_dep_recipes(d, doc, spdx_recipe): return dep_recipes -collect_dep_recipes[vardepsexclude] = "SSTATE_ARCHS" +collect_dep_recipes[vardepsexclude] = "ALL_MULTILIB_SSTATE_ARCHS" def collect_dep_sources(d, dep_recipes): import oe.sbom @@ -600,7 +600,7 @@ python do_create_runtime_spdx() { providers = collect_package_providers(d) pkg_arch = d.getVar("SSTATE_PKGARCH") -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("ALL_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() if not is_native: @@ -706,7 +706,7 @@ python do_create_runtime_spdx() { oe.sbom.write_doc(d, runtime_doc, pkg_arch, "runtime", spdx_deploy, indent=get_json_indent(d)) } -do_create_runtime_spdx[vardepsexclude] += "OVERRIDES SSTATE_ARCHS" +do_create_runtime_spdx[vardepsexclude] += "OVERRIDES ALL_MULTILIB_SSTATE_ARCHS" addtask do_create_runtime_spdx after do_create_spdx before do_build do_rm_work SSTATETASKS += "do_create_runtime_spdx" @@ -787,7 +787,7 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx import bb.compress.zstd providers = collect_package_providers(d) -package_archs = d.getVar("SSTATE_ARCHS").split() +package_archs = d.getVar("ALL_MULTILIB_SSTATE_ARCHS").split() package_archs.reverse() creation_time = datetime.now(tz=timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") @@ -939,4 +939,4 @@ def combine_spdx(d, rootfs_name, rootfs_deploydir, rootfs_spdxid, packages, spdx tar.addfile(info, fileobj=index_str) -combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS SSTATE_ARCHS" +combine_spdx[vardepsexclude] += "BB_NUMBER_THREADS ALL_MULTILIB_SSTATE_ARCHS" -- 1.8.3.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#202068): https://lists.openembedded.org/g/openembedded-core/message/202068 Mute This Topic: https://lists.openembedded.org/mt/107239146/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 09/14] kernel-fitimage: refactor do_assemble_fitimage_initramfs
On 7/4/24 2:09 AM, Adrian Freihofer wrote: From: Adrian Freihofer Make the fitimage_assemble function usable with absolute paths for the generated its and the fitImage file. Later on this will allow to take the linux.bin and the DTB files from the sstated deploy derectory and write the generated files to a separate folder with independent sstate-cache handling. Add 3 comments which are helpful for the next commit. Signed-off-by: Adrian Freihofer --- meta/classes-recipe/kernel-fitimage.bbclass | 49 ++--- 1 file changed, 34 insertions(+), 15 deletions(-) diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass index fbeb20596ea..e084dc57573 100644 --- a/meta/classes-recipe/kernel-fitimage.bbclass +++ b/meta/classes-recipe/kernel-fitimage.bbclass @@ -174,8 +174,9 @@ fitimage_emit_section_kernel() { ENTRYPOINT="${UBOOT_ENTRYPOINT}" if [ -n "${UBOOT_ENTRYSYMBOL}" ]; then + kernel_base=$(basename $3) ENTRYPOINT=`${HOST_PREFIX}nm vmlinux | \ - awk '$3=="${UBOOT_ENTRYSYMBOL}" {print "0x"$1;exit}'` + awk '$kernel_base=="${UBOOT_ENTRYSYMBOL}" {print "0x"$1;exit}'` fi cat << EOF >> $1 @@ -572,7 +573,7 @@ fitimage_assemble() { setupcount="" bootscr_id="" default_dtb_image="" - rm -f $1 arch/${ARCH}/boot/$2 + rm -f "$1" "arch/${ARCH}/boot/$(basename $2)" if [ -n "${UBOOT_SIGN_IMG_KEYNAME}" -a "${UBOOT_SIGN_KEYNAME}" = "${UBOOT_SIGN_IMG_KEYNAME}" ]; then bbfatal "Keys used to sign images and configuration nodes must be different." @@ -586,7 +587,7 @@ fitimage_assemble() { fitimage_emit_section_maint $1 imagestart uboot_prep_kimage - fitimage_emit_section_kernel $1 $kernelcount linux.bin "$linux_comp" + fitimage_emit_section_kernel $1 $kernelcount "$(readlink -f linux.bin)" "$linux_comp" # # Step 2: Prepare a DTB image section @@ -610,9 +611,20 @@ fitimage_assemble() { DTB_PATH="${KERNEL_OUTPUT_DIR}/$DTB" fi - # Strip off the path component from the filename + # Strip off the path component from the filename if "${@'false' if oe.types.boolean(d.getVar('KERNEL_DTBVENDORED')) else 'true'}"; then - DTB=`basename $DTB` + DTB=`basename $DTB` + fi + I'm purely reading the comment here (below). It talks about using the 'deploy' folder. I'm not sure what current behavior is, but with our (amd fpga) code, we've tried to completely move away from using ANYTHING out of the deploy directory. We found it horrible fragile [during the build] because the do_deploy can be delayed or not executed in some cases. (To be clear that isn't a bug!) Instead everything we've done has moved to using the workdir/recipe-sysroot to access files and components. + # Find DTBs without sub-folders when running in deploy folder + if [ ! -e "$DTB_PATH" ]; then + DTB=$(basename $DTB) + DTB_PATH=$(readlink -f $DTB) + fi + + # Fail as early as possible if there is still no DTB file found + if [ ! -e "$DTB_PATH" ]; then + bberror "Cannot find the DTB file at $DTB_PATH" fi # Set the default dtb image if it exists in the devicetree. @@ -665,9 +677,8 @@ fitimage_assemble() { if [ -n "${UBOOT_ENV}" ] && [ -d "${STAGING_DIR_HOST}/boot" ]; then if [ -e "${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY}" ]; then - cp ${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY} ${B} bootscr_id="${UBOOT_ENV_BINARY}" - fitimage_emit_section_boot_script $1 "$bootscr_id" ${UBOOT_ENV_BINARY} + fitimage_emit_section_boot_script $1 "$bootscr_id" "${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY}" else bbwarn "${STAGING_DIR_HOST}/boot/${UBOOT_ENV_BINARY} not found." fi @@ -676,9 +687,14 @@ fitimage_assemble() { # # Step 4: Prepare a setup section. (For x86) # + # Run from kernel build folder (bundled mode) if [ -e ${KERNEL_OUTPUT_DIR}/setup.bin ]; then setupcount=1 fitimage_emit_section_setup $1 $setupcount ${KERNEL_OUTPUT_DIR}/setup.bin Same comment here about the 'deploy' vs recipe-sysroot. It looks to me like it's doing all of this in the CWD, but I'm unclear what it has been set to. So this could be an issue? I think that's why it was KERNEL_OUTPUT_DIR before. + # Run from deploy folder (unbundled mode) + elif [ -e setu
Re: [OE-core] [PATCH 07/14] kernel-fitimage: fix external dtb check
On 7/4/24 2:09 AM, Adrian Freihofer wrote: From: Adrian Freihofer If EXTERNAL_KERNEL_DEVICETREE and dtb_image_sect are empty variables dtb_path ends up as "/" which is available on most Unix systems but probably not the dtb_path which is needed here. Checking for a file makes more sense and also solves the issue with the "/". Signed-off-by: Adrian Freihofer --- meta/classes-recipe/kernel-fitimage.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes-recipe/kernel-fitimage.bbclass b/meta/classes-recipe/kernel-fitimage.bbclass index 6e6898e5f6f..664ebc65d7c 100644 --- a/meta/classes-recipe/kernel-fitimage.bbclass +++ b/meta/classes-recipe/kernel-fitimage.bbclass @@ -429,7 +429,7 @@ fitimage_emit_section_config() { fi dtb_path="${EXTERNAL_KERNEL_DEVICETREE}/${dtb_image_sect}" - if [ -e "$dtb_path" ]; then + if [ -f "$dtb_path" ]; then compat=$(fdtget -t s "$dtb_path" / compatible | sed 's/ /", "/g') if [ -n "$compat" ]; then compatible_line="compatible = \"$compat\";" I can understand avoiding directories with this, but I've definitely seen cases where the dtb_path is pointing to a symbolic link. Would [ -f "$dtb_path" -o -L "$dtb_path" ]; (or similar) solve this? --Mark -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#201681): https://lists.openembedded.org/g/openembedded-core/message/201681 Mute This Topic: https://lists.openembedded.org/mt/107033892/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [yocto-security] The future of meta/conf/distro/include/cve-extra-exclusions.inc
On 6/18/24 9:21 AM, Andrew Murray wrote: Hi Marta, On Tue, 18 Jun 2024 at 14:55, Marta Rybczynska via lists.yoctoproject.org wrote: Hello all, During the cve-check work, the cve-extra-exclusion.inc file is causing difficulties. It is a global place for all CVEs, without linking them to the recipe or package affected. When we do not generate the CVE data at build, we have no information which of those ignores apply to each recipe - and we get the whole list for every single package... What about suggesting everyone to put CVE_STATUS by default in the affected recipe, and not in that file? I guess the reason cve-extra-exclusions.inc exists in the first place is indicated by this comment in the file: "# The file is not included by default since users should review this data to ensure # it matches their expectations and usage of the project." And the original commit: "The preferred methods for CVE resolution are: 1. Version upgrades where possible 2. Patches where not possible 3. Database updates where version info is incorrect 4. Exclusion from checking where it is determined that the CVE does not apply to our environment In some cases none of these methods are possible. For example the CVE may be decades old with no apparent resolution, and with broken links that make further research impractical. Some CVEs are vauge with no specific action the project can take too. This patch creates a mechanism for users to remove this type of CVE from the cve-check results via an optional include file. Based on an initial patch from Steve Sakoman but extended heavily by RP. " I interpret this as: we've decided to ignore some CVEs because they don't seem possible to patch and aren't relevant, but we recommend that our users review this because these may be contentious. I think we should move any non-contentious CVEs ignores to the relevant packages, forget about the remainder of the CVEs in the file, and then retire use of the file. I agree with this. The CVEs really need to be handled within the context of the recipe(s) themselves now. We've moved from 'project wide CVE info' to 'component specific CVE info', and the component level really is an individual recipe namespace. --Mark People that build products based on Yocto that care about vulnerabilities will likely have a means to monitor vulnerabilities going forward (e.g. Yocto's own cve-check combined with CVE_STATUS - or a third party), as part of that process they'll have a means to ignore CVEs that they don't care about. If there is any question of whether a CVE should be ignored then let's leave that to the user. They'll have to go through a list of unresolved CVEs anyway, so the few in cve-extra-exclusions doesn't seem like a problem to me. If we're too keen on ignoring CVEs (and ignore something an end user may wish to not ignore), then you'll make more work for the end user as they'll want to vet the CVE_STATUS ignores already made by the Yocto project to gain confidence in the project's efforts. Thanks, Andrew Murray -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#200873): https://lists.openembedded.org/g/openembedded-core/message/200873 Mute This Topic: https://lists.openembedded.org/mt/106741834/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] RFC: CVE check standalone (introducing yocto-vex-check)
I reviewed the RFC. It looks good to me. I had one minor comment on the 2/3 of patch. Reviewed-by: Mark Hatle On 5/31/24 9:06 AM, Marta Rybczynska wrote: Hello, We have been working on the standalone version of the cve-check that can run over an SBOM (SPDX+VEX-like information) since April. This is the moment for the RFC. This way we can verify use cases, discuss what is needed to add and do testing early! (and the curious people can start running it too) The tooling consists of a patchset over OE-core: https://lists.openembedded.org/g/openembedded-core/message/200074 <https://lists.openembedded.org/g/openembedded-core/message/200074> https://lists.openembedded.org/g/openembedded-core/message/200075 <https://lists.openembedded.org/g/openembedded-core/message/200075> and a modification heeded for the CVE backend (the Kernel CNA uses "linux" and not "linux_kernel") https://lists.openembedded.org/g/openembedded-core/message/200076 <https://lists.openembedded.org/g/openembedded-core/message/200076> The standalone tool RFC is here: https://gitlab.com/syslinbit/public/yocto-vex-check <https://gitlab.com/syslinbit/public/yocto-vex-check> This repo will migrate to the YP infra, likely in a form of multiple ones (we also have autobuilder changes, and will likely split binary data to a separate repository). The yocto-vex-check supports two databases today: NVD and CVE. How to use the tool (from the README) This tool allows to check SBOMs for vulnerabilities. You can take SBOM/VEX of a Yocto Project build and generate a fresh vulnerability status. It uses the SPDX archive as the list of packages to check, with the additional information from the VEX class. WARNING: this tool is under active development, file formats and options do change regularly. Functionalities: - reads YP SPDX 2.2 files and YP cve-check JSON files - converts SPDX/cve-check JSON to VEX (close to OpenVEX) - performs a check for vulnerabilities taking the VEX and using the specified database - uses output of the "vex" class of OpenEmbedded-core How to use: 1. Download the database to use: - for NVD, use cve-update-nvd2-native.py - for the CVE database, get the CVEv5 git repository: either the upsteam one at https://github.com/CVEProject/cvelistV5 <https://github.com/CVEProject/cvelistV5> or the one with OE-related fixes at https://github.com/mrybczyn/cvelistV5-overrides <https://github.com/mrybczyn/cvelistV5-overrides> (recommended) 2. Enable the vex class in your YP-based distribution build by adding to your local.conf: INHERIT += "vex" 3. Build your image as usual and note the location of the CVE JSON file (in the output of the build) 4. (Recommended for production builds) archive the CVE JSON file and the SPDX archive 5. Use the tool as described below at any moment to generate a fresh vulnerability status 6. (Optional) You can convert the output to text or CSV format using scripts/cve-report.py Descrition of tools: The high-level wrapper performs the whole process of a given database. The usage is as follows: ./wrap-yocto-vex-check.py -i -o -t -b -db -db-type where: -i is the CVE JSON file as given in the output of the build -o is the output directory for the updated CVE file -t is a temporary directory used during the scan -b is the build directory that contains the SPDX archive -db is the path to the database file (NVD) or downloaded git repository (CVE) -db-type is the database type: NVD or CVE Lower-level tools: - cve-update-nvd2-native.py - allows to download the NVD database - yocto-vex-check.py - step-by-step conversions; imports SPDX archive, allows to convert SPDX+CVE JSON to VEX, runs the vulnerability check on the given database Limitations: - today the SPDX generation in YP works for images and SDK, not for the world build - the MITRE (CVE) database contains a number of difficult-to-parse entries; especially before 2024. We know there are some bugs (one related to the linux_kernel), the fix is in progress. Questions/comments/use-cases welcome! Our questions for further steps: 1. We started with the assumption of SPDX as the package list to scan and the CVE JSON data as the supplementary data needed for the scan. However, SPDX archive isn't generated for world builds. We can 1) add the only remaining information to the VEX (CVE-JSON) which is the CPE list and allow to use the VEX file as source and 2) add the remaining archives to SPDX. At the end we will likely want both, also because SPDX2.2 requires the complete build, and that is long in case of world. 2. The tool currently supports either one backend (CVE, NVD), or the other. Results are different. We can merge results by post-processing or add a list of databases to use. In both cases, there is a difficulty to decide on priority of conflicting scan results. 3
Re: [OE-core] [RFC][OE-core 2/3] vex.bbclass: add a new class
On 5/31/24 8:01 AM, Marta Rybczynska wrote: The "vex" class generates the minimum information that is necessary by an external CVE checking tool. It is a drop-in replacement of "cve-check". It uses the same variables from recipes. It generates the JSON output format only. Signed-off-by: Marta Rybczynska Signed-off-by: Samantha Jalabert --- meta/classes/vex.bbclass | 332 +++ 1 file changed, 332 insertions(+) create mode 100644 meta/classes/vex.bbclass diff --git a/meta/classes/vex.bbclass b/meta/classes/vex.bbclass new file mode 100644 index 00..e196f73169 --- /dev/null +++ b/meta/classes/vex.bbclass ... + +CVE_CHECK_SUMMARY_DIR ?= "${LOG_DIR}/cve" +CVE_CHECK_SUMMARY_FILE_NAME ?= "cve-summary" +CVE_CHECK_SUMMARY_FILE_NAME_JSON = "cve-summary.json" Should the above be: CVE_CHECK_SUMMARY_FILE_NAME_JSON ?= "cve-summary.json" or CVE_CHECK_SUMMARY_FILE_NAME_JSON = "${CVE_CHECK_SUMMARY_FILE_NAME}.json" +CVE_CHECK_SUMMARY_INDEX_PATH = "${CVE_CHECK_SUMMARY_DIR}/cve-summary-index.txt" Any reson for the above to be ?= or ??= ? + +CVE_CHECK_LOG_JSON ?= "${T}/cve.json" + +CVE_CHECK_DIR ??= "${DEPLOY_DIR}/cve" +CVE_CHECK_RECIPE_FILE_JSON ?= "${CVE_CHECK_DIR}/${PN}_cve.json" +CVE_CHECK_MANIFEST_JSON ?= "${IMGDEPLOYDIR}/${IMAGE_NAME}.json" +CVE_CHECK_COPY_FILES ??= "1" +CVE_CHECK_CREATE_MANIFEST ??= "1" + +# Report Patched or Ignored CVEs +CVE_CHECK_REPORT_PATCHED ??= "1" + +CVE_CHECK_SHOW_WARNINGS ??= "1" + +# Skip CVE Check for packages (PN) +CVE_CHECK_SKIP_RECIPE ?= "" + +# Replace NVD DB check status for a given CVE. Each of CVE has to be mentioned +# separately with optional detail and description for this status. +# +# CVE_STATUS[CVE-1234-0001] = "not-applicable-platform: Issue only applies on Windows" +# CVE_STATUS[CVE-1234-0002] = "fixed-version: Fixed externally" +# +# Settings the same status and reason for multiple CVEs is possible +# via CVE_STATUS_GROUPS variable. +# +# CVE_STATUS_GROUPS = "CVE_STATUS_WIN CVE_STATUS_PATCHED" +# +# CVE_STATUS_WIN = "CVE-1234-0001 CVE-1234-0003" +# CVE_STATUS_WIN[status] = "not-applicable-platform: Issue only applies on Windows" +# CVE_STATUS_PATCHED = "CVE-1234-0002 CVE-1234-0004" +# CVE_STATUS_PATCHED[status] = "fixed-version: Fixed externally" +# +# All possible CVE statuses could be found in cve-check-map.conf +# CVE_CHECK_STATUSMAP[not-applicable-platform] = "Ignored" +# CVE_CHECK_STATUSMAP[fixed-version] = "Patched" +# +# CVE_CHECK_IGNORE is deprecated and CVE_STATUS has to be used instead. +# Keep CVE_CHECK_IGNORE until other layers migrate to new variables +CVE_CHECK_IGNORE ?= "" + +# Layers to be excluded +CVE_CHECK_LAYER_EXCLUDELIST ??= "" + +# Layers to be included +CVE_CHECK_LAYER_INCLUDELIST ??= "" + + +# set to "alphabetical" for version using single alphabetical character as increment release +CVE_VERSION_SUFFIX ??= "" + ... -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#200278): https://lists.openembedded.org/g/openembedded-core/message/200278 Mute This Topic: https://lists.openembedded.org/mt/106407311/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [scarthgap 2/2] gcc: Fix for CVE-2024-0151
I just realized that somehow I entered the wrong CVE in the body of the patch itself. This IS CVE-2024-0151, but somehow I entered CVE-2023-4039 is the patch body. Steve can you fix this as part of the merge, or do you want me to send a V2 or a follow on to fix this? --Mark On 5/24/24 3:12 PM, Mark Hatle via lists.openembedded.org wrote: Fix for insufficient argument checking in Secure state Entry functions in software using Cortex-M Security Extensions (CMSE), that has been compiled using toolchains that implement 'Arm v8-M Security Extensions Requirements on Development Tools' prior to version 1.4, allows an attacker to pass values to Secure state that are out of range for types smaller than 32-bits. Out of range values might lead to incorrect operations in secure state. Signed-off-by: Mark Hatle --- meta/recipes-devtools/gcc/gcc-13.2.inc| 1 + .../gcc/gcc/CVE-2024-0151.patch | 315 ++ 2 files changed, 316 insertions(+) create mode 100644 meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch diff --git a/meta/recipes-devtools/gcc/gcc-13.2.inc b/meta/recipes-devtools/gcc/gcc-13.2.inc index 603377a49a..abf177822b 100644 --- a/meta/recipes-devtools/gcc/gcc-13.2.inc +++ b/meta/recipes-devtools/gcc/gcc-13.2.inc @@ -68,6 +68,7 @@ SRC_URI = "${BASEURI} \ file://CVE-2023-4039.patch \ file://0026-aarch64-Fix-loose-ldpstp-check-PR111411.patch \ file://0027-Fix-gcc-vect-module-testcases.patch \ + file://CVE-2024-0151.patch \ " SRC_URI[sha256sum] = "e275e76442a6067341a27f04c5c6b83d8613144004c0413528863dc6b5c743da" diff --git a/meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch b/meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch new file mode 100644 index 00..01d55b5cdb --- /dev/null +++ b/meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch @@ -0,0 +1,315 @@ +arm: Zero/Sign extends for CMSE security + +This patch makes the following changes: + +1) When calling a secure function from non-secure code then any arguments + smaller than 32-bits that are passed in registers are zero- or sign-extended. +2) After a non-secure function returns into secure code then any return value + smaller than 32-bits that is passed in a register is zero- or sign-extended. + +This patch addresses the following CVE-2024-0151. + +gcc/ChangeLog: +PR target/114837 +* config/arm/arm.cc (cmse_nonsecure_call_inline_register_clear): + Add zero/sign extend. +(arm_expand_prologue): Add zero/sign extend. + +gcc/testsuite/ChangeLog: + +* gcc.target/arm/cmse/extend-param.c: New test. +* gcc.target/arm/cmse/extend-return.c: New test. + +CVE: CVE-2023-4039 +Upstream-Status: Backport [https://gcc.gnu.org/pipermail/gcc-patches/2024-April/649973.html] +Signed-off-by: Mark Hatle + +diff --git a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc +index 0217abc218d60956ce727e6d008d46b9176dddc5..ea0c963a4d67ecd70e1571624e84dfe46d757df9 100644 +--- a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc +@@ -19210,6 +19210,30 @@ cmse_nonsecure_call_inline_register_clear (void) + end_sequence (); + emit_insn_before (seq, insn); + ++/* The AAPCS requires the callee to widen integral types narrower ++ than 32 bits to the full width of the register; but when handling ++ calls to non-secure space, we cannot trust the callee to have ++ correctly done so. So forcibly re-widen the result here. */ ++tree ret_type = TREE_TYPE (fntype); ++if ((TREE_CODE (ret_type) == INTEGER_TYPE ++|| TREE_CODE (ret_type) == ENUMERAL_TYPE ++|| TREE_CODE (ret_type) == BOOLEAN_TYPE) ++&& known_lt (GET_MODE_SIZE (TYPE_MODE (ret_type)), 4)) ++ { ++machine_mode ret_mode = TYPE_MODE (ret_type); ++rtx extend; ++if (TYPE_UNSIGNED (ret_type)) ++ extend = gen_rtx_ZERO_EXTEND (SImode, ++gen_rtx_REG (ret_mode, R0_REGNUM)); ++else ++ extend = gen_rtx_SIGN_EXTEND (SImode, ++gen_rtx_REG (ret_mode, R0_REGNUM)); ++emit_insn_after (gen_rtx_SET (gen_rtx_REG (SImode, R0_REGNUM), ++ extend), insn); ++ ++ } ++ ++ + if (TARGET_HAVE_FPCXT_CMSE) + { + rtx_insn *last, *pop_insn, *after = insn; +@@ -23652,6 +23676,51 @@ arm_expand_prologue (void) + + ip_rtx = gen_rtx_REG (SImode, IP_REGNUM); + ++ /* The AAPCS requires the callee to widen integral types narrower ++ than 32 bits to the full width of the register; but when handling ++ calls to non-secure space, we cannot trust the callee to have ++ correctly done so. So forcibly re-widen the result here. */ ++ if (IS_CMSE_ENTRY (func_type)) ++{ ++ function_args_itera
[OE-core] [master][scarthgap][PATCH v2 ] binutils: Fix aarch64 disassembly abort
From: Mark Hatle Code backported from binutils development tree. aarch64: Remove asserts from operand qualifier decoders [PR31595] Given that the disassembler should never abort when decoding (potentially random) data, assertion statements in the `get_*reg_qualifier_from_value' function family prove problematic. ... Signed-off-by: Mark Hatle --- meta/recipes-devtools/binutils/binutils-2.42.inc | 1 + ...ove-asserts-from-operand-qualifier-decode.patch | 382 + 2 files changed, 383 insertions(+) create mode 100644 meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch diff --git a/meta/recipes-devtools/binutils/binutils-2.42.inc b/meta/recipes-devtools/binutils/binutils-2.42.inc index 3b6f47d..d2f4956 100644 --- a/meta/recipes-devtools/binutils/binutils-2.42.inc +++ b/meta/recipes-devtools/binutils/binutils-2.42.inc @@ -36,5 +36,6 @@ SRC_URI = "\ file://0013-Define-alignof-using-_Alignof-when-using-C11-or-newe.patch \ file://0014-Remove-duplicate-pe-dll.o-entry-deom-targ_extra_ofil.patch \ file://0015-gprofng-change-use-of-bignum-to-bigint.patch \ + file://0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch \ " S = "${WORKDIR}/git" diff --git a/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch new file mode 100644 index 000..7b52425 --- /dev/null +++ b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch @@ -0,0 +1,382 @@ +From 5b1c70bfe0d8f84dc28237d6150b7b9d57c791a8 Mon Sep 17 00:00:00 2001 +From: Victor Do Nascimento +Date: Tue, 16 Apr 2024 11:49:15 +0100 +Subject: [PATCH] aarch64: Remove asserts from operand qualifier decoders + [PR31595] + +Given that the disassembler should never abort when decoding +(potentially random) data, assertion statements in the +`get_*reg_qualifier_from_value' function family prove problematic. + +Consider the random 32-bit word W, encoded in a data segment and +encountered on execution of `objdump -D '. + +If: + + (W & ~opcode_mask) == valid instruction + +Then before `print_insn_aarch64_word' has a chance to report the +instruction as potentially undefined, an attempt will be made to have +the qualifiers for the instruction's register operands (if any) +decoded. If the relevant bits do not map onto a valid qualifier for +the matched instruction-like word, an abort will be triggered and the +execution of objdump aborted. + +As this scenario is perfectly feasible and, in light of the fact that +objdump must successfully decode all sections of a given object file, +it is not appropriate to assert in this family of functions. + +Therefore, we add a new pseudo-qualifier `AARCH64_OPND_QLF_ERR' for +handling invalid qualifier-associated values and re-purpose the +assertion conditions in qualifier-retrieving functions to be the +predicate guarding the returning of the calculated qualifier type. +If the predicate fails, we return this new qualifier and allow the +caller to handle the error as appropriate. + +As these functions are called either from within +`aarch64_extract_operand' or `do_special_decoding', both of which are +expected to return non-zero values, it suffices that callers return +zero upon encountering `AARCH64_OPND_QLF_ERR'. + +Ar present the error presented in the hypothetical scenario has been +encountered in `get_sreg_qualifier_from_value', but the change is made +to the whole family to keep the interface consistent. + +Bug: https://sourceware.org/PR31595 + +Upstream-Status: Backport [commit 2601b201e95ea0edab89342ee7137c74e88a8a79] + +Signed-off-by: Mark Hatle +--- + .../testsuite/binutils-all/aarch64/illegal.d | 1 + + .../testsuite/binutils-all/aarch64/illegal.s | 3 + + include/opcode/aarch64.h | 3 + + opcodes/aarch64-dis.c | 98 +++ + 4 files changed, 87 insertions(+), 18 deletions(-) + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +index 4b90a1d9f39..b69318aec85 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +@@ -8,5 +8,6 @@ Disassembly of section \.text: + + 0+000 <.*>: + [ ]+0:[ ]+68ea18cc[ ]+.inst[]+0x68ea18cc ; undefined ++[ ]+4:[ ]+9dc39839[ ]+.inst[]+0x9dc39839 ; undefined + #pass + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.s b/binutils/testsuite/binutils-all/aarch64/illegal.s +index 216cbe6f265..43668c6db55 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.s b/binutils/testsuite/binutils-all/aarch64/illegal.s +@@ -4,4 +4,7 @@ + # ldpsw x
[OE-core] [master][scarthgap][] binutils: Fix aarch64 disassembly abort
From: "Mark Hatle via lists.openembedded.org" Code backported from binutils development tree. aarch64: Remove asserts from operand qualifier decoders [PR31595] Given that the disassembler should never abort when decoding (potentially random) data, assertion statements in the `get_*reg_qualifier_from_value' function family prove problematic. ... Signed-off-by: Mark Hatle --- meta/recipes-devtools/binutils/binutils-2.42.inc | 1 + ...ove-asserts-from-operand-qualifier-decode.patch | 382 + 2 files changed, 383 insertions(+) create mode 100644 meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch diff --git a/meta/recipes-devtools/binutils/binutils-2.42.inc b/meta/recipes-devtools/binutils/binutils-2.42.inc index 3b6f47d..d2f4956 100644 --- a/meta/recipes-devtools/binutils/binutils-2.42.inc +++ b/meta/recipes-devtools/binutils/binutils-2.42.inc @@ -36,5 +36,6 @@ SRC_URI = "\ file://0013-Define-alignof-using-_Alignof-when-using-C11-or-newe.patch \ file://0014-Remove-duplicate-pe-dll.o-entry-deom-targ_extra_ofil.patch \ file://0015-gprofng-change-use-of-bignum-to-bigint.patch \ + file://0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch \ " S = "${WORKDIR}/git" diff --git a/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch new file mode 100644 index 000..7b52425 --- /dev/null +++ b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch @@ -0,0 +1,382 @@ +From 5b1c70bfe0d8f84dc28237d6150b7b9d57c791a8 Mon Sep 17 00:00:00 2001 +From: Victor Do Nascimento +Date: Tue, 16 Apr 2024 11:49:15 +0100 +Subject: [PATCH] aarch64: Remove asserts from operand qualifier decoders + [PR31595] + +Given that the disassembler should never abort when decoding +(potentially random) data, assertion statements in the +`get_*reg_qualifier_from_value' function family prove problematic. + +Consider the random 32-bit word W, encoded in a data segment and +encountered on execution of `objdump -D '. + +If: + + (W & ~opcode_mask) == valid instruction + +Then before `print_insn_aarch64_word' has a chance to report the +instruction as potentially undefined, an attempt will be made to have +the qualifiers for the instruction's register operands (if any) +decoded. If the relevant bits do not map onto a valid qualifier for +the matched instruction-like word, an abort will be triggered and the +execution of objdump aborted. + +As this scenario is perfectly feasible and, in light of the fact that +objdump must successfully decode all sections of a given object file, +it is not appropriate to assert in this family of functions. + +Therefore, we add a new pseudo-qualifier `AARCH64_OPND_QLF_ERR' for +handling invalid qualifier-associated values and re-purpose the +assertion conditions in qualifier-retrieving functions to be the +predicate guarding the returning of the calculated qualifier type. +If the predicate fails, we return this new qualifier and allow the +caller to handle the error as appropriate. + +As these functions are called either from within +`aarch64_extract_operand' or `do_special_decoding', both of which are +expected to return non-zero values, it suffices that callers return +zero upon encountering `AARCH64_OPND_QLF_ERR'. + +Ar present the error presented in the hypothetical scenario has been +encountered in `get_sreg_qualifier_from_value', but the change is made +to the whole family to keep the interface consistent. + +Bug: https://sourceware.org/PR31595 + +Upstream-Status: Backport [commit 2601b201e95ea0edab89342ee7137c74e88a8a79] + +Signed-off-by: Mark Hatle +--- + .../testsuite/binutils-all/aarch64/illegal.d | 1 + + .../testsuite/binutils-all/aarch64/illegal.s | 3 + + include/opcode/aarch64.h | 3 + + opcodes/aarch64-dis.c | 98 +++ + 4 files changed, 87 insertions(+), 18 deletions(-) + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +index 4b90a1d9f39..b69318aec85 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +@@ -8,5 +8,6 @@ Disassembly of section \.text: + + 0+000 <.*>: + [ ]+0:[ ]+68ea18cc[ ]+.inst[]+0x68ea18cc ; undefined ++[ ]+4:[ ]+9dc39839[ ]+.inst[]+0x9dc39839 ; undefined + #pass + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.s b/binutils/testsuite/binutils-all/aarch64/illegal.s +index 216cbe6f265..43668c6db55 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.s b/binutils/testsuite/binutils-all/aarch64/illega
Re: [OE-core] [scarthgap 1/2] binutils: Fix aarch64 disassembly abort
On 5/27/24 8:50 AM, Steve Sakoman wrote: This patch will need to be submitted and accepted for master before I can take it for scarthgap. I didn't submit because I thought there was a new binutils version in master-next. I guess I'm wrong. The patch should apply the same in master. --Mark Thanks, Steve On Fri, May 24, 2024 at 1:12 PM Mark Hatle via lists.openembedded.org wrote: Code backported from binutils development tree. aarch64: Remove asserts from operand qualifier decoders [PR31595] Given that the disassembler should never abort when decoding (potentially random) data, assertion statements in the `get_*reg_qualifier_from_value' function family prove problematic. ... Signed-off-by: Mark Hatle --- .../binutils/binutils-2.42.inc| 1 + ...sserts-from-operand-qualifier-decode.patch | 382 ++ 2 files changed, 383 insertions(+) create mode 100644 meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch diff --git a/meta/recipes-devtools/binutils/binutils-2.42.inc b/meta/recipes-devtools/binutils/binutils-2.42.inc index 3b6f47d4ce..d2f49560f3 100644 --- a/meta/recipes-devtools/binutils/binutils-2.42.inc +++ b/meta/recipes-devtools/binutils/binutils-2.42.inc @@ -36,5 +36,6 @@ SRC_URI = "\ file://0013-Define-alignof-using-_Alignof-when-using-C11-or-newe.patch \ file://0014-Remove-duplicate-pe-dll.o-entry-deom-targ_extra_ofil.patch \ file://0015-gprofng-change-use-of-bignum-to-bigint.patch \ + file://0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch \ " S = "${WORKDIR}/git" diff --git a/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch new file mode 100644 index 00..7b52425a38 --- /dev/null +++ b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch @@ -0,0 +1,382 @@ +From 5b1c70bfe0d8f84dc28237d6150b7b9d57c791a8 Mon Sep 17 00:00:00 2001 +From: Victor Do Nascimento +Date: Tue, 16 Apr 2024 11:49:15 +0100 +Subject: [PATCH] aarch64: Remove asserts from operand qualifier decoders + [PR31595] + +Given that the disassembler should never abort when decoding +(potentially random) data, assertion statements in the +`get_*reg_qualifier_from_value' function family prove problematic. + +Consider the random 32-bit word W, encoded in a data segment and +encountered on execution of `objdump -D '. + +If: + + (W & ~opcode_mask) == valid instruction + +Then before `print_insn_aarch64_word' has a chance to report the +instruction as potentially undefined, an attempt will be made to have +the qualifiers for the instruction's register operands (if any) +decoded. If the relevant bits do not map onto a valid qualifier for +the matched instruction-like word, an abort will be triggered and the +execution of objdump aborted. + +As this scenario is perfectly feasible and, in light of the fact that +objdump must successfully decode all sections of a given object file, +it is not appropriate to assert in this family of functions. + +Therefore, we add a new pseudo-qualifier `AARCH64_OPND_QLF_ERR' for +handling invalid qualifier-associated values and re-purpose the +assertion conditions in qualifier-retrieving functions to be the +predicate guarding the returning of the calculated qualifier type. +If the predicate fails, we return this new qualifier and allow the +caller to handle the error as appropriate. + +As these functions are called either from within +`aarch64_extract_operand' or `do_special_decoding', both of which are +expected to return non-zero values, it suffices that callers return +zero upon encountering `AARCH64_OPND_QLF_ERR'. + +Ar present the error presented in the hypothetical scenario has been +encountered in `get_sreg_qualifier_from_value', but the change is made +to the whole family to keep the interface consistent. + +Bug: https://sourceware.org/PR31595 + +Upstream-Status: Backport [commit 2601b201e95ea0edab89342ee7137c74e88a8a79] + +Signed-off-by: Mark Hatle +--- + .../testsuite/binutils-all/aarch64/illegal.d | 1 + + .../testsuite/binutils-all/aarch64/illegal.s | 3 + + include/opcode/aarch64.h | 3 + + opcodes/aarch64-dis.c | 98 +++ + 4 files changed, 87 insertions(+), 18 deletions(-) + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +index 4b90a1d9f39..b69318aec85 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +@@ -8,5 +8,6 @@ Disassembly of section \.text: + + 0+000 <.*>: + [ ]+0:[ ]+68ea18cc[ ]+.inst[]+0x68e
[OE-core] [scarthgap 1/2] binutils: Fix aarch64 disassembly abort
Code backported from binutils development tree. aarch64: Remove asserts from operand qualifier decoders [PR31595] Given that the disassembler should never abort when decoding (potentially random) data, assertion statements in the `get_*reg_qualifier_from_value' function family prove problematic. ... Signed-off-by: Mark Hatle --- .../binutils/binutils-2.42.inc| 1 + ...sserts-from-operand-qualifier-decode.patch | 382 ++ 2 files changed, 383 insertions(+) create mode 100644 meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch diff --git a/meta/recipes-devtools/binutils/binutils-2.42.inc b/meta/recipes-devtools/binutils/binutils-2.42.inc index 3b6f47d4ce..d2f49560f3 100644 --- a/meta/recipes-devtools/binutils/binutils-2.42.inc +++ b/meta/recipes-devtools/binutils/binutils-2.42.inc @@ -36,5 +36,6 @@ SRC_URI = "\ file://0013-Define-alignof-using-_Alignof-when-using-C11-or-newe.patch \ file://0014-Remove-duplicate-pe-dll.o-entry-deom-targ_extra_ofil.patch \ file://0015-gprofng-change-use-of-bignum-to-bigint.patch \ + file://0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch \ " S = "${WORKDIR}/git" diff --git a/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch new file mode 100644 index 00..7b52425a38 --- /dev/null +++ b/meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch @@ -0,0 +1,382 @@ +From 5b1c70bfe0d8f84dc28237d6150b7b9d57c791a8 Mon Sep 17 00:00:00 2001 +From: Victor Do Nascimento +Date: Tue, 16 Apr 2024 11:49:15 +0100 +Subject: [PATCH] aarch64: Remove asserts from operand qualifier decoders + [PR31595] + +Given that the disassembler should never abort when decoding +(potentially random) data, assertion statements in the +`get_*reg_qualifier_from_value' function family prove problematic. + +Consider the random 32-bit word W, encoded in a data segment and +encountered on execution of `objdump -D '. + +If: + + (W & ~opcode_mask) == valid instruction + +Then before `print_insn_aarch64_word' has a chance to report the +instruction as potentially undefined, an attempt will be made to have +the qualifiers for the instruction's register operands (if any) +decoded. If the relevant bits do not map onto a valid qualifier for +the matched instruction-like word, an abort will be triggered and the +execution of objdump aborted. + +As this scenario is perfectly feasible and, in light of the fact that +objdump must successfully decode all sections of a given object file, +it is not appropriate to assert in this family of functions. + +Therefore, we add a new pseudo-qualifier `AARCH64_OPND_QLF_ERR' for +handling invalid qualifier-associated values and re-purpose the +assertion conditions in qualifier-retrieving functions to be the +predicate guarding the returning of the calculated qualifier type. +If the predicate fails, we return this new qualifier and allow the +caller to handle the error as appropriate. + +As these functions are called either from within +`aarch64_extract_operand' or `do_special_decoding', both of which are +expected to return non-zero values, it suffices that callers return +zero upon encountering `AARCH64_OPND_QLF_ERR'. + +Ar present the error presented in the hypothetical scenario has been +encountered in `get_sreg_qualifier_from_value', but the change is made +to the whole family to keep the interface consistent. + +Bug: https://sourceware.org/PR31595 + +Upstream-Status: Backport [commit 2601b201e95ea0edab89342ee7137c74e88a8a79] + +Signed-off-by: Mark Hatle +--- + .../testsuite/binutils-all/aarch64/illegal.d | 1 + + .../testsuite/binutils-all/aarch64/illegal.s | 3 + + include/opcode/aarch64.h | 3 + + opcodes/aarch64-dis.c | 98 +++ + 4 files changed, 87 insertions(+), 18 deletions(-) + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +index 4b90a1d9f39..b69318aec85 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.d b/binutils/testsuite/binutils-all/aarch64/illegal.d +@@ -8,5 +8,6 @@ Disassembly of section \.text: + + 0+000 <.*>: + [ ]+0:[ ]+68ea18cc[ ]+.inst[]+0x68ea18cc ; undefined ++[ ]+4:[ ]+9dc39839[ ]+.inst[]+0x9dc39839 ; undefined + #pass + +diff --git a/binutils/testsuite/binutils-all/aarch64/illegal.s b/binutils/testsuite/binutils-all/aarch64/illegal.s +index 216cbe6f265..43668c6db55 100644 +--- a/binutils/testsuite/binutils-all/aarch64/illegal.s b/binutils/testsuite/binutils-all/aarch64/illegal.s +@@ -4,4 +4,7 @@ + # ldpsw x12, x6, [x6],#
[OE-core] [scarthgap 2/2] gcc: Fix for CVE-2024-0151
Fix for insufficient argument checking in Secure state Entry functions in software using Cortex-M Security Extensions (CMSE), that has been compiled using toolchains that implement 'Arm v8-M Security Extensions Requirements on Development Tools' prior to version 1.4, allows an attacker to pass values to Secure state that are out of range for types smaller than 32-bits. Out of range values might lead to incorrect operations in secure state. Signed-off-by: Mark Hatle --- meta/recipes-devtools/gcc/gcc-13.2.inc| 1 + .../gcc/gcc/CVE-2024-0151.patch | 315 ++ 2 files changed, 316 insertions(+) create mode 100644 meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch diff --git a/meta/recipes-devtools/gcc/gcc-13.2.inc b/meta/recipes-devtools/gcc/gcc-13.2.inc index 603377a49a..abf177822b 100644 --- a/meta/recipes-devtools/gcc/gcc-13.2.inc +++ b/meta/recipes-devtools/gcc/gcc-13.2.inc @@ -68,6 +68,7 @@ SRC_URI = "${BASEURI} \ file://CVE-2023-4039.patch \ file://0026-aarch64-Fix-loose-ldpstp-check-PR111411.patch \ file://0027-Fix-gcc-vect-module-testcases.patch \ + file://CVE-2024-0151.patch \ " SRC_URI[sha256sum] = "e275e76442a6067341a27f04c5c6b83d8613144004c0413528863dc6b5c743da" diff --git a/meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch b/meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch new file mode 100644 index 00..01d55b5cdb --- /dev/null +++ b/meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch @@ -0,0 +1,315 @@ +arm: Zero/Sign extends for CMSE security + +This patch makes the following changes: + +1) When calling a secure function from non-secure code then any arguments + smaller than 32-bits that are passed in registers are zero- or sign-extended. +2) After a non-secure function returns into secure code then any return value + smaller than 32-bits that is passed in a register is zero- or sign-extended. + +This patch addresses the following CVE-2024-0151. + +gcc/ChangeLog: +PR target/114837 +* config/arm/arm.cc (cmse_nonsecure_call_inline_register_clear): + Add zero/sign extend. +(arm_expand_prologue): Add zero/sign extend. + +gcc/testsuite/ChangeLog: + +* gcc.target/arm/cmse/extend-param.c: New test. +* gcc.target/arm/cmse/extend-return.c: New test. + +CVE: CVE-2023-4039 +Upstream-Status: Backport [https://gcc.gnu.org/pipermail/gcc-patches/2024-April/649973.html] +Signed-off-by: Mark Hatle + +diff --git a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc +index 0217abc218d60956ce727e6d008d46b9176dddc5..ea0c963a4d67ecd70e1571624e84dfe46d757df9 100644 +--- a/gcc/config/arm/arm.cc b/gcc/config/arm/arm.cc +@@ -19210,6 +19210,30 @@ cmse_nonsecure_call_inline_register_clear (void) + end_sequence (); + emit_insn_before (seq, insn); + ++/* The AAPCS requires the callee to widen integral types narrower ++ than 32 bits to the full width of the register; but when handling ++ calls to non-secure space, we cannot trust the callee to have ++ correctly done so. So forcibly re-widen the result here. */ ++tree ret_type = TREE_TYPE (fntype); ++if ((TREE_CODE (ret_type) == INTEGER_TYPE ++|| TREE_CODE (ret_type) == ENUMERAL_TYPE ++|| TREE_CODE (ret_type) == BOOLEAN_TYPE) ++&& known_lt (GET_MODE_SIZE (TYPE_MODE (ret_type)), 4)) ++ { ++machine_mode ret_mode = TYPE_MODE (ret_type); ++rtx extend; ++if (TYPE_UNSIGNED (ret_type)) ++ extend = gen_rtx_ZERO_EXTEND (SImode, ++gen_rtx_REG (ret_mode, R0_REGNUM)); ++else ++ extend = gen_rtx_SIGN_EXTEND (SImode, ++gen_rtx_REG (ret_mode, R0_REGNUM)); ++emit_insn_after (gen_rtx_SET (gen_rtx_REG (SImode, R0_REGNUM), ++ extend), insn); ++ ++ } ++ ++ + if (TARGET_HAVE_FPCXT_CMSE) + { + rtx_insn *last, *pop_insn, *after = insn; +@@ -23652,6 +23676,51 @@ arm_expand_prologue (void) + + ip_rtx = gen_rtx_REG (SImode, IP_REGNUM); + ++ /* The AAPCS requires the callee to widen integral types narrower ++ than 32 bits to the full width of the register; but when handling ++ calls to non-secure space, we cannot trust the callee to have ++ correctly done so. So forcibly re-widen the result here. */ ++ if (IS_CMSE_ENTRY (func_type)) ++{ ++ function_args_iterator args_iter; ++ CUMULATIVE_ARGS args_so_far_v; ++ cumulative_args_t args_so_far; ++ bool first_param = true; ++ tree arg_type; ++ tree fndecl = current_function_decl; ++ tree fntype = TREE_TYPE (fndecl); ++ arm_init_cumulative_args (&args_so_far_v, fntype, NULL_RTX, fndecl); ++ args_so_
[OE-core] [scarthgap 0/2] Toolchain fixes for Scarthgap
Two different toolchain fixes for Scarthgap. The binutils change was found in a AMD/Xilinx specific build script that attempts to disassemble some code as part of a firmware setup. This cortex-a53 baremetal firmware triggered a fault fixed by the issue. The newer versions of binutils have already accepted this change, but I'm not exactly sure which version(s) have it other then then the current development tree. The GCC change only affects cortex-M. This is simply a port of the GCC accepted change into Yocto Project for baremetal usage. Unfortunately I don't have any sort of test cases to show it works or doesn't. It does not break any of my existing test cases related to Linux for cortex-r baremetal. Again this change has already been accepted to newer versions of GCC. Mark Hatle (2): binutils: Fix aarch64 disassembly abort gcc: Fix for CVE-2024-0151 .../binutils/binutils-2.42.inc| 1 + ...sserts-from-operand-qualifier-decode.patch | 382 ++ meta/recipes-devtools/gcc/gcc-13.2.inc| 1 + .../gcc/gcc/CVE-2024-0151.patch | 315 +++ 4 files changed, 699 insertions(+) create mode 100644 meta/recipes-devtools/binutils/binutils/0016-aarch64-Remove-asserts-from-operand-qualifier-decode.patch create mode 100644 meta/recipes-devtools/gcc/gcc/CVE-2024-0151.patch -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#199860): https://lists.openembedded.org/g/openembedded-core/message/199860 Mute This Topic: https://lists.openembedded.org/mt/106288401/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master][scarthgap][PATCH] sstate.bbclass: Add _SSTATE_EXCLUDEDEPS_SYSROOT to vardepsexclude
When using tinfoil to control the build, multiple commands (serially) could trigger an error such as: When reparsing bb:do_package, the basehash value changed from ... to The metadata is not deterministic and this needs to be fixed. ERROR: The following commands may help: ERROR: $ bitbake esw-conf -cdo_package -Snone ERROR: Then: ERROR: $ bitbake esw-conf -cdo_package -Sprintdiff However following these commands it was not able to be reproduced. Forcing bitbake to dump the signatures and then running bitbake-diffsigs showed that the value of _SSTATE_EXCLUDEDEPS_SYSROOT was being set in one run, but was blank is a different version. Upon inspecting the code in sstate.bbclass, one usage (without the _) is already excludes, the leading _ version is used as a cache, only if set but is not actually required to be defined. So ignoring the value should work properly. Signed-off-by: Mark Hatle --- I found this issue on scarthgap, but I've seen similar symptoms back to langdale. Most likely it's an issue back to kirkstone, but clearly rare outside of a specific workdlow with tinfoil vs command line bitbake! meta/classes-global/sstate.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes-global/sstate.bbclass b/meta/classes-global/sstate.bbclass index 04539bbb99..76a7b59636 100644 --- a/meta/classes-global/sstate.bbclass +++ b/meta/classes-global/sstate.bbclass @@ -1115,7 +1115,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True, bb.parse.siggen.checkhashes(sq_data, missed, found, d) return found -setscene_depvalid[vardepsexclude] = "SSTATE_EXCLUDEDEPS_SYSROOT" +setscene_depvalid[vardepsexclude] = "SSTATE_EXCLUDEDEPS_SYSROOT _SSTATE_EXCLUDEDEPS_SYSROOT" BB_SETSCENE_DEPVALID = "setscene_depvalid" -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#198886): https://lists.openembedded.org/g/openembedded-core/message/198886 Mute This Topic: https://lists.openembedded.org/mt/105851669/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 36/36] xz: upgrade 5.4.6 -> 5.6.1 _WARNING_
I know this request is a week or so old.. But do NOT upgrade to 'xz' 5.6.0 or 5.6.1. It has been compromised: https://www.openwall.com/lists/oss-security/2024/03/29/4 --Mark On 3/14/24 8:40 AM, Richard Purdie wrote: On Wed, 2024-03-13 at 15:08 +0800, wangmy via lists.openembedded.org wrote: From: Wang Mingyu License-Update: *COPYING: Add the license for the XZ logo. Change most public domain parts to 0BSD. Update COPYING about the man pages of the scripts. *getopt.c MSVC: Don't #include . lib/getopt*.c: Include only HAVE_CONFIG_H is defined. lib: Update getopt.c from Gnulib with modifications. lib: Silence -Wsign-conversion in getopt.c. Add SPDX license identifiers to GPL, LGPL, and FSFULLR files. Changelog: = * liblzma: Fixed two bugs relating to GNU indirect function (IFUNC) with GCC. * xz: Changed the messages for thread reduction due to memory constraints to only appear under the highest verbosity level. * Build: - Fixed a build issue when the header file was present on the system but the Landlock system calls were not defined in . - The CMake build now warns and disables NLS if both gettext tools and pre-created .gmo files are missing. Previously, this caused the CMake build to fail. * Minor improvements to man pages. * Minor improvements to tests. https://autobuilder.yoctoproject.org/typhoon/#/builders/48/builds/8737 Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#197643): https://lists.openembedded.org/g/openembedded-core/message/197643 Mute This Topic: https://lists.openembedded.org/mt/105226831/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core][PATCH] testimage: allow to set runqemu bootparams option
Will this work in the case of a system boot? I know if we load the kernel directly to memory and boot, we can pass kernel command line through QEMU. However many boards I have worked with in the past (and still do) do a system level boot that starts with BIOS/Firmware, goes into a second stage loader (grub/u-boot) and then starts the kernel. How would these parameters end up going to the kernel in that case, or are they silently ignored? Is silently ignored on a system boot the behavior we want? --Mark On 3/1/24 9:05 AM, M. Seben via lists.openembedded.org wrote: From: michal seben Allow to set runqemu bootparams option using TEST_EXTRABOOTPARAMS variable. This option can be used .e.g. to mask service only for testimage task. Signed-off-by: michal seben --- meta/classes-recipe/testimage.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/meta/classes-recipe/testimage.bbclass b/meta/classes-recipe/testimage.bbclass index ad040ee8f0..6e2800e3c3 100644 --- a/meta/classes-recipe/testimage.bbclass +++ b/meta/classes-recipe/testimage.bbclass @@ -47,6 +47,8 @@ TESTIMAGE_FAILED_QA_ARTIFACTS += "${@bb.utils.contains('DISTRO_FEATURES', 'ptest # TEST_QEMUBOOT_TIMEOUT can be used to set the maximum time in seconds the launch code will wait for the login prompt. # TEST_OVERALL_TIMEOUT can be used to set the maximum time in seconds the tests will be allowed to run (defaults to no limit). # TEST_QEMUPARAMS can be used to pass extra parameters to qemu, e.g. "-m 1024" for setting the amount of ram to 1 GB. +# TEST_EXTRABOOTPARAMS can be used to set the 'bootparams' option for the runqemu script +# e.g. "systemd.mask=NetworkManager-wait-online.service" to mask particular service # TEST_RUNQEMUPARAMS can be used to pass extra parameters to runqemu, e.g. "gl" to enable OpenGL acceleration. # QEMU_USE_KVM can be set to "" to disable the use of kvm (by default it is enabled if target_arch == build_arch or both of them are x86 archs) @@ -358,7 +360,7 @@ def testimage_main(d): try: # We need to check if runqemu ends unexpectedly # or if the worker send us a SIGTERM -tc.target.start(params=d.getVar("TEST_QEMUPARAMS"), runqemuparams=d.getVar("TEST_RUNQEMUPARAMS")) +tc.target.start(params=d.getVar("TEST_QEMUPARAMS"), extra_bootparams=d.getVar("TEST_EXTRABOOTPARAMS"), runqemuparams=d.getVar("TEST_RUNQEMUPARAMS")) import threading try: threading.Timer(int(d.getVar("TEST_OVERALL_TIMEOUT")), handle_test_timeout, (int(d.getVar("TEST_OVERALL_TIMEOUT")),)).start() -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#196641): https://lists.openembedded.org/g/openembedded-core/message/196641 Mute This Topic: https://lists.openembedded.org/mt/104665175/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] core-image-full-cmdline: add package-management
There are plenty of systems where we don't want package-management enabled by default. This will just make core-image-full-cmdline less useful there. In the past, it's always been up to the user to enable full package-management in their configuration, default has been to not have it present. Has this changed? On 2/29/24 9:03 AM, Michael Opdenacker via lists.openembedded.org wrote: From: Michael Opdenacker Add "package-management" image feature to the core-image-full-cmdline image, to support package upgrade testing. Signed-off-by: Michael Opdenacker Suggested-by: Richard Purdie --- meta/recipes-extended/images/core-image-full-cmdline.bb | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/recipes-extended/images/core-image-full-cmdline.bb b/meta/recipes-extended/images/core-image-full-cmdline.bb index 4e1cf58d55..b034cd0aeb 100644 --- a/meta/recipes-extended/images/core-image-full-cmdline.bb +++ b/meta/recipes-extended/images/core-image-full-cmdline.bb @@ -1,7 +1,7 @@ SUMMARY = "A console-only image with more full-featured Linux system \ functionality installed." -IMAGE_FEATURES += "splash ssh-server-openssh" +IMAGE_FEATURES += "splash ssh-server-openssh package-management" IMAGE_INSTALL = "\ packagegroup-core-boot \ -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#196445): https://lists.openembedded.org/g/openembedded-core/message/196445 Mute This Topic: https://lists.openembedded.org/mt/104644618/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [poky] [Openembedded-architecture] [RFC PATCH] Add genericarm64 MACHINE using upstream defconfig
On 2/21/24 9:06 AM, Paul Barker wrote: On 21/02/2024 10:57, Ross Burton wrote: From: Ross Burton This is a new 64-bit "generic" Arm machine, that expects the hardware to be SystemReady IR compatible. This is slightly forward-leaning as there's not a _lot_ of SystemReady hardware in the wild, but most modern boards are and the number will only grow. Also, this is the only way to have a 'generic' machine as without standardised bootloaders and firmware it would be impossible. The base machine configuration isn't that exciting: it's a fully featured machine that supports most things, booting via UEFI and an initramfs. However, the kernel is more interesting. This RFC uses the upstream defconfig because unlike some other platforms, the arm64 defconfig is actively maintained with the goal of being a 'boots on most hardware' configuration. My argument is: why would we duplicate that effort? The "linux-yocto way" is configuration fragments and after a week of hair-pulling I do actually have fragments that boot on a BeaglePlay, but to say this was a tiresome and frustrating exercise would be understating it. So, a request for comments: is it acceptable to use the upstream defconfig in a reference BSP? Personally I'm torn: the Yocto way is fragments not monolithic configs, but repeating the effort to fragmentise the configuration and then also have it sufficiently modular that it can be used in pieces - instead of just being a large file split up into smaller files - is a lot of effort for what might end up being minimal gain. My fear is we end up with a fragmented configuration that can't be easily modified without breaking some platforms, and badly copies what the defconfig already does. I am in favour of this - I think the "genericarm64" machine should use the in-tree defconfig so that it can support the widest array of hardware. If someone wants to trim down the kernel for a particular platform then they should probably create a specific MACHINE anyway. If we take the other approach of building up the kernel config from fragments, how would we know that all SystemReady IR capable systems will be supported? Yocto Project doesn't have the resources to test every platform. I disagree here. I think it would be MUCH better to have a 'SystemReady IR' hardware configuration. So if SystemReady IR is desired, it is something that anyone can enable (starting with genericarm64). Remember the defconfig is going to have more then hardware configs in it. Will it have the right systemd configurations? Will is have the magic filesystem a random user wants? Will avoid having some other filesystem type that another user doesn't want? Building up the kernel, and considering SystemReady IR as a 'hardware feature', and then add in the additional things that are needed for whatever reason is a much more reasonable way to do this and make it useful to otthers. For the Renesas RZ SoCs I work on these days, the in-tree defconfig is the configuration we test with the mainline kernel. AMD does the same thing, for the kernel development it makes sense. Kernel is built and tested standalone from userspace. But with that said, I think it's the wrong way to do Yocto Project development. Yocto Project development needs further control and the separation of hardware and software configurations is pretty essential to having a system that can be customized appropriately. The defconfig can be used as a guide to the other configurations, but separating hardware and software configs is a necessary first step in my opinion. --Mark Thanks, -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#196005): https://lists.openembedded.org/g/openembedded-core/message/196005 Mute This Topic: https://lists.openembedded.org/mt/104502773/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH 1/1] qemu: Allow native and nativesdk versions on Linux older then 4.17
QEMU is a symptom of the real problem. SDK_OLDEST_KERNEL is set to 3.2.0. If that is what we keep it as, then we need to make sure that the nativesdk tooling works on a pre 4.17 kernel. If that value moves to 4.17 then we can't use the same uninative / buildtools-tarball with older (LTS) releases. So there is no "good" solution here. As you said, it's RPs decision, I see three courses here: 1) Accept qemu mmap calls could break silently on pre-4.17 kernels 2) Apply this patch, allowing SDK_OLDEST_KERNEL to remain at 3.2.0 3) Bump SDK_OLDEST_KERNEL to at least 4.17 or ? I'm not really partial to one solution or another in this case, but we do have an underlying issue that needs to be resolved or we're going to be chasing bug reports that we have no way to resolve. --Mark On 1/25/24 3:41 PM, Alexander Kanavin wrote: Initial release of Ubuntu 18.04 had 4.15 alright. Later point releases offered much newer kernels. It’s also altogether EOL since end of may 2023. Why should oe-core carry these? It’s RP’s call as qemu maintainer but my vote is a firm no, unless additional arguments are put forward. Alex On Thu 25. Jan 2024 at 22.04, Mark Hatle <mailto:mark.ha...@kernel.crashing.org>> wrote: From: Mark Hatle mailto:mark.ha...@amd.com>> Linux kernel 4.17 introduced two new mmap flags, MAP_FIXED_NOREPLACE and MAP_SHARED_VALIDATE. Starting with QEMU 8.1, these flags are now used and required for proper system operation. In order to build and run on a system older then 4.17, we need to emulate this new behavior. Not having a newer kernel could result in the mmap memory being allocated in a way that will cause failures without QEMU checking for these conditions. Note, memory allocation issues are rare in my experience so this is more of a 'just-in-case' behavior. SDK_OLDEST_KERNEL is currently set to 3.2.0, the only way this can claim that qemu works in an SDK is by checking the return values to emulate the expected behavior. Signed-off-by: Mark Hatle mailto:mark.ha...@amd.com>> Signed-off-by: Mark Hatle mailto:mark.ha...@kernel.crashing.org>> --- meta/recipes-devtools/qemu/qemu.inc | 12 + ...round-for-missing-MAP_FIXED_NOREPLAC.patch | 286 ++ ...round-for-missing-MAP_SHARED_VALIDAT.patch | 51 3 files changed, 349 insertions(+) create mode 100644 meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch create mode 100644 meta/recipes-devtools/qemu/qemu/0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc index ccb2880402..7c31a5aa83 100644 --- a/meta/recipes-devtools/qemu/qemu.inc +++ b/meta/recipes-devtools/qemu/qemu.inc @@ -39,6 +39,18 @@ SRC_URI = "https://download.qemu.org/${BPN}-${PV}.tar.xz <https://download.qemu.org/$%7BBPN%7D-$%7BPV%7D.tar.xz> \ " UPSTREAM_CHECK_REGEX = "qemu-(?P\d+(\.\d+)+)\.tar" +# SDK_OLDEST_KERNEL is set below 4.17, which is the minimum version required by QEMU >= 8.1 +# This is due to two MMAP flags being used at certain points +SRC_URI:append:class-nativesdk = " \ + file://0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch \ + file://0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch \ + " + +# Support building and using native version on pre 4.17 kernels +SRC_URI:append:class-native = " \ + file://0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch \ + file://0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch \ + " SRC_URI[sha256sum] = "bf00d2fa12010df8b0ade93371def58e632cb32a6bfdc5f5a0ff8e6a1fb1bf32" diff --git a/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch b/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch new file mode 100644 index 00..8941911fb3 --- /dev/null +++ b/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch @@ -0,0 +1,286 @@ +From fa9bcabe2387bb230ef82d62827ad6f93b8a1e61 Mon Sep 17 00:00:00 2001 +From: Frederic Konrad mailto:fkon...@amd.com>> +Date: Wed, 17 Jan 2024 18:15:06 + +Subject: [PATCH 1/2] linux-user/*: workaround for missing MAP_FIXED_NOREPLACE + +QEMU v8.1.0 recently requires MAP_FIXED_NOREPLACE flags implementation for mmap. + +This is missing from ubuntu 18.04, thus this patch catches the mmap calls which +could use that new flag and forwards them to mmap when MAP_FIXED_NOREPLACE +f
Re: [OE-core] [PATCH 1/1] qemu: Allow native and nativesdk versions on Linux older then 4.17
On 1/25/24 3:42 PM, Martin Jansa wrote: I did something similar for LGE when we were using 18.04 ubuntu, but IIRC it's not about the kernel version, but glibc being older than 2.27. 18.04 is already unsupported for a while, I don't think oe-core should support unsupported host OS versions (that's why I've never send my version). To make this change smaller you can just revert c42e77a90d9244c8caf76fe0e54f84200430a4e1 from qemu. If oe-core really needs to support this I can share my version as in: https://github.com/shr-project/meta-webosose/commit/7528c20bf3ba7576d4611f71f987a408ac8845c2 <https://github.com/shr-project/meta-webosose/commit/7528c20bf3ba7576d4611f71f987a408ac8845c2> This version includes an actual check that the mmap returned a proper value. The version here just sets the flags to avoid the compilation issue. Using the buildtools-tarball in your build will accomplish the same end result, hides the problem but doesn't verify the mmap will actually work (or error) as needed. --Mark Regards, On Thu, Jan 25, 2024 at 10:04 PM Mark Hatle <mailto:mark.ha...@kernel.crashing.org>> wrote: From: Mark Hatle mailto:mark.ha...@amd.com>> Linux kernel 4.17 introduced two new mmap flags, MAP_FIXED_NOREPLACE and MAP_SHARED_VALIDATE. Starting with QEMU 8.1, these flags are now used and required for proper system operation. In order to build and run on a system older then 4.17, we need to emulate this new behavior. Not having a newer kernel could result in the mmap memory being allocated in a way that will cause failures without QEMU checking for these conditions. Note, memory allocation issues are rare in my experience so this is more of a 'just-in-case' behavior. SDK_OLDEST_KERNEL is currently set to 3.2.0, the only way this can claim that qemu works in an SDK is by checking the return values to emulate the expected behavior. Signed-off-by: Mark Hatle mailto:mark.ha...@amd.com>> Signed-off-by: Mark Hatle mailto:mark.ha...@kernel.crashing.org>> --- meta/recipes-devtools/qemu/qemu.inc | 12 + ...round-for-missing-MAP_FIXED_NOREPLAC.patch | 286 ++ ...round-for-missing-MAP_SHARED_VALIDAT.patch | 51 3 files changed, 349 insertions(+) create mode 100644 meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch create mode 100644 meta/recipes-devtools/qemu/qemu/0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc index ccb2880402..7c31a5aa83 100644 --- a/meta/recipes-devtools/qemu/qemu.inc +++ b/meta/recipes-devtools/qemu/qemu.inc @@ -39,6 +39,18 @@ SRC_URI = "https://download.qemu.org/${BPN}-${PV}.tar.xz <https://download.qemu.org/$%7BBPN%7D-$%7BPV%7D.tar.xz> \ " UPSTREAM_CHECK_REGEX = "qemu-(?P\d+(\.\d+)+)\.tar" +# SDK_OLDEST_KERNEL is set below 4.17, which is the minimum version required by QEMU >= 8.1 +# This is due to two MMAP flags being used at certain points +SRC_URI:append:class-nativesdk = " \ + file://0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch \ + file://0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch \ + " + +# Support building and using native version on pre 4.17 kernels +SRC_URI:append:class-native = " \ + file://0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch \ + file://0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch \ + " SRC_URI[sha256sum] = "bf00d2fa12010df8b0ade93371def58e632cb32a6bfdc5f5a0ff8e6a1fb1bf32" diff --git a/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch b/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch new file mode 100644 index 00..8941911fb3 --- /dev/null +++ b/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch @@ -0,0 +1,286 @@ +From fa9bcabe2387bb230ef82d62827ad6f93b8a1e61 Mon Sep 17 00:00:00 2001 +From: Frederic Konrad mailto:fkon...@amd.com>> +Date: Wed, 17 Jan 2024 18:15:06 + +Subject: [PATCH 1/2] linux-user/*: workaround for missing MAP_FIXED_NOREPLACE + +QEMU v8.1.0 recently requires MAP_FIXED_NOREPLACE flags implementation for mmap. + +This is missing from ubuntu 18.04, thus this patch catches the mmap calls which +could use that new flag and forwards them to mmap when MAP_FIXED_NOREPLACE +flag isn't set or emulates them by checking the returned address w.r.t the +requested
[OE-core] [PATCH 0/1] Support running qemu on kernel older then 4.17
We were attempting to build qemu on an Ubuntu 18.04 system and ran into an issue where certain (newer) MMAP flags were not defined. After further tracking it was determined that QEMU 8.1 moved forward and only supports usage on kernel 4.17 or newer. Using the patch included with this, you can build for an older host, but more importantly you can build an SDK that includes QEMU that will execute on the 'SDK_OLDEST_KERNEL'. While I've not gone back and verified things work on 3.2.0 system, I have verified that Ubuntu 18.04 is working for me. (Ubuntu 18.04 is kernel 4.15) Mark Hatle (1): qemu: Allow native and nativesdk versions on Linux older then 4.17 meta/recipes-devtools/qemu/qemu.inc | 12 + ...round-for-missing-MAP_FIXED_NOREPLAC.patch | 286 ++ ...round-for-missing-MAP_SHARED_VALIDAT.patch | 51 3 files changed, 349 insertions(+) create mode 100644 meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch create mode 100644 meta/recipes-devtools/qemu/qemu/0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#194332): https://lists.openembedded.org/g/openembedded-core/message/194332 Mute This Topic: https://lists.openembedded.org/mt/103962277/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/1] qemu: Allow native and nativesdk versions on Linux older then 4.17
From: Mark Hatle Linux kernel 4.17 introduced two new mmap flags, MAP_FIXED_NOREPLACE and MAP_SHARED_VALIDATE. Starting with QEMU 8.1, these flags are now used and required for proper system operation. In order to build and run on a system older then 4.17, we need to emulate this new behavior. Not having a newer kernel could result in the mmap memory being allocated in a way that will cause failures without QEMU checking for these conditions. Note, memory allocation issues are rare in my experience so this is more of a 'just-in-case' behavior. SDK_OLDEST_KERNEL is currently set to 3.2.0, the only way this can claim that qemu works in an SDK is by checking the return values to emulate the expected behavior. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/recipes-devtools/qemu/qemu.inc | 12 + ...round-for-missing-MAP_FIXED_NOREPLAC.patch | 286 ++ ...round-for-missing-MAP_SHARED_VALIDAT.patch | 51 3 files changed, 349 insertions(+) create mode 100644 meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch create mode 100644 meta/recipes-devtools/qemu/qemu/0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch diff --git a/meta/recipes-devtools/qemu/qemu.inc b/meta/recipes-devtools/qemu/qemu.inc index ccb2880402..7c31a5aa83 100644 --- a/meta/recipes-devtools/qemu/qemu.inc +++ b/meta/recipes-devtools/qemu/qemu.inc @@ -39,6 +39,18 @@ SRC_URI = "https://download.qemu.org/${BPN}-${PV}.tar.xz \ " UPSTREAM_CHECK_REGEX = "qemu-(?P\d+(\.\d+)+)\.tar" +# SDK_OLDEST_KERNEL is set below 4.17, which is the minimum version required by QEMU >= 8.1 +# This is due to two MMAP flags being used at certain points +SRC_URI:append:class-nativesdk = " \ + file://0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch \ + file://0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch \ +" + +# Support building and using native version on pre 4.17 kernels +SRC_URI:append:class-native = " \ + file://0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch \ + file://0012-linux-user-workaround-for-missing-MAP_SHARED_VALIDAT.patch \ +" SRC_URI[sha256sum] = "bf00d2fa12010df8b0ade93371def58e632cb32a6bfdc5f5a0ff8e6a1fb1bf32" diff --git a/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch b/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch new file mode 100644 index 00..8941911fb3 --- /dev/null +++ b/meta/recipes-devtools/qemu/qemu/0011-linux-user-workaround-for-missing-MAP_FIXED_NOREPLAC.patch @@ -0,0 +1,286 @@ +From fa9bcabe2387bb230ef82d62827ad6f93b8a1e61 Mon Sep 17 00:00:00 2001 +From: Frederic Konrad +Date: Wed, 17 Jan 2024 18:15:06 + +Subject: [PATCH 1/2] linux-user/*: workaround for missing MAP_FIXED_NOREPLACE + +QEMU v8.1.0 recently requires MAP_FIXED_NOREPLACE flags implementation for mmap. + +This is missing from ubuntu 18.04, thus this patch catches the mmap calls which +could use that new flag and forwards them to mmap when MAP_FIXED_NOREPLACE +flag isn't set or emulates them by checking the returned address w.r.t the +requested address. + +Signed-off-by: Frederic Konrad +Signed-off-by: Francisco Iglesias + +Upstream-Status: Inappropriate [OE specific] + +The upstream only supports the last two major releases of an OS. The ones +they have declared all have kernel 4.17 or newer. + +See: +https://xilinx.slack.com/archives/D04G2647CTV/p1705074697942019 + +https://www.qemu.org/docs/master/about/build-platforms.html + + The project aims to support the most recent major version at all times for up + to five years after its initial release. Support for the previous major + version will be dropped 2 years after the new major version is released or + when the vendor itself drops support, whichever comes first. + +Signed-off-by: Mark Hatle +--- + linux-user/elfload.c| 7 +++-- + linux-user/meson.build | 1 + + linux-user/mmap-fixed.c | 63 + + linux-user/mmap-fixed.h | 39 + + linux-user/mmap.c | 31 +++- + linux-user/syscall.c| 1 + + 6 files changed, 125 insertions(+), 17 deletions(-) + create mode 100644 linux-user/mmap-fixed.c + create mode 100644 linux-user/mmap-fixed.h + +Index: qemu-8.2.0/linux-user/elfload.c +=== +--- qemu-8.2.0.orig/linux-user/elfload.c qemu-8.2.0/linux-user/elfload.c +@@ -22,6 +22,7 @@ + #include "qemu/error-report.h" + #include "target_signal.h" + #include "accel/tcg/debuginfo.h" ++#include "mmap-fixed.h" + + #ifdef TARGET_ARM + #include "target/arm/cpu-features.h" +@@ -2765,9 +2766,9 @@ static abi_ulong create_elf_tables(abi_u
Re: [OE-core] Removing Github release SRC_URIs from oe-core recipes?
On 1/15/24 7:50 AM, Jasper Orschulko via lists.openembedded.org wrote: Hi Alex, Okay, I've read the README file in that repo, and if i understood it right, the process is: - run fossology - have a human inspect the output, and correct it on a file by file basis (tremendous waste of time and limited developer resources even when done the 'open source way' if you ask me but whatevs) - place the corrected output into the above repository Correct. Do you really really need the 'human corrected' part of all this? Unfortunately we do need the "human corrected" part (I wish we wouldn't). We have industry customers (and in turn our legal department) that demand a license compliance report and clearing on a "per-file" basis. Currently available scanning tools are unfortunately way to unreliable for usage without any human interaction (believe me, we tried). Will this be stupid amount of work? Yes. Is this compliance obligations gone mad? IMO, absolutely yes. But unfortunately this seems to be the way the industry is heading. We as a supplier company are getting more and more requests of this sort from our clients (big players in the public transport and automotive industry) who won't "play" with us unless these obligations are fulfilled. I've also heared similar stories from other companies. (That's what you get when you let company lawyers go wild 😉️) So I'm not happy with the situation but doing the best I can, given the requirements. In our case that means sharing our license clearings (which we have to do in any case) with the open source community, in the hope that other companies have similar challenges and that we can get some crowd-sourcing going. It will never possibly cover all of the packages you need to ship and match all their versions. So what we are currently striving for is covering all target relevant packages (aka without any special suffixes) of a "basic linux build" (aka image-core-minimal) for LTS releases (starting with kirkstone). Additionally, meta-ossselot will have logic for re-use of other package versions (so does osselot as a whole), limiting curation to the diff. I also have some ideas on dealing with openembedded patches added to point releases (currently another pain point). One note, when I worked on this a few years ago (pre-2020) each source file was correlated to an entry in our SPDX database. The licenses were manually reviewed and that was all attached to a file's hash. This way it didn't matter if you were looking at an upstream tarball, an upstream git repository, or any other mode of getting the source code. Tracking was effectively "human reviewed" SPDX was compared against the patched file manifest. Any sources that didn't match the checksum were flagged for review. A master database of checksums were also available to see if the file was duplicated in other repositories as well. This also allowed comparison of license/copyright and possibly patent data between different projects. (and ALSO allowed exceptions for generated m4, configure.in, etc files!) This manual review work absolutely needs to be done for some parts of the world and some company requirements... but be careful not to make the work so complex it can't be completed. --Mark Best, Jasper -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#193686): https://lists.openembedded.org/g/openembedded-core/message/193686 Mute This Topic: https://lists.openembedded.org/mt/103730186/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH RFC] bitbake.conf/pseudo: Switch from exclusion list to inclusion list
On 12/11/23 2:35 PM, Ross Burton wrote: On 11 Dec 2023, at 17:35, Richard Purdie via lists.openembedded.org wrote: Currently, pseudo tracks all files referenced within it's presence unless they're listed in an exclusion list. The exclusion list has grown to be fairly unwieldy. This patch swaps PSEUDO_IGNORE_PATHS for PSEUDO_INCLUDE_PATHS which in theory should be easier and more explicit to maintain. I’ve never understood the rationale behind exclude over include (then again there’s lots I don’t understand), so I’m very much for this in principle. History: It was originally everything, because files were referenced, linked and copied around random parts of the system and/or build and needed preservation of components. This was very early Yocto Project and before. (Psuedo existed for many years before OE/YP used it!) Then the build/work directories became more standardized, and the components 'copying in' from elsewhere was greatly reduced... but there were points where reference files were still used (think .m4 files or tex templates for instance for the kinds of things...) Since we didn't know what all of the reference points (to the host) were, we decided to be conservative to add exclusion. So excluding paths where we could count on owners/groups (and more importantly permissions) being standard became the norm. That's why the primary exclusion was: /usr/,/etc/,/lib,/dev/,/run/ Over time though we've gotten our dependencies understood and really severely limited the external components we use in the build. (including .m4, autoconf, tex, etc etc etc..) So moving to an INCLUDE seems to make more sense to me. Since we have a clear understanding of the path structure we use /tmp - used by the compiler (and tons of other tooling) for temporary files, we need this for sure. /proc - there are referenced entries that MIGHT be needed, especially links to /proc/mounts (where the link inherits perms) /dev - I'm not sure if this is needed, but things like /dev/null and /dev/shm, another tmpfs, mean we probably need to track this. /run - often contains other short-term temp files, especially for things like trying to process pre/post package scripting on rootfs configuration. Then the items in WORKDIR of course. IN THE PAST, we needed source code to be covered both to handle symlink references, as well as specific copy functions. But I suspect this is not needed any longer. The actual build directory also is likely not needed anymore (but had been in the past!) Leaving us the image, package, rootfs, etc directories. So I think this maps well to "how did we get here". The one question I ask, is why we're using ',' as a separator and not ':' like standard Unix style paths. But I suspect the answer is Windows level support but I'd be very tempted to move it back to ':' to make this more standard like other path separators. (But that can be done at another time!) --Mark Ross -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#192242): https://lists.openembedded.org/g/openembedded-core/message/192242 Mute This Topic: https://lists.openembedded.org/mt/103113368/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [Openembedded-architecture] [OE-core] Core workflow: sstate for all, bblock/bbunlock, tools for why is sstate not being reused?
On 11/5/23 1:43 PM, Adrian Freihofer wrote: On Sat, 2023-11-04 at 11:09 +, Richard Purdie wrote: On Sat, 2023-11-04 at 11:29 +0100, adrian.freiho...@gmail.com wrote: Hi Alex, hi Richard After some internal discussions, I would like to clarify my previous answers on this topic. * Usually there are two different workflows - application developers: could use an SDK with a locked sstate-cache. - Yocto/BSP developers: need an unlocked SDK. They change the recipes. * A locked SDK - can work with setscene from SSTATE_MIRRORS - setscene does caching in the SSTATE_DIR (no issue about that) - But network problems can occur during the initial build because bitbake executes many independent setscene tasks. Opening so many independent connections slows down the build, especially if the server treats them as a denial of service attack. - The denial of service problem is difficult to solve because each setscene task runs in its own bibtake task. Reusing a connection to download multiple sstate artifacts seems almost impossible. This is much easier to solve with separate sstate download script. FWIW, we did have a similar issue with do_fetch overloading servers/proxies/ISPs and added: do_fetch[number_threads] = "4" Finding the right place to put a thread limit on overall setscene tasks is harder but in theory possible. Or perhaps a "network capable tasks" thread limit? Is the overload caused by the initial query of sstate presence, or, does it happen when the setscene tasks themselves run? The most extreme situation is probably bitbake --setscene-only with an empty TMPDIR. Each of the setscene tasks establishes a new connection. A server receives so many connections that it treats them as a denial of service attack by throttling. A separate script would allow the same connection to be reused to download all the required artifacts. Limiting the number of threads does not really solve the issue because there are still the same amount of connections which get quickly opened. * An unlocked SDK - Tries to download the sstate cache for changed recipes and their dependencies, which obviously can't work. - The useless download requests slow down the build considerably and cause a high load on the servers without any benefit. Is this sstate over http(s) or something else? I seem to remember you mentioning sftp. If this were using sftp, it would be horribly slow as it was designed for a light overhead "does this exist?" check which http(s) can manage well. Yes, we are evaluating sftp. You are right, it is not optimal from a performance point of view. For example S3 is much faster. A compromise is to set up a limited number of parallel sftp connections. This has worked very well so far. The question of why we use sftp brings us to a larger topic that is probably relevant for almost all Yocto users, but not for the Yocto project itself: Security. There is usually a git server infrastructure that makes it possible to protect Git repositories with finely graded access policies. As the sstate-cache contains the same source code, the protection concept for the Git repositories must also be applied to the sstate-cache artifacts. First of all a user authentication is required for the sstate-mirror. An obvious idea is to use the same user authentication for the sstate- cache server as for the Git server. In addition to https, ssh is also often used for git repositories. SSH even offers some advantages in terms of user-friendliness and security (if a ssh agent is used). This consideration finally leads us to use the sftp protocol for the sstate mirror. This is also relatively easy to administer: Simply copy the user's public ssh keys from the git server to the sftp server. While being able to support ssh (or a related protocol) is useful, you need to also remember that MANY MANY organizations absolutely block SSH access through their firewalls. So _requiring_ sftp would be bad. Allowing it's usage would be good. As for logging in, https is transport 'security' but not authentication without additional helpers. I think it's absolutely reasonable to say https access either needs an external helper for authentication purposes or it's un-authenticated. If you want (internal to the company) then ssh/sftp or similar using the ssh-agent (or similar) should be the suggested approach. We don't want to exclude anyone, but we want to be clear on the limitations based on an organization's specific choice. If one then wants to scale an sstate-cache server for many different projects and users, one quickly wishes for an option for authorization at artifact level. Ideally, the access rights to the source code would be completely transferred to the associated sstate artifacts. For such an authorization the ssate mirror server would require the SRC_URI which was used to compile the sstate artifact. With thi
Re: [OE-core] Recent failures on master depmodwrapper-cross
I wanted to confirm, this did fix the problem for me. Thanks! On 10/16/23 10:54 AM, Yoann Congal wrote: Le lun. 16 oct. 2023 à 17:37, Mark Hatle a écrit : On 10/16/23 7:17 AM, Yoann Congal wrote: Hi Mark, Le lun. 16 oct. 2023 à 02:40, Mark Hatle via lists.openembedded.org a écrit : Running a number of builds recently, a small number of them failed. Note we ran about 20 builds and only 3 failed like this, the others completed. I'm wondering if maybe there is a race condition with the new code? FYI, RP did point me to this thread (that I did miss sorry) and I (co-author of the "new code") will send two patches to prevent the error the happen again. RP's hypothesis is that this is linked to S=WORKDIR recipes. Did all your crashed build happened on "depmodwrapper-cross" (Which is S=WORKDIR) ? Yes, they all failed on depmodwrapper-cross. Realy looks like the S=WORKDIR case ! Happy to test changes for you. Thanks! I've sent patches here : [PATCH 1/2] insane: skip unimplemented-ptest on S=WORKDIR recipes https://lists.openembedded.org/g/openembedded-core/message/189301 [PATCH 2/2] insane: unimplemented-ptest: ignore source file errors https://lists.openembedded.org/g/openembedded-core/message/189302 I've tested that the depmodwrapper-cross recipe is now skipped for the unimplemented-ptest check. Regards, --Mark Thanks! Error: ERROR: depmodwrapper-cross-1.0-r0 do_patch: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: 0001: *** 0002:do_qa_patch(d) 0003: File: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/sources/poky/meta/classes-global/insane.bbclass', lineno: 1379, function: do_qa_patch 1375:elif os.path.exists(os.path.join(srcdir, "t")) and any(filename.endswith('.t') for filename in os.listdir(os.path.join(srcdir, 't'))): 1376:oe.qa.handle_error("unimplemented-ptest", "%s: perl Test:: based tests detected" % d.getVar('PN'), d) 1377: 1378:# Detect pytest-based tests *** 1379:elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): 1380:oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) 1381: 1382:# Detect meson-based tests 1383:elif os.path.exists(os.path.join(srcdir, "meson.build")) and match_line_in_files(srcdir, "**/meson.build", r'\s*test\s*\('): File: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/sources/poky/meta/classes-global/insane.bbclass', lineno: 1357, function: match_line_in_files 1353: ### 1354:def match_line_in_files(toplevel, filename_glob, line_regex): 1355:import pathlib 1356:toppath = pathlib.Path(toplevel) *** 1357:for entry in toppath.glob(filename_glob): 1358:try: 1359:with open(entry, 'r', encoding='utf-8', errors='ignore') as f: 1360:for line in f.readlines(): 1361:if re.match(line_regex, line): File: '/usr/lib/python3.10/pathlib.py', lineno: 1034, function: glob 1030:drv, root, pattern_parts = self._flavour.parse_parts((pattern,)) 1031:if drv or root: 1032:raise NotImplementedError("Non-relative patterns are unsupported") 1033:selector = _make_selector(tuple(pattern_parts), self._flavour) *** 1034:for p in selector.select_from(self): 1035:yield p 1036: 1037:def rglob(self, pattern): 1038:"""Recursively yield all existing files (of any kind, including File: '/usr/lib/python3.10/pathlib.py', lineno: 493, function: _select_from 0489:yielded = set() 0490:try: 0491:successor_select = self.successor._select_from 0492:for starting_point in self._iterate_directories(parent_path, is_dir, scandir): *** 0493:for p in successor_select(starting_point, is_dir, exists, scandir): 0494:if p not in yielded: 0495:yield p 0496:yielded.add(p) 0497:finally: File: '/usr/lib/python3.10/pathlib.py', lineno: 440, function: _select_from 0436:_Selector.__init__(self, child_parts, flavour) 0437: 0438:def _select_from(self, parent_path, is_dir, exists, scandir): 0439:try: *** 0440:with scandir(parent_path) as scandir_it: 0441:entries = list(scandir_it) 0442:for entry in ent
Re: [OE-core] Recent failures on master depmodwrapper-cross
On 10/16/23 7:17 AM, Yoann Congal wrote: Hi Mark, Le lun. 16 oct. 2023 à 02:40, Mark Hatle via lists.openembedded.org a écrit : Running a number of builds recently, a small number of them failed. Note we ran about 20 builds and only 3 failed like this, the others completed. I'm wondering if maybe there is a race condition with the new code? FYI, RP did point me to this thread (that I did miss sorry) and I (co-author of the "new code") will send two patches to prevent the error the happen again. RP's hypothesis is that this is linked to S=WORKDIR recipes. Did all your crashed build happened on "depmodwrapper-cross" (Which is S=WORKDIR) ? Yes, they all failed on depmodwrapper-cross. Happy to test changes for you. --Mark Thanks! Error: ERROR: depmodwrapper-cross-1.0-r0 do_patch: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: 0001: *** 0002:do_qa_patch(d) 0003: File: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/sources/poky/meta/classes-global/insane.bbclass', lineno: 1379, function: do_qa_patch 1375:elif os.path.exists(os.path.join(srcdir, "t")) and any(filename.endswith('.t') for filename in os.listdir(os.path.join(srcdir, 't'))): 1376:oe.qa.handle_error("unimplemented-ptest", "%s: perl Test:: based tests detected" % d.getVar('PN'), d) 1377: 1378:# Detect pytest-based tests *** 1379:elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): 1380:oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) 1381: 1382:# Detect meson-based tests 1383:elif os.path.exists(os.path.join(srcdir, "meson.build")) and match_line_in_files(srcdir, "**/meson.build", r'\s*test\s*\('): File: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/sources/poky/meta/classes-global/insane.bbclass', lineno: 1357, function: match_line_in_files 1353: ### 1354:def match_line_in_files(toplevel, filename_glob, line_regex): 1355:import pathlib 1356:toppath = pathlib.Path(toplevel) *** 1357:for entry in toppath.glob(filename_glob): 1358:try: 1359:with open(entry, 'r', encoding='utf-8', errors='ignore') as f: 1360:for line in f.readlines(): 1361:if re.match(line_regex, line): File: '/usr/lib/python3.10/pathlib.py', lineno: 1034, function: glob 1030:drv, root, pattern_parts = self._flavour.parse_parts((pattern,)) 1031:if drv or root: 1032:raise NotImplementedError("Non-relative patterns are unsupported") 1033:selector = _make_selector(tuple(pattern_parts), self._flavour) *** 1034:for p in selector.select_from(self): 1035:yield p 1036: 1037:def rglob(self, pattern): 1038:"""Recursively yield all existing files (of any kind, including File: '/usr/lib/python3.10/pathlib.py', lineno: 493, function: _select_from 0489:yielded = set() 0490:try: 0491:successor_select = self.successor._select_from 0492:for starting_point in self._iterate_directories(parent_path, is_dir, scandir): *** 0493:for p in successor_select(starting_point, is_dir, exists, scandir): 0494:if p not in yielded: 0495:yield p 0496:yielded.add(p) 0497:finally: File: '/usr/lib/python3.10/pathlib.py', lineno: 440, function: _select_from 0436:_Selector.__init__(self, child_parts, flavour) 0437: 0438:def _select_from(self, parent_path, is_dir, exists, scandir): 0439:try: *** 0440:with scandir(parent_path) as scandir_it: 0441:entries = list(scandir_it) 0442:for entry in entries: 0443:if self.dironly: 0444:try: Exception: FileNotFoundError: [Errno 2] No such file or directory: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/build/tmp/work/zynqmp_generic-xilinx-linux/depmodwrapper-cross/1.0/sstate-build-create_spdx' ERROR: Logfile of failure stored in: /scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/build/tmp/work/zynqmp_generic-xilinx-linux/depmodwrapper-cross/1.0/temp/log.do_patch.1966488 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View
[OE-core] Recent failures on master depmodwrapper-cross
Running a number of builds recently, a small number of them failed. Note we ran about 20 builds and only 3 failed like this, the others completed. I'm wondering if maybe there is a race condition with the new code? Error: ERROR: depmodwrapper-cross-1.0-r0 do_patch: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: 0001: *** 0002:do_qa_patch(d) 0003: File: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/sources/poky/meta/classes-global/insane.bbclass', lineno: 1379, function: do_qa_patch 1375:elif os.path.exists(os.path.join(srcdir, "t")) and any(filename.endswith('.t') for filename in os.listdir(os.path.join(srcdir, 't'))): 1376:oe.qa.handle_error("unimplemented-ptest", "%s: perl Test:: based tests detected" % d.getVar('PN'), d) 1377: 1378:# Detect pytest-based tests *** 1379:elif match_line_in_files(srcdir, "**/*.py", r'\s*(?:import\s*pytest|from\s*pytest)'): 1380:oe.qa.handle_error("unimplemented-ptest", "%s: pytest-based tests detected" % d.getVar('PN'), d) 1381: 1382:# Detect meson-based tests 1383:elif os.path.exists(os.path.join(srcdir, "meson.build")) and match_line_in_files(srcdir, "**/meson.build", r'\s*test\s*\('): File: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/sources/poky/meta/classes-global/insane.bbclass', lineno: 1357, function: match_line_in_files 1353: ### 1354:def match_line_in_files(toplevel, filename_glob, line_regex): 1355:import pathlib 1356:toppath = pathlib.Path(toplevel) *** 1357:for entry in toppath.glob(filename_glob): 1358:try: 1359:with open(entry, 'r', encoding='utf-8', errors='ignore') as f: 1360:for line in f.readlines(): 1361:if re.match(line_regex, line): File: '/usr/lib/python3.10/pathlib.py', lineno: 1034, function: glob 1030:drv, root, pattern_parts = self._flavour.parse_parts((pattern,)) 1031:if drv or root: 1032:raise NotImplementedError("Non-relative patterns are unsupported") 1033:selector = _make_selector(tuple(pattern_parts), self._flavour) *** 1034:for p in selector.select_from(self): 1035:yield p 1036: 1037:def rglob(self, pattern): 1038:"""Recursively yield all existing files (of any kind, including File: '/usr/lib/python3.10/pathlib.py', lineno: 493, function: _select_from 0489:yielded = set() 0490:try: 0491:successor_select = self.successor._select_from 0492:for starting_point in self._iterate_directories(parent_path, is_dir, scandir): *** 0493:for p in successor_select(starting_point, is_dir, exists, scandir): 0494:if p not in yielded: 0495:yield p 0496:yielded.add(p) 0497:finally: File: '/usr/lib/python3.10/pathlib.py', lineno: 440, function: _select_from 0436:_Selector.__init__(self, child_parts, flavour) 0437: 0438:def _select_from(self, parent_path, is_dir, exists, scandir): 0439:try: *** 0440:with scandir(parent_path) as scandir_it: 0441:entries = list(scandir_it) 0442:for entry in entries: 0443:if self.dironly: 0444:try: Exception: FileNotFoundError: [Errno 2] No such file or directory: '/scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/build/tmp/work/zynqmp_generic-xilinx-linux/depmodwrapper-cross/1.0/sstate-build-create_spdx' ERROR: Logfile of failure stored in: /scratch/jenkins-BUILDS-eSDK-dev_eSDK-eSDK-master-next-pipeline-8_ZynqMpDrFull/build/tmp/work/zynqmp_generic-xilinx-linux/depmodwrapper-cross/1.0/temp/log.do_patch.1966488 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#189254): https://lists.openembedded.org/g/openembedded-core/message/189254 Mute This Topic: https://lists.openembedded.org/mt/101987519/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [Openembedded-architecture] Security processes: YP needs
On 9/15/23 2:59 AM, Marta Rybczynska wrote: On Wed, Sep 13, 2023 at 6:28 PM Mark Hatle wrote: * Visibility of the security work of the YP There is much work on security in the YP, but it lacks visibility. Is there a common nexus for this work? eg. do most of the folks who are doing security work tend to congregate on the security sublist? Security means different things to different people. I.e. 1) Secure design - Is the system designed to have security services, if so are the defaults setup to both be appropriate and also functional? 2) Additional security software - i.e. meta-security, what additional software can be available to enhance security design/implementation of the system 3) Security (bug) response - This is where I see a lack of common nexus for work. We don't have a good place to discuss CVE specific information. Now the question really is, should we have a separate space. CVEs are just bugs. Bugs are usually worked on via the main mailing list. So that argument says no, we shouldn't have a special list. BUT the perception is CVEs are "special", so maybe a list specifically to discuss the ramifications of a CVE, fix/mitigation strategy or similar would make sense -- keeping actual bug fixes to the main mailing list? It might e interesting to have opinion on people who are submitting CVE fixes... Would keeping that discussion in a separate place be helpful? Ya, a security mailing list can be interesting for those types of discussions, but ultimately the code needs to go to the regular pull list -- which depending on the project (and level of discussions) it may make sense just to have those discussions on the regular list. It's tricky, and I'm not sure what the right answer is here. * SRTool We might decide to use it again. It allows one to do much but requires constant commitment. I think I passed over the wiki pages and presentations for SRTool once, a while ago. But I didn't pay much attention at the time because it wasn't clear *what it did*. After reviewing it again, I think it might be the kind of tooling I've been searching for to help my team coordinate our CVE response work. I'll test it out and see if it is something I can use/contribute towards. SR Tool requires constant feeding and maintenance from staff, at a minimum to do initial triage work. This means we need a small group of individuals who can take the new items (and changes to existing) and comment on a regular (daily) basis. This is the part we've not been terribly successful in the past. People are great at delivering patches, but trying to do the proactive (before cve-checker) evaluation of CVEs is an activity that often feels like busy work, so it's easy to get behind on and never catch up. I would love to see the project use SR Tool to manage CVE information, (bugzilla is where the bugs need to be managed and processed), as well as cve-checker to be able to continue to feed information or what we believe the current state of things are. This combination would give us per-emptive notification of new items (SR-Tool), and late validation of items (cve-checker) on the perceived state of things. SRTools code base (https://git.yoctoproject.org/srtool) has seen no changes for 4 years. If we decide to use, we'll also need to maintain the tool. Is Windriver going to update the fork? David (adding in copy), do you have any information? Otherwise we would need to maintain our version, and update to the code to take into account how the world have changed. For example, with the CVE v5 JSON, using the CVE database directly for the feed of new CVE list makes more sense than using NVD, for example. For the reason of performance and delay. When SRTool was developed, that data wasn't available. Last time I used it was almost exactly 4 years ago. The tool itself is pretty simple, it's the data import/export that is the complex bit(s). Maybe the lesson here isn't to use SR Tool, but take some of the concepts from it and maybe implement something ourselves (in the future). The key things are: 1) Automatic import from external CVE/Security sources (not all security items are CVEs) 2) A way to triage the information, and do LOCAL edits of the information - Way for the user to query what's new? - Way for the user to query what's changed since last time? - Way for the user to query other things... - Local edit could be YP 'status' - Local edit could add public OR private information about a given item - Local edits should be able to link a problem with a bug and release - Local edits should be able to TRACK progress of a bug - Local edits to CREATE security items not otherwise known (either YP specific, or based on bug reports, etc) - A way to temporarily set things as 'restricted', only for specific people to view until it
Re: [OE-core] [Openembedded-architecture] Security processes: YP needs
On 9/13/23 11:00 AM, Alex Stewart wrote: Thanks for driving this Marta. Internally and externally, it feels like we're just on the cusp of everyone *suddenly caring* about our security response strategy. So it's good to see that we're making moves in that direction. In general, this list looks complete to me. I'm primarily interested in the response coordination, triage, and tracking usecases. Those are the biggest pain points for my team, at the moment. And that is primarily driven by a lack of tooling. More responses inline. On 9/13/23 07:52, Marta Rybczynska via lists.openembedded.org wrote: [You don't often get email from rybczynska=gmail@lists.openembedded.org. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ] Hello, I've been working recently on collecting what works and what doesn't in YP security processes. The goal is to go forward and define an actionable strategy! Today, I'd like to share with you the summary of what I have heard as needs from several people (those in Cc:). I want the community to comment and tell us what you find important and what you'd like to see added or changed from this list. * CVEs: Visibility if YP is vulnerable or not People want to be able to check/look up a specific CVE; it might be a CVE unrelated to YP (eg. package not included, Windows issue). The cve-checker result is a part of the solution, but people also want to know which CVEs do not apply. I'm not sure I understand this usecase. Is there a reason those people can't/won't just lookup the CVE on the NIST site? Management goes to an engineer and says "Customer XYZ says we need a statement if CVE-2024-12345 affects us. Can you please comment?" Engineer goes to the Yocto Project "list", and looks the number up and doesn't find it. Does this mean we're affects? We're not affected? We were affected, but it's been fixed (if so when?), etc? So then they have to go to NIST, look at the CVE, find the information and do the evaluation if Yocto Project is affected. Instead what (I have observed) is that people who like to go to a single list (for Yocto Project) information, look up a CVE and get a clear statement of: This affects us, this does not affect us, we did not evaluate it or it was fixed by commit XYZ in branch Then if the item is "not evaluated", they can THEN got to NIST for their own evaluation. This saves a huge amount of time for people who are regularly requested to respond to these messages. * CVEs: synchronization of the work on fixes Currently, there is no synchronization; multiple parties might be working on the same fix while nobody is working on another. There might be duplication of work. Ross has https://wiki.yoctoproject.org/wiki/CVE_Status Has there been any discussion of adopting the OpenVEX document standard that the Chainguard guys are putting together? [1] It seems like the VEX use-cases align well with our needs around tracking and coordinating CVE response between YP member and individual developers. I've been considering it for my internal use for a while. And also considering replacing the existing cve_check output JSON with OpenVEX, once it has stabilized. [1] https://github.com/openvex/spec * Triaging of security issues Related to CVE fixes and includes issues reported directly to the YP. Some issues are more likely to be serious for embedded products (attack by network), so not all has the same priority. I'll note here that some of us are sinners and do actually support network-attached (and internet-attached) embedded devices. :) But the greater point of OS vendors being able to triage and assign vendor-specific severities to incoming issues is absolutely important to my use-cases. * Private security communication A way to send a notification of a non-public security issue. For researchers, other projects etc. The security alias exists, but only some people know about its existence. * Visibility of the security work of the YP There is much work on security in the YP, but it lacks visibility. Is there a common nexus for this work? eg. do most of the folks who are doing security work tend to congregate on the security sublist? Security means different things to different people. I.e. 1) Secure design - Is the system designed to have security services, if so are the defaults setup to both be appropriate and also functional? 2) Additional security software - i.e. meta-security, what additional software can be available to enhance security design/implementation of the system 3) Security (bug) response - This is where I see a lack of common nexus for work. We don't have a good place to discuss CVE specific information. Now the question really is, should we have a separate space. CVEs are just bugs. Bugs are usually worked on via the main mailing list. So that argument says no, we shouldn't have a special list. BUT the perception is CVEs are "special", so maybe a lis
[OE-core] [PATCH] tcf-agent: Update to 1.8.0 release
New 1.8 release of tcf-agent. Implements DWARF 5 support and various bug fixes. Changelog since last SRCREV: Releng: Upversion TCF to 1.8 Fixed possible SEGFAULT after error message queue overflow Fixed misspelling in a comment TCF Agent: update breakpoint error message Fixed regression: possible segfault in run_safe_events() Bug 581978 - TCF agent wrong handle the call frame debug info generated by LLVM 16 for RISC-V DWARF: a bit faster implementation of dio_ReadAddressX() Bug 581971 - Failed to handle loclist for DWARF 5 Fixed handling of situation when a context resumed or exited during breakpoint evaluation Bug 581799 - when loads .debug_info section from dwarf 5 file, the content of some part are zero Updated examples/daytime/readme.txt Signed-off-by: Mark Hatle --- meta/recipes-devtools/tcf-agent/tcf-agent_git.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/recipes-devtools/tcf-agent/tcf-agent_git.bb b/meta/recipes-devtools/tcf-agent/tcf-agent_git.bb index 9e77f12b53..7d151d4642 100644 --- a/meta/recipes-devtools/tcf-agent/tcf-agent_git.bb +++ b/meta/recipes-devtools/tcf-agent/tcf-agent_git.bb @@ -6,8 +6,8 @@ BUGTRACKER = "https://bugs.eclipse.org/bugs/"; LICENSE = "EPL-1.0 | EDL-1.0" LIC_FILES_CHKSUM = "file://edl-v10.html;md5=522a390a83dc186513f0500543ad3679" -SRCREV = "4a2c4baaccbc8c29ce0297705de9a4e096d57ce5" -PV = "1.7.0+git${SRCPV}" +SRCREV = "1f11747e83ebf4f53e8d17f430136f92ec378709" +PV = "1.8.0+git${SRCPV}" UPSTREAM_CHECK_GITTAGREGEX = "(?P(\d+(\.\d+)+))" SRC_URI = "git://git.eclipse.org/r/tcf/org.eclipse.tcf.agent.git;protocol=https;branch=master \ -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#185624): https://lists.openembedded.org/g/openembedded-core/message/185624 Mute This Topic: https://lists.openembedded.org/mt/100609626/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core][langdale] oeqa/qemurunner: update exception class for QMP API changes
From: Ross Burton Signed-off-by: Ross Burton Signed-off-by: Alexandre Belloni (cherry picked from commit c1841ab1e7b4e078cea77001e83e733764bb65ea) Signed-off-by: Mark Hatle --- meta/lib/oeqa/utils/qemurunner.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py index a455b3b389..ca75ce7e70 100644 --- a/meta/lib/oeqa/utils/qemurunner.py +++ b/meta/lib/oeqa/utils/qemurunner.py @@ -356,7 +356,7 @@ class QemuRunner: except OSError as msg: self.logger.warning("Failed to connect qemu monitor socket: %s File: %s" % (msg, msg.filename)) return False -except qmp.QMPConnectError as msg: +except qmp.legacy.QMPError as msg: self.logger.warning("Failed to communicate with qemu monitor: %s" % (msg)) return False finally: -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#183162): https://lists.openembedded.org/g/openembedded-core/message/183162 Mute This Topic: https://lists.openembedded.org/mt/99666400/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] Langdale?
I know there won't be any more releases of langdale (as well as automatic backports for security and other things.) However, I found a bug in the oeqa qemurunner (fixed in Mickledore). Would you be willing to take the backport fix (one line) or is the branch closed for further development? Thanks! --Mark -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#183153): https://lists.openembedded.org/g/openembedded-core/message/183153 Mute This Topic: https://lists.openembedded.org/mt/99648680/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH][pseudo] Move __*xstat* and __xmknod functions to new subport 'old__x'
Ok, so adding this patch will move from the legacy behavior to the new behavior (not using the legacy stat interfaces). This will only work on machines where the software has been built for only the newer exposed interfaces. I am surprised Debian 11 failed, as it should have been new enough the old interfaces don't exist. I don't know about debian11 or stream8. Can you modify the patch you applied and re-run your tests? Try this change: In the patch: meta/recipes-devtools/pseudo/files/new_glibc.patch, go down to the patch chunk for ports/linux/subports and then change: ++if ${CC} -c -o dummy.o dummy.c >/dev/null 2>&1; then ++echo "linux/old__x" ++fi to ++if true; then ++echo "linux/old__x" ++fi The change above SHOULD restore the old compatible interface behavior to pseudo. This will verify that the other changes did not introduce this fault. Assuming this works, the intergration patch that I'll work on will be different. The code check-in will be similar to the original path proposed (with the README updated). A local patch to the integration will exist to make the change listed above for x86, x86_64 and aarch64. This will restore legacy behavior on those architectures until we get to a point where the suspect interfaces are no longer being used. I will also be adding a second commit that defines wrappers for the obsolete interfaces, so if they ARE used they will trigger an abort, so we can more quickly detect the machines. Ultimately we want to stop wrapping the obsolete interfaces once the distributions are no longer using them, but we need a way to detect this. (I would like a switch to configure to enable the obsolete interfaces, but I've not figured out how to pass configure options into the subports function call. I'll continue to look into that, but it really shouldn't be necessary outside of the YP use-case where we want one pseudo to run across a variety of hosts.) --Mark On 6/1/23 9:20 AM, Alexandre Belloni via lists.openembedded.org wrote: On 01/06/2023 09:15:25-0500, Mark Hatle wrote: Did you or someone else manually add this patch for testing? I wasn't aware that it had gone in for any sort of testing eyt. I'd like to see the integration used so I can understand how the test was performed. Sure, I added the patch, here: https://git.yoctoproject.org/poky-contrib/commit/?h=abelloni/master-next&id=9b8298bf6dc6 --Mark On 6/1/23 5:43 AM, Alexandre Belloni wrote: Hello Mark, This causes failures on opensuse154 and debian11 workers. I've tried to get pseudo.log for the failures but they are not present on debian11 and are not interesting for opensuse154. debian11 failures look like that: https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/7184/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/7184/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/7152/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/7152/steps/13/logs/stdio opensuse: https://autobuilder.yoctoproject.org/typhoon/#/builders/120/builds/2819/steps/14/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/7597/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/7211/steps/13/logs/stdio stream8 is similar: https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2939/steps/13/logs/stdio I also suspect this causes: ERROR: cmdline-shebang-wrapper-test-1.0-r0 do_install: Wrapper permissions for /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/tmp/work/core2-64-poky-linux/cmdline-shebang-wrapper-test/1.0-r0/image/usr/bin/test.real not preserved. Found 600 but expected 400 ERROR: cmdline-shebang-wrapper-test-1.0-r0 do_install: ExecutionError('/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/tmp/work/core2-64-poky-linux/cmdline-shebang-wrapper-test/1.0-r0/temp/run.do_install.322689', 1, None, None) ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/tmp/work/core2-64-poky-linux/cmdline-shebang-wrapper-test/1.0-r0/temp/log.do_install.322689 NOTE: recipe cmdline-shebang-wrapper-test-1.0-r0: task do_install: Failed ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb:do_install) failed with exit code '1' On 30/05/2023 20:01:26-0700, Mark Hatle via lists.openembedded.org wrote: Changes to eliminate __*.c function usage were based on the patch: From: JiaLing Zhang Subject: [OE-core] [PATCH v4] Fixes pseudo build in loongarch64 Fixes [YOCTO #15110] Some functions used in the project have been removed from glibc. After the removal of these functions,
Re: [OE-core] [PATCH][pseudo] Move __*xstat* and __xmknod functions to new subport 'old__x'
Did you or someone else manually add this patch for testing? I wasn't aware that it had gone in for any sort of testing eyt. I'd like to see the integration used so I can understand how the test was performed. --Mark On 6/1/23 5:43 AM, Alexandre Belloni wrote: Hello Mark, This causes failures on opensuse154 and debian11 workers. I've tried to get pseudo.log for the failures but they are not present on debian11 and are not interesting for opensuse154. debian11 failures look like that: https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/7184/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/42/builds/7184/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/7152/steps/12/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/7152/steps/13/logs/stdio opensuse: https://autobuilder.yoctoproject.org/typhoon/#/builders/120/builds/2819/steps/14/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/20/builds/7597/steps/13/logs/stdio https://autobuilder.yoctoproject.org/typhoon/#/builders/53/builds/7211/steps/13/logs/stdio stream8 is similar: https://autobuilder.yoctoproject.org/typhoon/#/builders/117/builds/2939/steps/13/logs/stdio I also suspect this causes: ERROR: cmdline-shebang-wrapper-test-1.0-r0 do_install: Wrapper permissions for /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/tmp/work/core2-64-poky-linux/cmdline-shebang-wrapper-test/1.0-r0/image/usr/bin/test.real not preserved. Found 600 but expected 400 ERROR: cmdline-shebang-wrapper-test-1.0-r0 do_install: ExecutionError('/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/tmp/work/core2-64-poky-linux/cmdline-shebang-wrapper-test/1.0-r0/temp/run.do_install.322689', 1, None, None) ERROR: Logfile of failure stored in: /home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/tmp/work/core2-64-poky-linux/cmdline-shebang-wrapper-test/1.0-r0/temp/log.do_install.322689 NOTE: recipe cmdline-shebang-wrapper-test-1.0-r0: task do_install: Failed ERROR: Task (/home/pokybuild/yocto-worker/oe-selftest-fedora/build/build-st-1782300/meta-selftest/recipes-test/wrapper/cmdline-shebang-wrapper-test.bb:do_install) failed with exit code '1' On 30/05/2023 20:01:26-0700, Mark Hatle via lists.openembedded.org wrote: Changes to eliminate __*.c function usage were based on the patch: From: JiaLing Zhang Subject: [OE-core] [PATCH v4] Fixes pseudo build in loongarch64 Fixes [YOCTO #15110] Some functions used in the project have been removed from glibc. After the removal of these functions, the architecture in glibc will not include the removed functions. This patch resolves the usage and compilation issues on the loongarch64 architecture Signed-off-by: JiaLing Zhang This code is NOT loongarch64 specific, but implements support for newer glibc where the __*x*stat and __xmknod* functions are no longer present in headers as of roughly glibc 2.33. The functions, on x86, x86_64 and aarch64 may still be present for compatibility but new software should no longer be using it. Pseudo can likely change it's default behavior unless support for really old hosts is still desired. Signed-off-by: Mark Hatle --- ports/linux/guts/fopen64.c | 4 +- ports/linux/guts/freopen64.c | 4 +- ports/linux/guts/fstat.c | 8 ++- ports/linux/guts/fstat64.c | 14 +++- ports/linux/guts/fstatat.c | 9 ++- ports/linux/guts/fstatat64.c | 42 +++- ports/linux/guts/lstat.c | 2 +- ports/linux/guts/lstat64.c | 2 +- ports/linux/guts/mknod.c | 2 +- ports/linux/guts/mknodat.c | 71 +++- ports/linux/guts/mkostemp64.c| 2 +- ports/linux/guts/openat.c| 18 ++--- ports/linux/guts/stat.c | 2 +- ports/linux/guts/stat64.c| 2 +- ports/linux/old__x/README| 28 ports/linux/{ => old__x}/guts/__fxstat.c | 0 ports/linux/{ => old__x}/guts/__fxstat64.c | 0 ports/linux/{ => old__x}/guts/__fxstatat.c | 0 ports/linux/{ => old__x}/guts/__fxstatat64.c | 0 ports/linux/{ => old__x}/guts/__lxstat.c | 0 ports/linux/{ => old__x}/guts/__lxstat64.c | 0 ports/linux/{ => old__x}/guts/__xmknod.c | 0 ports/linux/{ => old__x}/guts/__xmknodat.c | 0 ports/linux/{ => old__x}/guts/__xstat.c | 0 ports/linux/{ => old__x}/guts/__xstat64.c| 0 ports/linux/old__x/guts/fstat.c | 15 + ports/linux/old__x/guts/fstat64.c| 15 + ports/linux/old__x/guts/fstatat.c| 15 + ports/linux/old__x/guts/fstatat64.c | 15 +
Re: [OE-core] [PATCH][pseudo] Move __*xstat* and __xmknod functions to new subport 'old__x'
On 5/30/23 10:18 PM, Seebs wrote: On Tue, 30 May 2023 19:54:41 -0700 Mark Hatle wrote: - int existed = (real___xstat64(_STAT_VER, path, &buf) != -1); + int existed = (base_stat64(path, &buf) != -1); Honestly, with the benefit of hindsight, I actually can't even think why I ever thought I should be using those directly instead of through the base_foo wrappers. Probably for a reason. ... Just in case, definitely don't merge this before testing it in case there's some insane reason for which the base_* wrappers didn't work here. Ya, it's going to require some soak time in the master-next and various builds I suspect. ... diff --git a/ports/linux/old__x/README b/ports/linux/old__x/README new file mode 100644 index 000..c94413f --- /dev/null +++ b/ports/linux/old__x/README @@ -0,0 +1,28 @@ +Older glibcs contain stat functions such as: + +__fxstat +__fxstatat +__lxstat +__xstat + +__fxstat64 +__fxstatat64 +__lxstat64 +__xstat64 + +The format of these functions use the _STAT_VER defintion. New glibc no +longer define or utilize these functions, so neither can we. + +We only use this subport when the functions are present, this is checked +by with the existence of _STAT_VER. "by with the existence"? Looks like a typo. Ya, I forgot to clean that up, how does this look as a replacement: Older glibcs contain xstat functions such as: __fxstat __fxstatat __lxstat __xstat __fxstat64 __fxstatat64 __lxstat64 __xstat64 These functions are defined when the _STAT_VER is also defined. Similarly these xmknod functions are also only defined in older glibcs: __xmknod __xmknodat These functions are defind when the _MKNOD_VER is also defined. This subport is automatically enabled with BOTH _STAT_VER and _MKNOD_VER are present. Alternatively if the user force enables subport, it should work as long as the symbols for these functions are still available. Anyway, looks reasonable to me, thanks! -s -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#181986): https://lists.openembedded.org/g/openembedded-core/message/181986 Mute This Topic: https://lists.openembedded.org/mt/99234918/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH][pseudo] Move __*xstat* and __xmknod functions to new subport 'old__x'
This code only switched the usage of the removed __ functions based on availability of _STAT_VER and _MKNOD_VER in the ports/linux/subport file. If you change the conditional to 'if true ; then' the system will enable the older style code unconditionally. I did verify that appears to be working for me. If anyone has a good idea how to adjust the configure script to pass the data to the subport file, that would be ideal. Default behavior of checking the system, optional behaving of an --enable-compat-xfunctions -- or something like that. As for the loongarch64, this code should work there as it's now written. I do not have a way to test this, so I'm hoping JiaLing Zhang will be able to build and test this. All of my testing was accomplished using the 'make test' target. More extensive testing will be needed before we can trust this for OE, but it's probably closer to something that does what we need. (There is still more duplication then I would like.. hopefully reviewers will be able to suggest something.) --Mark On 5/30/23 10:01 PM, Mark Hatle via lists.openembedded.org wrote: Changes to eliminate __*.c function usage were based on the patch: From: JiaLing Zhang Subject: [OE-core] [PATCH v4] Fixes pseudo build in loongarch64 Fixes [YOCTO #15110] Some functions used in the project have been removed from glibc. After the removal of these functions, the architecture in glibc will not include the removed functions. This patch resolves the usage and compilation issues on the loongarch64 architecture Signed-off-by: JiaLing Zhang This code is NOT loongarch64 specific, but implements support for newer glibc where the __*x*stat and __xmknod* functions are no longer present in headers as of roughly glibc 2.33. The functions, on x86, x86_64 and aarch64 may still be present for compatibility but new software should no longer be using it. Pseudo can likely change it's default behavior unless support for really old hosts is still desired. Signed-off-by: Mark Hatle --- ports/linux/guts/fopen64.c | 4 +- ports/linux/guts/freopen64.c | 4 +- ports/linux/guts/fstat.c | 8 ++- ports/linux/guts/fstat64.c | 14 +++- ports/linux/guts/fstatat.c | 9 ++- ports/linux/guts/fstatat64.c | 42 +++- ports/linux/guts/lstat.c | 2 +- ports/linux/guts/lstat64.c | 2 +- ports/linux/guts/mknod.c | 2 +- ports/linux/guts/mknodat.c | 71 +++- ports/linux/guts/mkostemp64.c| 2 +- ports/linux/guts/openat.c| 18 ++--- ports/linux/guts/stat.c | 2 +- ports/linux/guts/stat64.c| 2 +- ports/linux/old__x/README| 28 ports/linux/{ => old__x}/guts/__fxstat.c | 0 ports/linux/{ => old__x}/guts/__fxstat64.c | 0 ports/linux/{ => old__x}/guts/__fxstatat.c | 0 ports/linux/{ => old__x}/guts/__fxstatat64.c | 0 ports/linux/{ => old__x}/guts/__lxstat.c | 0 ports/linux/{ => old__x}/guts/__lxstat64.c | 0 ports/linux/{ => old__x}/guts/__xmknod.c | 0 ports/linux/{ => old__x}/guts/__xmknodat.c | 0 ports/linux/{ => old__x}/guts/__xstat.c | 0 ports/linux/{ => old__x}/guts/__xstat64.c| 0 ports/linux/old__x/guts/fstat.c | 15 + ports/linux/old__x/guts/fstat64.c| 15 + ports/linux/old__x/guts/fstatat.c| 15 + ports/linux/old__x/guts/fstatat64.c | 15 + ports/linux/old__x/guts/lstat.c | 15 + ports/linux/old__x/guts/lstat64.c| 15 + ports/linux/old__x/guts/mknod.c | 15 + ports/linux/old__x/guts/mknodat.c| 15 + ports/linux/old__x/guts/mkostemp64.c | 53 +++ ports/linux/old__x/portdefs.h| 40 +++ ports/linux/old__x/pseudo_wrappers.c | 48 + ports/linux/old__x/wrapfuncs.in | 18 + ports/linux/portdefs.h | 19 -- ports/linux/pseudo_wrappers.c| 43 ports/linux/subports | 15 + ports/linux/wrapfuncs.in | 26 +++ pseudo_client.h | 38 +-- 42 files changed, 520 insertions(+), 110 deletions(-) create mode 100644 ports/linux/old__x/README rename ports/linux/{ => old__x}/guts/__fxstat.c (100%) rename ports/linux/{ => old__x}/guts/__fxstat64.c (100%) rename ports/linux/{ => old__x}/guts/__fxstatat.c (100%) rename ports/linux/{ => old__x}/guts/__fxstatat64.c (100%) rename ports/linux/{ => old__x}/guts/__lxstat.c (100%) rename ports/linux/{ => old__x}/guts/__l
[OE-core] [PATCH][pseudo] Move __*xstat* and __xmknod functions to new subport 'old__x'
Changes to eliminate __*.c function usage were based on the patch: From: JiaLing Zhang Subject: [OE-core] [PATCH v4] Fixes pseudo build in loongarch64 Fixes [YOCTO #15110] Some functions used in the project have been removed from glibc. After the removal of these functions, the architecture in glibc will not include the removed functions. This patch resolves the usage and compilation issues on the loongarch64 architecture Signed-off-by: JiaLing Zhang This code is NOT loongarch64 specific, but implements support for newer glibc where the __*x*stat and __xmknod* functions are no longer present in headers as of roughly glibc 2.33. The functions, on x86, x86_64 and aarch64 may still be present for compatibility but new software should no longer be using it. Pseudo can likely change it's default behavior unless support for really old hosts is still desired. Signed-off-by: Mark Hatle --- ports/linux/guts/fopen64.c | 4 +- ports/linux/guts/freopen64.c | 4 +- ports/linux/guts/fstat.c | 8 ++- ports/linux/guts/fstat64.c | 14 +++- ports/linux/guts/fstatat.c | 9 ++- ports/linux/guts/fstatat64.c | 42 +++- ports/linux/guts/lstat.c | 2 +- ports/linux/guts/lstat64.c | 2 +- ports/linux/guts/mknod.c | 2 +- ports/linux/guts/mknodat.c | 71 +++- ports/linux/guts/mkostemp64.c| 2 +- ports/linux/guts/openat.c| 18 ++--- ports/linux/guts/stat.c | 2 +- ports/linux/guts/stat64.c| 2 +- ports/linux/old__x/README| 28 ports/linux/{ => old__x}/guts/__fxstat.c | 0 ports/linux/{ => old__x}/guts/__fxstat64.c | 0 ports/linux/{ => old__x}/guts/__fxstatat.c | 0 ports/linux/{ => old__x}/guts/__fxstatat64.c | 0 ports/linux/{ => old__x}/guts/__lxstat.c | 0 ports/linux/{ => old__x}/guts/__lxstat64.c | 0 ports/linux/{ => old__x}/guts/__xmknod.c | 0 ports/linux/{ => old__x}/guts/__xmknodat.c | 0 ports/linux/{ => old__x}/guts/__xstat.c | 0 ports/linux/{ => old__x}/guts/__xstat64.c| 0 ports/linux/old__x/guts/fstat.c | 15 + ports/linux/old__x/guts/fstat64.c| 15 + ports/linux/old__x/guts/fstatat.c| 15 + ports/linux/old__x/guts/fstatat64.c | 15 + ports/linux/old__x/guts/lstat.c | 15 + ports/linux/old__x/guts/lstat64.c| 15 + ports/linux/old__x/guts/mknod.c | 15 + ports/linux/old__x/guts/mknodat.c| 15 + ports/linux/old__x/guts/mkostemp64.c | 53 +++ ports/linux/old__x/portdefs.h| 40 +++ ports/linux/old__x/pseudo_wrappers.c | 48 + ports/linux/old__x/wrapfuncs.in | 18 + ports/linux/portdefs.h | 19 -- ports/linux/pseudo_wrappers.c| 43 ports/linux/subports | 15 + ports/linux/wrapfuncs.in | 26 +++ pseudo_client.h | 38 +-- 42 files changed, 520 insertions(+), 110 deletions(-) create mode 100644 ports/linux/old__x/README rename ports/linux/{ => old__x}/guts/__fxstat.c (100%) rename ports/linux/{ => old__x}/guts/__fxstat64.c (100%) rename ports/linux/{ => old__x}/guts/__fxstatat.c (100%) rename ports/linux/{ => old__x}/guts/__fxstatat64.c (100%) rename ports/linux/{ => old__x}/guts/__lxstat.c (100%) rename ports/linux/{ => old__x}/guts/__lxstat64.c (100%) rename ports/linux/{ => old__x}/guts/__xmknod.c (100%) rename ports/linux/{ => old__x}/guts/__xmknodat.c (100%) rename ports/linux/{ => old__x}/guts/__xstat.c (100%) rename ports/linux/{ => old__x}/guts/__xstat64.c (100%) create mode 100644 ports/linux/old__x/guts/fstat.c create mode 100644 ports/linux/old__x/guts/fstat64.c create mode 100644 ports/linux/old__x/guts/fstatat.c create mode 100644 ports/linux/old__x/guts/fstatat64.c create mode 100644 ports/linux/old__x/guts/lstat.c create mode 100644 ports/linux/old__x/guts/lstat64.c create mode 100644 ports/linux/old__x/guts/mknod.c create mode 100644 ports/linux/old__x/guts/mknodat.c create mode 100644 ports/linux/old__x/guts/mkostemp64.c create mode 100644 ports/linux/old__x/portdefs.h create mode 100644 ports/linux/old__x/pseudo_wrappers.c create mode 100644 ports/linux/old__x/wrapfuncs.in diff --git a/ports/linux/guts/fopen64.c b/ports/linux/guts/fopen64.c index e76da69..33ccd3a 100644 --- a/ports/linux/guts/fopen64.c +++ b/ports/linux/guts/fopen64.c @@ -11,7 +11,7 @@ struct stat64 buf; int save_errno; - int existed = (real___xstat64(_STAT_VER, path, &buf) != -1)
Re: [OE-core] [PATCH v4] Fixes pseudo build in loongarch64
I am working on changing the v4 into a more generic implementation based on Seebs' and others comments. I'll likely be sending something to the list as an RFC later today or tomorrow. --Mark On 5/30/23 11:43 AM, Seebs wrote: On Tue, 30 May 2023 19:33:03 +0800 zhangjial...@loongson.cn wrote: +#ifdef __loongarch64 No. Use the configuration mechanisms or whatever to determine the *actual distinction you care about*, which is "does glibc have __xstat64", give that a name, and use the name. Some day, someone else will make a glibc that doesn't have __xstat64. The change to adopt that should be one line in configure or the like, not hundreds of changes scattered throughout the code. +int existed = (real_stat64(path, &buf) != -1); +#else int existed = (real___xstat64(_STAT_VER, path, &buf) != -1); - +#endif This looks suspiciously similar to the logic you introduce elsewhere for base_stat, base_fstat, etcetera. Reproducing that up here: +#ifdef __loongarch64 +#define base_fstatat(dirfd, path, buf, flags) real_fstatat64( dirfd, path, buf, flags) +#else #define base_fstatat(dirfd, path, buf, flags) real___fxstatat64(_STAT_VER, dirfd, path, buf, flags) +#endif Again, shouldn't be using a specific arch flag, should be using a symbolic flag, but: Given this, maybe we should just be using base_fstatat, base_stat, etcetera, in these lines, rather than calling a specific real___foo function directly. Or alternatively, we should be using pseudo_stat, etcetera, which also exist, and I honestly don't remember why there are two sets of those. So the change should look something like - int existed = ((real___xstat64(_STAT_VER, path, &buf) != -1); + int existed = ((base_stat(path, &buf) != -1); with suitable updates to the way we define the base_foo functions, making them contingent on something like HAVE_XSTAT. This would cover the vast majority of these changes, but not all of them. --- a/ports/linux/guts/fstat.c +++ b/ports/linux/guts/fstat.c @@ -7,8 +7,17 @@ * int fstat(int fd, struct stat *buf) *int rc = -1; */ - +#ifdef __loongarch64 + struct stat64 buf64; +/* populate buffer with complete data */ + real_fstat(fd, buf); +/* obtain fake data */ + rc = wrap_fstat64(fd, &buf64); +/* overwrite */ + pseudo_stat32_from64(buf, &buf64); +#else rc = wrap___fxstat(_STAT_VER, fd, buf); +#endif /* return rc; * } Here, we really do have a meaningful change; in the standard linux port, we know that we can just forward fstat to __fxstat. Probably the best model for the canonical way to fix this is to look at the oldclone/newclone subports. When you have changes this substantive, we don't want to do them with large inline #ifdefs; we want to describe these as distinct subports, which have a different set of functions. So in the hypothetical new "xstat" port, we'd have the existing fstat and __fxstat wrappers, and in the new "noxstat" port, we'd just have an fstat wrapper that uses the same logic. diff --git a/templates/wrapfuncs.c b/templates/wrapfuncs.c index 93bb671..14a42e2 100644 --- a/templates/wrapfuncs.c +++ b/templates/wrapfuncs.c @@ -13,7 +13,9 @@ * script if you want to modify this. */ @body +#ifndef __loongarch64 static ${type} (*real_${name})(${decl_args}) = ${real_init}; +#endif ${maybe_skip} I don't understand how this part can work at all; I was under the impression that we actually did in fact use these real_foo objects, and this seems to remove them all, so I don't even get how this is compiling or running, because after this, there's no real_foo function pointers at all? Unfortunately, it turns out bitrot has taken my laptop and I can't build pseudo on it natively because _STAT_VER no longer exists. So I think the hypothetical distant future in which we conceivably care about a system on which the real___xstatat(_STAT_VER, ...) calls fail and isn't __longarch64 actually happened over a year ago and we just didn't notice. "Oops." -s -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#181957): https://lists.openembedded.org/g/openembedded-core/message/181957 Mute This Topic: https://lists.openembedded.org/mt/99217464/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] Initramfs recipes - How to set IMAGE_FSTYPES
A little background, looking at oe-core and poky for background, I see the initramfs images contain: IMAGE_FSTYPES = "${INITRAMFS_FSTYPES}" I see this in both oe-core meta/recipes-core/images/core-image-minimal-initramfs.bb and core-image-tiny-initramfs.bb. The "meta-security" layer also seems to the do the same in it's 'dm-verify-image-iniramfs.bb' recipe as well. However, when I look at meta-openembedded/meta-initramfs (initramfs-debug-image and initramfs-kexecboot-image), I see: # Some BSPs use IMAGE_FSTYPES_ which would override # an assignment to IMAGE_FSTYPES so we need anon python python() { d.setVar("IMAGE_FSTYPES", d.getVar("INITRAMFS_FSTYPES")) } This is the exact issue I'm trying to understand. (Ignoring the comment above uses the older style override syntax), the issue I have is someone is setting: IMAGE_FSTYPES: = "something" INITRAMFS_FSTYPES: = "something else" Then the initramfs recipes in oe-core are failing to set IMAGE_FSTYPES to INITRAMFS_FSTYPES due to the IMAGE_FSTYPES override, which causes the types to not be correct. My question then is, is setting IMAGE_FSTYPES: valid, if not should be add a sanity check to tell people to NOT do it? If it is valid to do, is the workaround/fix in meta-openembedded the right solution? If using the python approach is correct, then I'm happy to generate patches to oe-core but I'm just not sure what the expected usage is. Thanks! --Mark -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#181028): https://lists.openembedded.org/g/openembedded-core/message/181028 Mute This Topic: https://lists.openembedded.org/mt/98769851/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master][mickledore][PATCH v2] sanity.bbclass: Update minimum gcc version to 8.0
From: Mark Hatle With a gcc older then 8.0, mesa-native will fail to build with the error: sorry, unimplemented: non-trivial designated initializers not supported According to https://docs.mesa3d.org/install.html?highlight=gcc+version#compile the required minimum compiler version is now GCC 8.0. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- v2: Fixed typo in commit message. meta -> mesa meta/classes-global/sanity.bbclass | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/classes-global/sanity.bbclass b/meta/classes-global/sanity.bbclass index 2d1ff7072c..abb52fbe21 100644 --- a/meta/classes-global/sanity.bbclass +++ b/meta/classes-global/sanity.bbclass @@ -475,7 +475,7 @@ def check_wsl(d): bb.warn("You are running bitbake under WSLv2, this works properly but you should optimize your VHDX file eventually to avoid running out of storage space") return None -# Require at least gcc version 7.5. +# Require at least gcc version 8.0 # # This can be fixed on CentOS-7 with devtoolset-6+ # https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/ @@ -488,8 +488,8 @@ def check_gcc_version(sanity_data): build_cc, version = oe.utils.get_host_compiler_version(sanity_data) if build_cc.strip() == "gcc": -if bb.utils.vercmp_string_op(version, "7.5", "<"): -return "Your version of gcc is older than 7.5 and will break builds. Please install a newer version of gcc (you could use the project's buildtools-extended-tarball or use scripts/install-buildtools).\n" +if bb.utils.vercmp_string_op(version, "8.0", "<"): +return "Your version of gcc is older than 8.0 and will break builds. Please install a newer version of gcc (you could use the project's buildtools-extended-tarball or use scripts/install-buildtools).\n" return None # Tar version 1.24 and onwards handle overwriting symlinks correctly -- 2.17.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#179695): https://lists.openembedded.org/g/openembedded-core/message/179695 Mute This Topic: https://lists.openembedded.org/mt/98061607/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master][mickledore][PATCH] sanity.bbclass: Update minimum gcc version to 8.0
From: Mark Hatle With a gcc older then 8.0, meta-native will fail to build with the error: sorry, unimplemented: non-trivial designated initializers not supported According to https://docs.mesa3d.org/install.html?highlight=gcc+version#compile the required minimum compiler version is now GCC 8.0. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/classes-global/sanity.bbclass | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/meta/classes-global/sanity.bbclass b/meta/classes-global/sanity.bbclass index 2d1ff7072c..abb52fbe21 100644 --- a/meta/classes-global/sanity.bbclass +++ b/meta/classes-global/sanity.bbclass @@ -475,7 +475,7 @@ def check_wsl(d): bb.warn("You are running bitbake under WSLv2, this works properly but you should optimize your VHDX file eventually to avoid running out of storage space") return None -# Require at least gcc version 7.5. +# Require at least gcc version 8.0 # # This can be fixed on CentOS-7 with devtoolset-6+ # https://www.softwarecollections.org/en/scls/rhscl/devtoolset-6/ @@ -488,8 +488,8 @@ def check_gcc_version(sanity_data): build_cc, version = oe.utils.get_host_compiler_version(sanity_data) if build_cc.strip() == "gcc": -if bb.utils.vercmp_string_op(version, "7.5", "<"): -return "Your version of gcc is older than 7.5 and will break builds. Please install a newer version of gcc (you could use the project's buildtools-extended-tarball or use scripts/install-buildtools).\n" +if bb.utils.vercmp_string_op(version, "8.0", "<"): +return "Your version of gcc is older than 8.0 and will break builds. Please install a newer version of gcc (you could use the project's buildtools-extended-tarball or use scripts/install-buildtools).\n" return None # Tar version 1.24 and onwards handle overwriting symlinks correctly -- 2.17.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#179672): https://lists.openembedded.org/g/openembedded-core/message/179672 Mute This Topic: https://lists.openembedded.org/mt/98052854/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] mesa-native fails on master
I'm fine with saying either Ubuntu 18.04 is no longer 'supported', or running some kind of check that says gcc 7.5.0 is too old.. But I'm not sure what minimum (host) compiler version would be needed to avoid this error. I'm happy to work up a patch to adjust the minimum gcc version, if someone can give me a bit of a clue what the appropriate minimum should be. The current check (in sanity.bbclass) looks for 7.5 or newer. --Mark On 4/3/23 9:48 PM, Khem Raj wrote: On Mon, Apr 3, 2023 at 7:39 PM Chen Qi wrote: I just met the same issue. My host is ubuntu18, gcc version is 7.5.0. Regards, Qi -Original Message- From: openembedded-core@lists.openembedded.org On Behalf Of Mark Hatle Sent: Tuesday, April 4, 2023 10:25 AM To: Patches and discussions about the oe-core layer Subject: [OE-core] mesa-native fails on master It's been a few weeks since I last built master, but starting with today's pull I'm getting errors building mesa-native -- which pretty much kills the whole build. I'm on an Ubuntu 18.04 host, and have the 'buildtools-tarball' from the last 4.1.3 loaded so I can get to the minimum version of python to run a build. (I tried with buildtools-tarball-extended and that DOES appear to work. If this is expected, you can ignore the rest of this. But I expected the system to complain before trying to build everything if the compiler wasn't new enough.) The error is: brw_simd_selection.cpp:205:7: sorry, unimplemented: non-trivial designated initializers not supported The host gcc is 7.5.0 What is strange is I had 2 successful builds, and then it started failing. All fresh builds. I've no idea why one was successful and the others failed. I'm wondering if maybe there is some sort of race in mesa, but lowered my parallel build (-j) to 1 and I'm still getting it. You need a newer gcc compiler > 7.x I just abandoned using ubuntu 18.04 see - https://lists.openembedded.org/g/openembedded-core/message/178783?p=%2C%2C%2C20%2C0%2C0%2C0%3A%3Acreated%2C0%2Cnon-trivial+designated+initializers+not+supported%2C20%2C2%2C0%2C97647023 Reproducer: git clone https://git.yoctoproject.org/poky cd poky wget http://downloads.yoctoproject.org/releases/yocto/yocto-4.1.3/buildtools/x86_64-buildtools-nativesdk-standalone-4.1.3.sh bash x86_64-buildtools-nativesdk-standalone-4.1.3.sh -d buildtools -y . ./buildtools/environment-... . ./oe-init-build-env bitbake mesa-native -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#179670): https://lists.openembedded.org/g/openembedded-core/message/179670 Mute This Topic: https://lists.openembedded.org/mt/98052337/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] mesa-native fails on master
It's been a few weeks since I last built master, but starting with today's pull I'm getting errors building mesa-native -- which pretty much kills the whole build. I'm on an Ubuntu 18.04 host, and have the 'buildtools-tarball' from the last 4.1.3 loaded so I can get to the minimum version of python to run a build. (I tried with buildtools-tarball-extended and that DOES appear to work. If this is expected, you can ignore the rest of this. But I expected the system to complain before trying to build everything if the compiler wasn't new enough.) The error is: brw_simd_selection.cpp:205:7: sorry, unimplemented: non-trivial designated initializers not supported The host gcc is 7.5.0 What is strange is I had 2 successful builds, and then it started failing. All fresh builds. I've no idea why one was successful and the others failed. I'm wondering if maybe there is some sort of race in mesa, but lowered my parallel build (-j) to 1 and I'm still getting it. Reproducer: git clone https://git.yoctoproject.org/poky cd poky wget http://downloads.yoctoproject.org/releases/yocto/yocto-4.1.3/buildtools/x86_64-buildtools-nativesdk-standalone-4.1.3.sh bash x86_64-buildtools-nativesdk-standalone-4.1.3.sh -d buildtools -y . ./buildtools/environment-... . ./oe-init-build-env bitbake mesa-native -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#179657): https://lists.openembedded.org/g/openembedded-core/message/179657 Mute This Topic: https://lists.openembedded.org/mt/98052337/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH] tcf-agent: Update to current version
From: Mark Hatle While there has not been an official release in some time, the latest version fixes a large number of bugs and adds support for Dwarf 5, among other things. (Dwarf 5 is default in gcc 12 and newer.) Short change log since the last version: TCF Agent: Narrow down LoadLibrary search scope TCF Agent: faster breakpoint planting Bug 581214 - Expression evaluation fails for a data member that is defined in a structure, union or class that is of type DW_AT_data_bit_offset attribute (DWARF5) TCF Agent: reduced memory footprint TCF Agent: fixed build error on CygWin: CYGWIN_VERSION_CYGWIN_CONV undefined TCF Agent: workaround for GCC compilation error: function may return address of local variable Bug 581034 - "long long int" data type can't be recognized from dwarf5 debug info generated by llvm15 TCF Agent: Removal of unused variable causing clang-15 build error TCF Agent: fixed: malformed HTTP request can crash the agent TCF Agent: fixed assertion failure in the Breakpoints service TCF Agent: GDB Remote Serial Protocol: fixed handling of process ID 0 Bug 580644 - steps into the function of shared library does not work on PPC TCF Agent: add PowerPC 32-bit and big-endian builds in the Dockerfile TCF Agent: check glibc version when calling pthread_setname_np() Bug 580600 - -Wmisleading-indentation issue for gcc version less than 6.0 Bug 580489 - failed to handle .debug_rnglists section sometimes Bug 580450 - failed to read the full compilation unit tag in dwarf 5 when customized sections Bug 580414 - failed to handle debug sections for DWARF 5 Bug 580326 - Can not display the type of global variable defined in a sharedlib Fixed text formatting in Makefile.inc Bug 580279 - VERSION grep in agent/Makefile.inc need update for vxWorks Bug 580089 - pid2id() failed to get a right id TCF Agent: improved comments TCF Agent: a few more asserts TCF Agent: improved X86 disassembler Bug 580002 - [tcf-dev] Speedup compute_reverse_lookup_indices Bug 579989 - compile error by vs2008 on windows TCF Agent: improved X86 disassembler Bug 579947 - NULL pointer caused the tcf-server crashed when load ELF file that compiled by LLVM 14 with dwarf-5 TCF Agent: fixed Coverity warning TCF Agent: fixed Coverity warning TCF Agent: new function in HTTP server API: closed() call-back Merge "TCF Agent: Fix potential memory/resource leaks" TCF Agent: Fix potential memory/resource leaks TCF Agent: Fix unsafe strcpy TCF Agent: Respect certain ENABLE_* macros Bug 579412 - incorrect or missing copyright information TCF Agent: Lazy PC initialization Bug 579362 - the process IDs should be pid_t instead of UINT32 in waitpid.c and waitpid.h files Bug 579378 - update copyright year to 2022 Bug 579274 - unknown option -Wmisleading-indentation issue for gcc version less than 6.0 TCF Agent: ARM v8 stack crawl: fixed handling of ERET instruction TCF Agent: fixed handling of line info file names when file contains a mix DWARF 5 and DWARF 3 TCF Tests: fixed incorrect error reports in DWARF reader test TCF Agent: disabled bogus warnings misleading-indentation TCF Agent: fixed handling of line info file names in DWARF 5 TCF Agent: Expressions: fixed handling of a static field of a struct stored in a register TCF Agent: fixed error when building with c++-11 TCF Agent: DWARF 5: improved handling of location expressions TCF Agent: DWARF reader: fixed regression - "FORM_FLAG expected" error TCF Agent: DWARF reader: new object flag DOIF_inlined TCF Tests: improved tests of var declarations TCF Agent: fixed regression: memory corruption in the symbols proxy TCF Agent: fixed issues with handling GCC-11 debug info TCF Agent: fixed warning: ISO C90 forbids mixed declarations and code TCF Agent: Fix stack frame cleanup on error TCF Agent: Add missing parentheses Bug 578201 - Sometimes variables show as "N/A - Cannot read target memory. Input/output error" Bug 577936 - dprintf not work as expected TCF Agent: lazy initialization in rand32() TCF Agent: new API function: rand32() TCF Agent: fixed possible buffer overflow when calling fscanf() Bug 577174 - Sometimes variables show as "N/A - Value of register is unknown in the selected frame" Bug 577064 - Union type variables don’t show correctly if based on a register TCF Agent: fixed: tmp_vprintf and loc_vprintf can segfault on Linux Bug 577001 - DW_AT_high_pc with DW_FORM_udata form not handled TCF Agent: fixed pthread_cond_timedwait() on Windows TCF Agent: Expressions service: improved error message Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- .../tcf-agent/tcf-agent/ldflags.patch | 29 ++- .../tcf-agent/tcf-agent_git.bb
Re: [OE-core] [RFC PATCH v2] qemu: update 7.1.0 -> 7.2.0
On 12/16/22 9:45 AM, Alexander Kanavin wrote: Disable slirp by default: qemu no longer carries libslirp in-tree, and enabling slirp requires providing external libslirp first (available from e.g. meta-virtualization). I think disabling slirp (by default) is going to be problematic. Most of my configurations and those of people I work with use slirp. Forcing people to include meta-virtualization in all configurations may not be reasonable either. I suggest that we bring libslirp from meta-virtualization to oe-core to preserve existing behavior. --Mark Another noteworthy change is: x86: TCG support for AVX, AVX2, F16C, FMA3 and VAES instructions ... which means both meta-intel and qemu x86 targets can now fully utilize Haswell-and-later instruction set with benefits for performance in emulation and on silicon. Changelog: https://wiki.qemu.org/ChangeLog/7.2 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#174763): https://lists.openembedded.org/g/openembedded-core/message/174763 Mute This Topic: https://lists.openembedded.org/mt/95712037/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-Core][master][PATCH] openssh: remove RRECOMMENDS to rng-tools for sshd package
In the kernel is the jitter entropy system. It should work on all platforms that have high res timers available. (This is the same mechanism that haveged was using before as well. So no change in the RNG quality, just now built into the kernel itself.) The only place we've observed an issue with the new approach is on a qemu machine that was moderately loaned. The jitter was apparently too large (our theory) and caused the crng init to happen VERY slowly. (This would not have been resolved using rngd.) All-in-all, I thought this change had already been made, and think it should go in. (We tested this on microblaze, and performance with and without haveged was the same. I don't have number, but I expect the same behaviors on any architecture.) --Mark On 12/9/22 11:02 AM, Khem Raj wrote: would be good to know some numbers on non-arm/non-x86 systems too. On Thu, Dec 8, 2022 at 11:05 PM Xiangyu Chen wrote: It appears that rngd is not needed as of linux-5.6 and later[1] and should not be installed by default since the purpose of rngd is to provide additional trusted sources of entropy. We did some testing on real hardware, the result seems to support that we no longer need rngd by default on kernel v5.6 and later. Testing result as below: 1. observing the crng init stage. the "random: crng init done" always available before fs being mounted. 2. generating random number without rngd. testing command: dd if=/dev/random of=/dev/null status=progress on Marvell CN96xx RDB board, speed almost 20.4 MB/s without block on NXP i.mx6q board, speed almost 31.9 MB/s without block on qemu x86-64, speed almost 2.6MB/s without block 3. using rngtest command without rngd testing command: rngtest -c 1000 https://github.com/torvalds/linux/commit/30c08efec8884fb106b8e57094baa51bb4c44e32 Signed-off-by: Xiangyu Chen --- meta/recipes-connectivity/openssh/openssh_9.1p1.bb | 9 + 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/meta/recipes-connectivity/openssh/openssh_9.1p1.bb b/meta/recipes-connectivity/openssh/openssh_9.1p1.bb index 85f97b1bbb..23ae8d5b0c 100644 --- a/meta/recipes-connectivity/openssh/openssh_9.1p1.bb +++ b/meta/recipes-connectivity/openssh/openssh_9.1p1.bb @@ -52,15 +52,12 @@ SYSTEMD_SERVICE:${PN}-sshd = "sshd.socket" inherit autotools-brokensep ptest -PACKAGECONFIG ??= "rng-tools" +PACKAGECONFIG ??= "" PACKAGECONFIG[kerberos] = "--with-kerberos5,--without-kerberos5,krb5" PACKAGECONFIG[ldns] = "--with-ldns,--without-ldns,ldns" PACKAGECONFIG[libedit] = "--with-libedit,--without-libedit,libedit" PACKAGECONFIG[manpages] = "--with-mantype=man,--with-mantype=cat" -# Add RRECOMMENDS to rng-tools for sshd package -PACKAGECONFIG[rng-tools] = "" - EXTRA_AUTORECONF += "--exclude=aclocal" # login path is hardcoded in sshd @@ -160,10 +157,6 @@ FILES:${PN}-keygen = "${bindir}/ssh-keygen" RDEPENDS:${PN} += "${PN}-scp ${PN}-ssh ${PN}-sshd ${PN}-keygen ${PN}-sftp-server" RDEPENDS:${PN}-sshd += "${PN}-keygen ${@bb.utils.contains('DISTRO_FEATURES', 'pam', 'pam-plugin-keyinit pam-plugin-loginuid', '', d)}" -RRECOMMENDS:${PN}-sshd:append:class-target = "\ -${@bb.utils.filter('PACKAGECONFIG', 'rng-tools', d)} \ -" - # gdb would make attach-ptrace test pass rather than skip but not worth the build dependencies RDEPENDS:${PN}-ptest += "${PN}-sftp ${PN}-misc ${PN}-sftp-server make sed sudo coreutils" -- 2.34.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#174478): https://lists.openembedded.org/g/openembedded-core/message/174478 Mute This Topic: https://lists.openembedded.org/mt/95556189/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [master,langdale][PATCH] qemurunner.py: Support fallback to older QEMU qmp library
Ping, any comments on this? On 11/21/22 1:20 PM, Mark Hatle wrote: From: Mark Hatle Some vendor versions of QEMU may be on older versions that do not have qmp.legacy, but instead of qmp. Default to the integrated library versions, fall back to the older one and then if neither is available catch the exception and provide a more human readable error. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oeqa/utils/qemurunner.py | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py index f175f8a1de..436caef273 100644 --- a/meta/lib/oeqa/utils/qemurunner.py +++ b/meta/lib/oeqa/utils/qemurunner.py @@ -328,8 +328,14 @@ class QemuRunner: try: os.chdir(os.path.dirname(qmp_port)) try: - from qmp.legacy import QEMUMonitorProtocol + try: + from qmp.legacy import QEMUMonitorProtocol + except ModuleNotFoundError: + from qmp import QEMUMonitorProtocol self.qmp = QEMUMonitorProtocol(os.path.basename(qmp_port)) +except ModuleNotFoundError as msg: +self.logger.warning("Failed to load QEMUMonitorProtocol from qmp.legacy or qmp: %s" % (msg)) +return False except OSError as msg: self.logger.warning("Failed to initialize qemu monitor socket: %s File: %s" % (msg, msg.filename)) return False -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#174246): https://lists.openembedded.org/g/openembedded-core/message/174246 Mute This Topic: https://lists.openembedded.org/mt/95180018/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [master,langdale][PATCH] qemurunner.py: Support fallback to older QEMU qmp library
From: Mark Hatle Some vendor versions of QEMU may be on older versions that do not have qmp.legacy, but instead of qmp. Default to the integrated library versions, fall back to the older one and then if neither is available catch the exception and provide a more human readable error. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/lib/oeqa/utils/qemurunner.py | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/meta/lib/oeqa/utils/qemurunner.py b/meta/lib/oeqa/utils/qemurunner.py index f175f8a1de..436caef273 100644 --- a/meta/lib/oeqa/utils/qemurunner.py +++ b/meta/lib/oeqa/utils/qemurunner.py @@ -328,8 +328,14 @@ class QemuRunner: try: os.chdir(os.path.dirname(qmp_port)) try: - from qmp.legacy import QEMUMonitorProtocol + try: + from qmp.legacy import QEMUMonitorProtocol + except ModuleNotFoundError: + from qmp import QEMUMonitorProtocol self.qmp = QEMUMonitorProtocol(os.path.basename(qmp_port)) +except ModuleNotFoundError as msg: +self.logger.warning("Failed to load QEMUMonitorProtocol from qmp.legacy or qmp: %s" % (msg)) +return False except OSError as msg: self.logger.warning("Failed to initialize qemu monitor socket: %s File: %s" % (msg, msg.filename)) return False -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#173661): https://lists.openembedded.org/g/openembedded-core/message/173661 Mute This Topic: https://lists.openembedded.org/mt/95180018/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] Running weston as non-weston user
On 11/1/22 3:03 PM, Gundlupet Raju, Sandeep wrote: Hello, We wanted to know what the actual use case is for running weston as weston user only in OE core http://cgit.openembedded.org/openembedded-core/tree/meta/recipes-graphics/wayland/weston-init/weston.service#n44 How is this expected to be used for another user. Is this intended to be run as the primary machine user or is there another approach? For a system where weston starts itself, it’s expected to start with the weston user only? Can we add a existing user to weston group and make it to work? An additional question: For a system w/o mouse and keyboard, can we ssh in and execute our wayland tests from remote? If so, what user should we be using (weston), can we define our own user? --Mark Thanks, Sandeep -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172385): https://lists.openembedded.org/g/openembedded-core/message/172385 Mute This Topic: https://lists.openembedded.org/mt/94719795/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] mesa: Add native patch via a variable
On 10/28/22 5:59 AM, Alexander Kanavin wrote: Thanks. What I would want to understand is whether this arrangement should have its own MACHINE_FEATURE, e.g. 'vendor-egl' or similar, and then mesa proper would be configured accordingly to disable bits that come from the vendor. Is there a particular need for a separate recipe? When libmali support is enabled in our configuration we do the following: define a mali400 MACHINEOVERRIDE Then we use PREFERRED_PROVIDER to switch: https://github.com/Xilinx/meta-xilinx/blob/master/meta-xilinx-core/conf/machine/include/machine-xilinx-default.inc#L25 virtual/libgles1 virtual/libgles2 virtual/egl virtual/libgl virtual/mesa So the mesa-gl the PROVIDES are: virtual/libgl virtual/mesa while in mesa: PROVIDES = " \ ${@bb.utils.contains('PACKAGECONFIG', 'opengl', 'virtual/libgl', '', d)} \ ${@bb.utils.contains('PACKAGECONFIG', 'gles', 'virtual/libgles1 virtual/libgles2 virtual/libgles3', '', d)} \ ${@bb.utils.contains('PACKAGECONFIG', 'egl', 'virtual/egl', '', d)} \ ${@bb.utils.contains('PACKAGECONFIG', 'gbm', 'virtual/libgbm', '', d)} \ virtual/mesa \ " While it might be possible to switch around the PROVIDES using distibution configuration or similar, this has been difficult to get right in the past. This is why the split happened, it's one or the other based on the configuration. --Mark Alex On Thu, 27 Oct 2022 at 02:46, Mark Hatle wrote: On 10/26/22 1:01 PM, Alexander Kanavin wrote: Is there a combination of libmali recipe and mesa-gl somewhere public so we could try to pick it apart and see what links to where? libmali is ALWAYS implementation specific. You will need to pick a vendor and build for it. What is generic is the application interface (from what I've been told.). I can give you the AMD (Xilinx) instructions. But you likely won't be able to actually execute the code without a corresponding board. (AFAIK QEMU doesn't have any mali400 emulation.) And to be clear, libmali provides the interfaces, but it is NOT a replacement to mesa, it works WITH mesa, just the 'gl' part (not the gles part). The following is an abbreviated attempt to show how the pieces interact with each other. It may not be 100% correct, I am definitely not an expert at this. (For example, I don't understand why it's "DRM" in some places but "DRI" in others. These seem to be 'generally' the same systems.) But it's how the parts and pieces appear to be put together based on a functioning system. In a 'normal' system (mesa): mesa provides: libglapi.so.0 libEGL.so.1 libGLESv2.so.2 (and others) The CONFIG_DRM_LIMA backend and whatever the display port DRM (CONFIG_DRM_XLNX) go through a dlopened backend module, which USUALLY talks to a kernel module for any hardware access through a standard (known) kernel interface. In our case the dynamic modules would be the lima_dri.so and xlnx_dri.so. glmark2-es2-wayland -> libwayland-client libwayland-cursor libwayland-egl.so.1 I believe the libwayland-egl talks to weston "somehow" for the egl interfaces, via the libwayland-client? but I'm fuzzy here. Weston has two related items: drm-backend.o -> libglapi.so.0 (dlopened) This bit uses mesa to access the DRM backend to actually display the stuff.. gl-backend.so -> libwayland-server libEGL.so.1 libGLESv2.so.2 This backend is used to render the items. So the glmark2 does it's benchmarking and renders (gl-backend) through Weston backends to the display port (drm-backend). In the above case it all comes from mesa, the drm and libELG/libGLESv2 parts. The following shows a libmali configured system as a contrasts to the above: libmali-xlnx provides: libEGL.so.1 libGLESv1_CM.so.1 libGLESv2.so.2 libmali.so.9 libgbm.so.1 (also provides specific backends for fbdev, headless, wayland and x11) It talks to a backend kernel module called 'mali.ko' mesa provides: libGL.so.1 libglapi.so.0 and also swrast dri interface glmark2-es2-wayland -> libwayland-client libwayland-cursor libwayland-egl.so.1 (same as above) drm-backend.o -> libwayland-server libdrm.so.2 libMali.so.9 gl-backend.so -> libwayland-server libMali.so.9 So in the above, libMali is providing all of the APIs of libEGL and libGLESv2, as well as (enough of) libglapi for the built in raster implementation. Compiling things like Chromium that will link directly, you move from linking to libGL.so from mesa to linking in libMali, along with libGL from mesa. The libMali will provide the core set of APIs, while the libGL will augement them with additional APIs. Cairo is another utilit
Re: [OE-core] [PATCH v2] mesa: Add native patch via a variable
On 10/26/22 1:01 PM, Alexander Kanavin wrote: Is there a combination of libmali recipe and mesa-gl somewhere public so we could try to pick it apart and see what links to where? libmali is ALWAYS implementation specific. You will need to pick a vendor and build for it. What is generic is the application interface (from what I've been told.). I can give you the AMD (Xilinx) instructions. But you likely won't be able to actually execute the code without a corresponding board. (AFAIK QEMU doesn't have any mali400 emulation.) And to be clear, libmali provides the interfaces, but it is NOT a replacement to mesa, it works WITH mesa, just the 'gl' part (not the gles part). The following is an abbreviated attempt to show how the pieces interact with each other. It may not be 100% correct, I am definitely not an expert at this. (For example, I don't understand why it's "DRM" in some places but "DRI" in others. These seem to be 'generally' the same systems.) But it's how the parts and pieces appear to be put together based on a functioning system. In a 'normal' system (mesa): mesa provides: libglapi.so.0 libEGL.so.1 libGLESv2.so.2 (and others) The CONFIG_DRM_LIMA backend and whatever the display port DRM (CONFIG_DRM_XLNX) go through a dlopened backend module, which USUALLY talks to a kernel module for any hardware access through a standard (known) kernel interface. In our case the dynamic modules would be the lima_dri.so and xlnx_dri.so. glmark2-es2-wayland -> libwayland-client libwayland-cursor libwayland-egl.so.1 I believe the libwayland-egl talks to weston "somehow" for the egl interfaces, via the libwayland-client? but I'm fuzzy here. Weston has two related items: drm-backend.o -> libglapi.so.0 (dlopened) This bit uses mesa to access the DRM backend to actually display the stuff.. gl-backend.so -> libwayland-server libEGL.so.1 libGLESv2.so.2 This backend is used to render the items. So the glmark2 does it's benchmarking and renders (gl-backend) through Weston backends to the display port (drm-backend). In the above case it all comes from mesa, the drm and libELG/libGLESv2 parts. The following shows a libmali configured system as a contrasts to the above: libmali-xlnx provides: libEGL.so.1 libGLESv1_CM.so.1 libGLESv2.so.2 libmali.so.9 libgbm.so.1 (also provides specific backends for fbdev, headless, wayland and x11) It talks to a backend kernel module called 'mali.ko' mesa provides: libGL.so.1 libglapi.so.0 and also swrast dri interface glmark2-es2-wayland -> libwayland-client libwayland-cursor libwayland-egl.so.1 (same as above) drm-backend.o -> libwayland-server libdrm.so.2 libMali.so.9 gl-backend.so -> libwayland-server libMali.so.9 So in the above, libMali is providing all of the APIs of libEGL and libGLESv2, as well as (enough of) libglapi for the built in raster implementation. Compiling things like Chromium that will link directly, you move from linking to libGL.so from mesa to linking in libMali, along with libGL from mesa. The libMali will provide the core set of APIs, while the libGL will augement them with additional APIs. Cairo is another utility that similar modifies in this way. To re-iterate, it's NOT a "swap libmali" and re-use everything. You need to rebuild anything that links to libgles or libgl or even libglapi with this new configuration. It's API compatible, NOT ABI compatible! I hope this helps. As far as letting you play with it, I'm working through the final stuff and should have something I can push to langdale/master on the meta-xilinx and meta-xilinx-tools soon. --Mark My (perhaps confused) understanding is similar to Joshua's: 'gl' is basically 'opengl in x11' or 'opengl rendered to memory', and if a system uses neither, then it is not needed. Alex On Wed, 26 Oct 2022 at 19:48, Joshua Watt wrote: On Wed, Oct 26, 2022 at 12:21 PM Mark Hatle wrote: On 10/26/22 11:03 AM, Alexander Kanavin wrote: On Wed, 26 Oct 2022 at 16:35, Mark Hatle wrote: (I just saw this, so a little late on the reply, but..) mesa-gl is ABSOLUTELY still being used. It's needed for libmali usage. Not everyone wants to use lima support for graphics. It was broken into two separate packages so that is was VERY clear if you were using mesa for 'everything' (mesa), or mesa JUST for 'gl' (not gles). Okay, I'm not going to propose this, but can you clarify, what is the current use for 'gl'? Is it just opengl 3d in standalone x server based systems (which is slowly dying), or is there something else to it? Are you specifically talking about libmali with the following statements? I would say that in general libmali is probably not doing things in th
Re: [OE-core] [PATCH v2] mesa: Add native patch via a variable
On 10/26/22 12:48 PM, Joshua Watt wrote: On Wed, Oct 26, 2022 at 12:21 PM Mark Hatle wrote: On 10/26/22 11:03 AM, Alexander Kanavin wrote: On Wed, 26 Oct 2022 at 16:35, Mark Hatle wrote: (I just saw this, so a little late on the reply, but..) mesa-gl is ABSOLUTELY still being used. It's needed for libmali usage. Not everyone wants to use lima support for graphics. It was broken into two separate packages so that is was VERY clear if you were using mesa for 'everything' (mesa), or mesa JUST for 'gl' (not gles). Okay, I'm not going to propose this, but can you clarify, what is the current use for 'gl'? Is it just opengl 3d in standalone x server based systems (which is slowly dying), or is there something else to it? Are you specifically talking about libmali with the following statements? I would say that in general libmali is probably not doing things in the "normal" way if so, although I'm not trying to say that it isn't a legitimate way to do it. https://github.com/Xilinx/meta-xilinx/blob/master/meta-xilinx-core/recipes-graphics/libgles/libmali-xlnx.bb libmali is providing all of the ELG/GLES/GLES2 and KHR headers. It is providing the libegl, libgles1, libgles2, libgbm. Also provides interfaces to fbdev, X11, and Wayland usage. But all of this is specific to ONLY gles 1 and 2 interfaces. Anything that needs "OpenGL", i.e. Wayland/Weston, X11, Chromium, etc etc etc. These all end up linking to a combination of libmali and mesa-gl. In general (maybe not for libmali) Wayland and Weston themselves don't use OpenGL, or mesa-gl. Individual clients may be able to use it, although it's rare in my (incomplete) experience. Most of the OpenGL users in Wayland/Weston are going through XWayland/X11 and using GLX. Weston won't build without a functional OpenGL/libgles configuration. It errors about missing interfaces. Even with libmali, some of the items need to be disabled as they require libgles3, which libmali does not support. Pretty much anything that depends on 'virtual/gl' (vs virtual/libgles1,2,3). libmali provides virtual/libgles1 and virtual/libgles2 and a few other things that meta-gl doesn't. libmali also doesn't include the DRM/DRI parts. This comes from mesa-gl. In general, DRM/DRI doesn't come from mesa-gl. libdrm is its own recipe, and libgbm comes from mesa proper, or from some other dedicated vendor specific libgbm recipe. https://git.yoctoproject.org/poky/tree/meta/recipes-graphics/mesa/mesa-gl_22.2.0.bb This enables the packageconfig for the gallium set. Our bbappend enabled the dri3 and gallium specifically: https://github.com/Xilinx/meta-xilinx/blob/master/meta-xilinx-core/recipes-graphics/mesa/mesa-gl_%25.bbappend This allows the display out for any application using wayland/xwayland. If this is sidetracking the discussion please ignore; I'm just a little confused by the above statements because either A) libmali is really weird B) my understanding of things is _way_ off, or C) the statements aren't quite correct. libmali just provides a basic set of interfaces that call the mali co-processor to perform actions. The interfaces happen to be libgles1 and 2 and defined by Kronos. It does NOT provide any way to display content, the DRI/DRM interfaces are used for this. So you end up with a configuration for a system that COULD be as simple as libmali + fbdev, where the application renders something with libmali, then uses fbdev to display it. Or you can use X11, or you can use Wayland/Weston. We have a configuration where Chromium is linked to BOTH mesa-gl and libmali in order to do it's rendering, while using Wayland as the compositor/display interfaces... which goes through the DRM system. But ultimately the openGL part is just an engine to "do something", which usually involves drawing shapes into memory. The 'display' side of things happens elsewhere.. and this elsewhere CAN be in mesa, it CAN be in wayland, it can be fbdev, etc... lots of ways, but unless you are creating all custom apps -- you need to be able to use mesa and the existing X11 or Wayland/Weston interfaces (via mesa) in order to do this. --Mark --Mark Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172173): https://lists.openembedded.org/g/openembedded-core/message/172173 Mute This Topic: https://lists.openembedded.org/mt/94420106/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] mesa: Add native patch via a variable
On 10/26/22 11:03 AM, Alexander Kanavin wrote: On Wed, 26 Oct 2022 at 16:35, Mark Hatle wrote: (I just saw this, so a little late on the reply, but..) mesa-gl is ABSOLUTELY still being used. It's needed for libmali usage. Not everyone wants to use lima support for graphics. It was broken into two separate packages so that is was VERY clear if you were using mesa for 'everything' (mesa), or mesa JUST for 'gl' (not gles). Okay, I'm not going to propose this, but can you clarify, what is the current use for 'gl'? Is it just opengl 3d in standalone x server based systems (which is slowly dying), or is there something else to it? Anything that needs "OpenGL", i.e. Wayland/Weston, X11, Chromium, etc etc etc. These all end up linking to a combination of libmali and mesa-gl. Pretty much anything that depends on 'virtual/gl' (vs virtual/libgles1,2,3). libmali provides virtual/libgles1 and virtual/libgles2 and a few other things that meta-gl doesn't. libmali also doesn't include the DRM/DRI parts. This comes from mesa-gl. --Mark Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172165): https://lists.openembedded.org/g/openembedded-core/message/172165 Mute This Topic: https://lists.openembedded.org/mt/94420106/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH v2] mesa: Add native patch via a variable
On 10/19/22 5:35 AM, Alexander Kanavin wrote: This also begs the question if mesa-gl is actually used by anyone, and what warrants its continued inclusion in core as opposed to product/BSP layers. I'll propose dropping the recipe and merging .inc into mesa.bb once patches start flowing. The original commit was: https://git.yoctoproject.org/poky/commit/?h=master-next&id=015cb13 (I just saw this, so a little late on the reply, but..) mesa-gl is ABSOLUTELY still being used. It's needed for libmali usage. Not everyone wants to use lima support for graphics. It was broken into two separate packages so that is was VERY clear if you were using mesa for 'everything' (mesa), or mesa JUST for 'gl' (not gles). --Mark Alex On Wed, 19 Oct 2022 at 08:54, Alexander Kanavin via lists.openembedded.org wrote: On Wed, 19 Oct 2022 at 08:34, Khem Raj wrote: On Tue, Oct 18, 2022 at 10:42 PM Alexander Kanavin wrote: Can you show examples where this is needed? Custom variable does not seem a good idea. I also worry that it’ll break automated updates (it’s bad enough with mesa-gl, this might make it worse). https://github.com/ndechesne/meta-qcom/blob/master/recipes-graphics/mesa/mesa_git.bb "require recipes-graphics/mesa/mesa.inc" Using recipe includes across layers like this is not a good idea. It imposes requirements on core to continue providing the .inc, and not doing anything with mesa that can 'break' users of that include. Please make it self-contained. Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#172160): https://lists.openembedded.org/g/openembedded-core/message/172160 Mute This Topic: https://lists.openembedded.org/mt/94420106/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [langdale,master][PATCH] insane.bbclass: Allow hashlib version that only accepts on parameter
Some versions of hashlib don't appear to implement the second FIPS related argument. Detect this and support both versions. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- This was found on an internal Ubuntu 18.04 container. Unfortunately I don't have access to the container itself but this resolves the issue. meta/classes-global/insane.bbclass | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/meta/classes-global/insane.bbclass b/meta/classes-global/insane.bbclass index db34b4bdb5..dc46857a19 100644 --- a/meta/classes-global/insane.bbclass +++ b/meta/classes-global/insane.bbclass @@ -555,7 +555,10 @@ python populate_lic_qa_checksum() { import hashlib lineno = 0 license = [] -m = hashlib.new('MD5', usedforsecurity=False) +try: +m = hashlib.new('MD5', usedforsecurity=False) +except TypeError: +m = hashlib.new('MD5') for line in f: lineno += 1 if (lineno >= beginline): -- 2.17.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#171515): https://lists.openembedded.org/g/openembedded-core/message/171515 Mute This Topic: https://lists.openembedded.org/mt/94168126/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] wic/wks boot using labels instead of partitions
Thank you, this worked perfectly. On 9/17/22 1:20 AM, Markus Volk wrote: --use-label This should do it. Am Fr, 16. Sep 2022 um 16:41:48 -0500 schrieb Mark Hatle : Is there a way to tell the _generated_ fstab to use labels to boot and not hard coded partitions? If I remove the --ondisk attribute from the wks file it defaults to /dev/sda. What I tried was: part /boot --source bootimg-partition --fstype=vfat --label boot --active --align 4 size 16 part / --source rootfs --fstype=ext4 --label root --align 4 If I manually add --no-fstab-update to the wks entries and add the following to the system default fstab: LABEL=boot /boot vfat default 0 2 This works. But I'd like to automate it within the wks. (Alternatively I could use uuid instead of label booting, but either case, I need to only boot with the uuid or label -- not the partition as the location of the disk can change.) --Mark -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#170836): https://lists.openembedded.org/g/openembedded-core/message/170836 Mute This Topic: https://lists.openembedded.org/mt/93732744/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] wic/wks boot using labels instead of partitions
Is there a way to tell the _generated_ fstab to use labels to boot and not hard coded partitions? If I remove the --ondisk attribute from the wks file it defaults to /dev/sda. What I tried was: part /boot --source bootimg-partition --fstype=vfat --label boot --active --align 4 size 16 part / --source rootfs --fstype=ext4 --label root --align 4 If I manually add --no-fstab-update to the wks entries and add the following to the system default fstab: LABEL=boot /boot vfat default 0 2 This works. But I'd like to automate it within the wks. (Alternatively I could use uuid instead of label booting, but either case, I need to only boot with the uuid or label -- not the partition as the location of the disk can change.) --Mark -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#170823): https://lists.openembedded.org/g/openembedded-core/message/170823 Mute This Topic: https://lists.openembedded.org/mt/93732744/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [Openembedded-architecture] Adding more information to the SBOM
On 9/16/22 10:18 AM, Alberto Pianon wrote: ... trimmed ... I also can see the issue with multiple sources in SRC_URI, although you should be able to map those back if you assume subtrees are "owned" by given SRC_URI entries. I suspect there may be a SPDX format limit in documenting that piece? I'm replying in reverse order: - there is a SPDX format limit, but it is by design: a SPDX package entity is a single sw distribution unit, so it may have only one downloadLocation; if you have more than one downloadLocation, you must have more than one SPDX package, according to SPDX specs; I think my interpretation of this is different. I've got a view of 'sourcing materials', and then verifying the are what we think they are and can be used the way we want. The "upstream sources" (and patches) are really just 'raw materials' that we use the Yocto Project to combined to create "the source". So for the purpose of the SPDX, each upstream source _may_ have a corresponding SPDX, but for the binaries their source is the combined unit.. not multiple SPDXes. Think of it something like: upstream source1 - SPDX upstream source2 - SPDX upstream patch recipe patch1 recipe patch2 In the above, each of those items would be combined by the recipe system to construct the source used to build an individual recipe (and collection of packages). Automation _IS_ used to combine the components [unpack/fetch] and _MAY_ be used to generated a combined SPDX. So your "upstream" location for this recipe is the local machine's source archive. The SPDX for the local recipe files can merge the SPDX information they know (and if it's at a file level) can use checksums to identify the items not captured/modified by the patches for further review (either manual or automation like fossology). In the case where an upstream has SPDX data, you should be able to inherit MOST files this way... but the output is specific to your configuration and patches. 1 - SPDX | 2 - SPDX | patch|---> recipe specific SPDX patch| patch| In some cases someone may want to generate SPDX data for the 3 patches, but that may or may not be useful in this context. - I understand that my solution is a bit hacky; but IMHO any other *post-mortem* solution would be far more hacky; the real solution would be collecting required information directly in do_fetch and do_unpack I've not looked at the current SPDX spec, but past versions has a notes section. Assuming this is still present you can use it to reference back to how this component was constructed and the upstream source URIs (and SPDX files) you used for processing. This way nothing really changes in do_fetch or do_unpack. (You may want to find a way to capture file checksums and what the source was for a particular file.. but it may not really be necessary!) - I also understand that we should reduce pain, otherwise nobody would use our solution; the simplest and cleanest way I can think about is collecting just package (in the SPDX sense) files' relative paths and checksums at every stage (fetch, unpack, patch, package), and leave data processing (i.e. mapping upstream source packages -> recipe's WORKDIR package -> debug source package -> binary packages -> binary image) to a separate tool, that may use (just a thought) a graph database to process things more efficiently. Even it do_patch nothing really changes, other then again you may want to capture checksums to identify thingsthat need further processing. This approach greatly simplifies things, and gives people doing code reviews the insight into what is the source used when shipping the binaries (which is really an important aspect of this), as well as which recipe and "build" (really fetch/unpack/patch) were used to construct the sources. If they want to investigate the sources further back to their provider, then the notes would have the information for that, and you could transition back to the "raw materials" providers. Where I became puzzled is where you say "Information about debug sources for each actual binary file is then taken from tmp/pkgdata//extended/*.json.zstd". This is the data we added and use for the spdx class so you shouldn't need to reinvent that piece. It should be the exact same data the spdx class uses. you're right, but in the context of a POC it was easier to extract them directly from json files than from SPDX data :) It's just a POC to show that required information may be retrieved in some way, implementation details do not matter I was also puzzled about the difference between rpm and the other package backends. The exact same files are packaged by all the package backends so the checksums from do_package should be fine. Here I may miss some piece of information. I looked at files in tmp/pkgdata but I couldn't find package file checksums anywhere: that is why I parsed rpm packages. But if su
Re: [OE-core] [Openembedded-architecture] Adding more information to the SBOM
On 9/14/22 9:56 AM, Joshua Watt wrote: On Wed, Sep 14, 2022 at 9:16 AM Marta Rybczynska wrote: Dear all, (cross-posting to oe-core and *-architecture) In the last months, we have worked in Oniro on using the create-spdx class for both IP compliance and security. During this work, Alberto Pianon has found that some information is missing from the SBOM and it does not contain enough for Software Composition Analysis. The main missing point is the relation between the actual upstream sources and the final binaries (create-spdx uses composite sources). I believe we map the binaries to the source code from the -dbg packages; is the premise that this is insufficient? Can you elaborate more on why that is, I don't quite understand. The debug sources are (basically) what we actually compiled (e.g. post-do_patch) to produce the binary, and you can in turn follow these back to the upstream sources with the downloadLocation property. When I last looked at this, it was critical that the analysis be: binary -> patched & configured source (dbg package) -> how the sources were constructed. As Joshua said above. I believe all of the information is present for this as you can tie the binary (through debug symbols) back to the debug package.. and the source of the debug package back to the sources that constructed it via heuristics. (If you enable the git patch mechanism. It should even be possible to use git blame to find exactly what upstreams constructed the patched sources. For generated content, it's more difficult -- but for those items usually there is a header which indicates what generated the content so other heuristics can be used. Alberto has worked on how to obtain the missing data and now has a POC. This POC provides full source-to-binary tracking of Yocto builds through a couple of scripts (intended to be transformed into a new bbclass at a later stage). The goal is to add the missing pieces of information in order to get a "real" SBOM from Yocto, which should, at a minimum: Please be a little careful with the wording; SBoMs have a lot of uses, and many of them we can satisfy with what we currently generate; it may not do the exact use case you are looking for, but that doesn't mean it's not a "real" SBoM :) - carefully describe what is found in a final image (i.e. binary files and their dependencies), since that is what is actually distributed and goes into the final product; - describe how such binary files have been generated and where they come from (i.e. upstream sources, including patches and other stuff added from meta-layers); provenance is important for a number of reasons related to IP Compliance and security. Full compliance will require binaries mapped to patched source to upstream sources _AND_ the instructions (layer/recipe/configuration) used to build them. But it's up to the local legal determination to figure out 'how far you really need to go', vs just "here are the layers I used to build my project".) The aim is to become able to: - map binaries to their corresponding upstream source packages (and not to the "internal" source packages created by recipes by combining multiple upstream sources and patches) - map binaries to the source files that have been actually used to build them - which usually are a small subset of the whole source package With respect to IP compliance, this would allow to, among other things: - get the real license text for each binary file, by getting the license of the specific source files it has been generated from (provided by Fossology, for instance), - and not the main license stated in the corresponding recipe (which may be as confusing as GPL-2.0-or-later & LGPL-2.1-or-later & BSD-3-Clause & BSD-4-Clause, or even worse) IIUC this is the difference between the "Declared" license and the "Concluded" license. You can report both, and I think create-spdx.bbclass can currently do this with its rudimentary source license scanning. You really do want both and it's a great way to make sure that the "Declared" license (that is the license in the recipe) reflects the reality of the source code. And the thing to keep in mind is that in a given package the "Declared" is usually what a LICENSE file or header says. But the "Concluded" has levels of quality behind them. The first level of quality is "Declared". The next level is automation (something like fossology), the next level is human reviewed, and the highest level is "lawyer reviewed". So being able to inject SPDX information with Concluded values for evaluation and track the 'quality level' has always been something I wanted to do, but never had time. At the time, my idea was a database (and/or bbappend) for each component that would included pre-processed SPDX data for each recipe. This data would run through a validation step to show it actually matches the patched sources. (If any file checksums do NOT match, then they would be flagged for follo
Re: [OE-core] [Openembedded-architecture] Configuration fragments
Wind River already has a mechanism to do something like this, called templates. https://github.com/WindRiver-Labs/wr-template/tree/WRLINUX_10_21_BASE Each template can have (optionally): # README - template README file # require - list of other templates required for this one # template.conf - template configuration fragment # image.inc - image fragment # bsp-pkgs.conf - BSP specific configuration fragment # bsp-pkgs.inc - BSP specific image fragment # The 'bsp-pkgs' files can only be in a template in a layer that provides a # specific conf/machine/${MACHINE}.conf file and layers it may contain, # otherwise they will be ignored I'm not saying what is implemented is a perfect solution, but it made a 'configuration fragment' approach to system configuration much easier when I worked with it. (For examples see: https://github.com/WindRiver-Labs/wrlinux/tree/WRLINUX_10_21_BASE/templates/feature) You could do things (in your local.conf, or machine.conf or distro.conf) like: WRTEMPLATE += "dpdk" This would add the dpdk recipes to your image and also configure the kernel for dpdk. https://github.com/WindRiver-Labs/wrlinux/tree/WRLINUX_10_21_BASE/templates/feature/dpdk --Mark On 9/1/22 10:29 AM, Alexander Kanavin wrote: On Thu, 18 Aug 2022 at 11:27, Richard Purdie wrote: The intent is the user could add something like: require conf/yocto-autobuilder/x32.inc or require conf/yocto-autobuilder/multilib-mipsn32.inc and similar to their local.conf and more easily replicate configurations. This could be a subset of a broader feature, one that allows adding and removing pre-canned 'config fragments' to local.conf. The proposal is to have such fragments live in meta-somelayer/conf/fragments/path/to/some/fragment (I think allowing multiple levels is beneficial here), and each fragment would start with a comment explaining what it does followed by the actual settings. Then adding fragments can be done by appending them to local.conf, and removing by looking for markers in local.conf and editing it out, which probably could use a helper tool/subcommand. Layers could also declare 'non-optional' fragments which get included automatically by the fact of including the layer into the build, which means layer.conf can go back to being short and sweet and to the point. Thoughts? I would like to see if we can switch the eSDK to use the json format btw. I suspect that may be one of your next steps? :) I don't think I understand this, can you explain? Alex -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#170229): https://lists.openembedded.org/g/openembedded-core/message/170229 Mute This Topic: https://lists.openembedded.org/mt/93407466/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] rootfs-postcommands: Make /etc/timestamp consistent with image
On 8/29/22 6:59 PM, William A. Kennington III via lists.openembedded.org wrote: This makes the determination of the timestamp for the /etc/timestamp file consistent with mtimes in the generated image. This is desirable to make the built image reproducible with the git commit date instead of the current date. This is only going to pay attention to poky, or oe-core... the rootfs will vary with a change to ANY layer in the system. If you really want to be consistent you would need to iterate over all of the layers and look at every file in the system for the timestamp (or if they're git repositories last commit in each layer..) All of this doesn't seem like it really makes sense to me which is why a static timestamp was defined previously. (with the previous change to the time stamps, I've had more then one user get concerned they were 'using old software' when they just built something from scratch -- this will get even more confusing if it's only based on the oe-core/poky repository.) --Mark Change-Id: I7d9fe32906aa93baf53948aa40b7a98fb05dd384 Signed-off-by: William A. Kennington III --- meta/classes-recipe/rootfs-postcommands.bbclass | 12 +++- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/meta/classes-recipe/rootfs-postcommands.bbclass b/meta/classes-recipe/rootfs-postcommands.bbclass index 215e38e33d..8d710186d7 100644 --- a/meta/classes-recipe/rootfs-postcommands.bbclass +++ b/meta/classes-recipe/rootfs-postcommands.bbclass @@ -312,12 +312,14 @@ python write_image_manifest () { # Can be used to create /etc/timestamp during image construction to give a reasonably # sane default time setting rootfs_update_timestamp () { - if [ "${REPRODUCIBLE_TIMESTAMP_ROOTFS}" != "" ]; then - # Convert UTC into %4Y%2m%2d%2H%2M%2S - sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S` - else - sformatted=`date -u +%4Y%2m%2d%2H%2M%2S` + if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then + REPRODUCIBLE_TIMESTAMP_ROOTFS=`git -C "${COREBASE}" log -1 --pretty=%ct 2>/dev/null` || true + if [ "$REPRODUCIBLE_TIMESTAMP_ROOTFS" = "" ]; then + REPRODUCIBLE_TIMESTAMP_ROOTFS=`stat -c%Y ${@bb.utils.which(d.getVar("BBPATH"), "conf/bitbake.conf")}` + fi fi + # Convert UTC into %4Y%2m%2d%2H%2M%2S + sformatted=`date -u -d @${REPRODUCIBLE_TIMESTAMP_ROOTFS} +%4Y%2m%2d%2H%2M%2S` echo $sformatted > ${IMAGE_ROOTFS}/etc/timestamp bbnote "rootfs_update_timestamp: set /etc/timestamp to $sformatted" } -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#170042): https://lists.openembedded.org/g/openembedded-core/message/170042 Mute This Topic: https://lists.openembedded.org/mt/93339153/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] pseudo deadlock in rust via malloc backtrace
On 8/4/22 11:27 AM, Richard Purdie wrote: On Thu, 2022-08-04 at 12:36 +0100, Khem Raj wrote: On Thu, Aug 4, 2022 at 10:05 AM Richard Purdie wrote: Just to note that where we've been seeing pseudo hangs in rust, it seems jemalloc calls obtain_malloc_conf() which calls readlink() which pseudo intercepts and then calls malloc() which loops back to jemalloc which deadlocks. What fun. (Going way back) the way this was avoided in the distant past was having a static variable of whatever the 'max size' was already allocated in a few key locations and using that variable instead of a malloc. (Of course due to it having a fixed length a size check and such needed to be done.) I don't really know the context at this point for this item, but it's an alternative way to solve this problem, but will require memory to be allocated at compile time, either in a function or a small global pool for usage. It does make me wonder if there are other function in the system that use malloc and if so could they use an alternative, if not the 'is_init' might be the only way to handle it. --Mark I guess pseudo is doing this extra malloc right ? Yes Maybe disable instrumentation in jemalloc when building non target packages. This might avoid the issue at least Sadly it doesn't help as it wants to load it isn't configuration and the system setup from /proc too. Cheers, Richard -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168976): https://lists.openembedded.org/g/openembedded-core/message/168976 Mute This Topic: https://lists.openembedded.org/mt/92809824/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 4/4] elfutils: Microblaze does not support symvers
Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/recipes-devtools/elfutils/elfutils_0.187.bb | 3 +++ 1 file changed, 3 insertions(+) diff --git a/meta/recipes-devtools/elfutils/elfutils_0.187.bb b/meta/recipes-devtools/elfutils/elfutils_0.187.bb index d03da61353..561112c580 100644 --- a/meta/recipes-devtools/elfutils/elfutils_0.187.bb +++ b/meta/recipes-devtools/elfutils/elfutils_0.187.bb @@ -49,6 +49,9 @@ RDEPENDS:${PN}-ptest += "libasm libelf bash make coreutils ${PN}-binutils iprout EXTRA_OECONF:append:class-target = " --disable-tests-rpath" +# symver functions not currently supported on microblaze +EXTRA_OECONF:append:class-target:microblaze = " --disable-symbol-versioning" + RDEPENDS:${PN}-ptest:append:libc-glibc = " glibc-utils glibc-dbg glibc-dev" INSANE_SKIP:${PN}-ptest = "debug-deps dev-deps" -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168522): https://lists.openembedded.org/g/openembedded-core/message/168522 Mute This Topic: https://lists.openembedded.org/mt/92639282/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 3/4] openssl: Move microblaze to linux-latomic config
When building with the previous a number of atomic functions come back as undefined. Switching to linux-latomic fixes this. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/recipes-connectivity/openssl/openssl_3.0.5.bb | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/meta/recipes-connectivity/openssl/openssl_3.0.5.bb b/meta/recipes-connectivity/openssl/openssl_3.0.5.bb index e50ff7f8c5..04aff04fab 100644 --- a/meta/recipes-connectivity/openssl/openssl_3.0.5.bb +++ b/meta/recipes-connectivity/openssl/openssl_3.0.5.bb @@ -77,7 +77,7 @@ do_configure () { esac target="$os-${HOST_ARCH}" case $target in - linux-arc) + linux-arc | linux-microblaze*) target=linux-latomic ;; linux-arm*) @@ -105,7 +105,7 @@ do_configure () { linux-*-mips64 | linux-mips64 | linux-*-mips64el | linux-mips64el) target=linux64-mips64 ;; - linux-microblaze* | linux-nios2* | linux-sh3 | linux-sh4 | linux-arc*) + linux-nios2* | linux-sh3 | linux-sh4 | linux-arc*) target=linux-generic32 ;; linux-powerpc) -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168521): https://lists.openembedded.org/g/openembedded-core/message/168521 Mute This Topic: https://lists.openembedded.org/mt/92639281/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 0/4] Various fixes for master
While working on forward porting microblaze support I ran into the following patches. default-distrovars.inc change, we've been doing this in our petalinux distro -- but it seems like a better idea to do it generically. The runqemu one is a missing space in code that I THINK is only used by microblaze systems currently. (Or at least Xilinx systems with the custom qemu-xilinx.) The other two are simple recipe updates. Mark Hatle (4): runqemu: Add missing space on default display option default-distrovars: seccomp doesn't support microblaze openssl: Move microblaze to linux-latomic config elfutils: Microblaze does not support symvers meta/conf/distro/include/default-distrovars.inc| 3 +++ meta/recipes-connectivity/openssl/openssl_3.0.5.bb | 4 ++-- meta/recipes-devtools/elfutils/elfutils_0.187.bb | 3 +++ scripts/runqemu| 2 +- 4 files changed, 9 insertions(+), 3 deletions(-) -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168518): https://lists.openembedded.org/g/openembedded-core/message/168518 Mute This Topic: https://lists.openembedded.org/mt/92639278/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/4] runqemu: Add missing space on default display option
Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- scripts/runqemu | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/runqemu b/scripts/runqemu index b4c1ae6d83..b6fc212ebe 100755 --- a/scripts/runqemu +++ b/scripts/runqemu @@ -1375,7 +1375,7 @@ class BaseConfig(object): elif "-display sdl" in output: self.sdl = True else: -self.qemu_opt += '-display none' +self.qemu_opt += ' -display none' if self.sdl == True or self.gtk == True or self.egl_headless == True: -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168520): https://lists.openembedded.org/g/openembedded-core/message/168520 Mute This Topic: https://lists.openembedded.org/mt/92639280/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 2/4] default-distrovars: seccomp doesn't support microblaze
Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/conf/distro/include/default-distrovars.inc | 3 +++ 1 file changed, 3 insertions(+) diff --git a/meta/conf/distro/include/default-distrovars.inc b/meta/conf/distro/include/default-distrovars.inc index 9f4617be01..230bab84dd 100644 --- a/meta/conf/distro/include/default-distrovars.inc +++ b/meta/conf/distro/include/default-distrovars.inc @@ -16,6 +16,9 @@ DISTRO_FEATURES_DEFAULT:remove:riscv32 = "seccomp" # seccomp is not yet ported to ARC DISTRO_FEATURES_DEFAULT:remove:arc = "seccomp" +# seccomp is not yet ported to microblaze +DISTRO_FEATURES_DEFAULT:remove:microblaze = "seccomp" + DISTRO_FEATURES_DEFAULT ?= "acl alsa bluetooth debuginfod ext2 ipv4 ipv6 largefile pcmcia usbgadget usbhost wifi xattr nfs zeroconf pci 3g nfc x11 vfat seccomp" DISTRO_FEATURES ?= "${DISTRO_FEATURES_DEFAULT}" IMAGE_FEATURES ?= "" -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168519): https://lists.openembedded.org/g/openembedded-core/message/168519 Mute This Topic: https://lists.openembedded.org/mt/92639279/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [OE-core] [PATCH] sstate: inside the threadedpool don't write to the shared localdata
Note, it appears this bug is in honister as well. On 3/6/22 3:08 PM, Jose Quaresma wrote: When inside the threadedpool we make a copy of the localdata to avoid some race condition, so we need to use this new localdata2 and stop write the shared localdata. Signed-off-by: Jose Quaresma --- meta/classes/sstate.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/sstate.bbclass b/meta/classes/sstate.bbclass index dc9a2c085b..7aca415159 100644 --- a/meta/classes/sstate.bbclass +++ b/meta/classes/sstate.bbclass @@ -988,7 +988,7 @@ def sstate_checkhashes(sq_data, d, siginfo=False, currentcount=0, summary=True, localdata2 = bb.data.createCopy(localdata) srcuri = "file://" + sstatefile -localdata.setVar('SRC_URI', srcuri) +localdata2.setVar('SRC_URI', srcuri) bb.debug(2, "SState: Attempting to fetch %s" % srcuri) import traceback -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162853): https://lists.openembedded.org/g/openembedded-core/message/162853 Mute This Topic: https://lists.openembedded.org/mt/89597961/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 1/1] insane.bbclass: Update insane.bbclass to work on FIPS enabled hosts
hashlib.md5() is not permitted on a FIPS enabled host system. This is due to md5 not being an approved hash algorithm. Instead use: hashlib.new('MD5', usedforsecurity=False) This is allowed, as it's clear the hash is used for a non-security purpose. Using an md5 to identify when a license has changed is permitted, as we're not using it for file integrity. Signed-off-by: Mark Hatle Signed-off-by: Mark Hatle --- meta/classes/insane.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta/classes/insane.bbclass b/meta/classes/insane.bbclass index 890e865a8f..29b9b3d466 100644 --- a/meta/classes/insane.bbclass +++ b/meta/classes/insane.bbclass @@ -549,7 +549,7 @@ python populate_lic_qa_checksum() { import hashlib lineno = 0 license = [] -m = hashlib.md5() +m = hashlib.new('MD5', usedforsecurity=False) for line in f: lineno += 1 if (lineno >= beginline): -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162520): https://lists.openembedded.org/g/openembedded-core/message/162520 Mute This Topic: https://lists.openembedded.org/mt/8940/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[OE-core] [PATCH 0/1] FIPS host support
The patch here, and one sent to bitbake-devel together enable basic support for a FIPS-140 host system. What was identified were a few users of md5, which is not allowed for any security part of the system. It can still be used to identify changes and similar non-security activities. (OE already uses sha256 for file integrity.) In addition to this, it's possible that a recipe may attempt to use md5 during the build process. In oe-core, the only user is 'ovmf'. At this time I don't intend to provide a fix for ovmf, but everything else in core works properly now. Mark Hatle (1): insane.bbclass: Update insane.bbclass to work on FIPS enabled hosts meta/classes/insane.bbclass | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) -- 2.25.1 -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#162521): https://lists.openembedded.org/g/openembedded-core/message/162521 Mute This Topic: https://lists.openembedded.org/mt/8941/21656 Group Owner: openembedded-core+ow...@lists.openembedded.org Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-