We’re using Poky Kirkstone(LTS) with the latest version for our SDK. Recently, we're trying to enable the feature of creating SBOM in SPDX. But we met a sstate-cache checking failure issue when enabling SPDX to generate SBOM. It will be appreciated if you have any idea for the root cause. Refer to below details for the issue.
https://docs.yoctoproject.org/4.0.10/singleindex.html#creating-a-software-bill-of-materials ( https://urldefense.com/v3/__https:/docs.yoctoproject.org/4.0.10/singleindex.html*creating-a-software-bill-of-materials__;Iw!!PeEy7nZLVv0!iEYzUu9NzMxHXrypN88tmMw1XE3I6xJIUHhsKfEvEXx92zX6oPews42VwnTYHjVn4vmJR2pk0CEn$ ) *==== Here is an easy way to reproduce the issue ====* *1. pull cbuild-ng project* jleng@shbuild8:/yocto/system-team/jleng$ git clone https://github.com/lengjingzju/cbuild-ng.git ( https://urldefense.com/v3/__https:/github.com/lengjingzju/cbuild-ng.git__;!!PeEy7nZLVv0!iEYzUu9NzMxHXrypN88tmMw1XE3I6xJIUHhsKfEvEXx92zX6oPews42VwnTYHjVn4vmJRzxUjhi5$ ) jleng@shbuild8:/yocto/system-team/jleng$ cd cbuild-ng *2. Use the latest code of Yocto kirkstone and enable SBOM* jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng$ git diff diff --git a/scripts/yocto.env b/ scripts/yocto.env index ed8bb35..b7f1316 100755 --- a/scripts/yocto.env +++ b/scripts/yocto.env @@ -60,7 +60,7 @@ export ENV_TOP_OUT ENV_CFG_ROOT POKY_URL=git:// git.yoctoproject.org/poky ( https://urldefense.com/v3/__http:/git.yoctoproject.org/poky__;!!PeEy7nZLVv0!iEYzUu9NzMxHXrypN88tmMw1XE3I6xJIUHhsKfEvEXx92zX6oPews42VwnTYHjVn4vmJR6uBUfNh$ ) POKY_BRANCH=kirkstone -POKY_REV=30656ea54de82cabb818f07af711b96211421c95 +POKY_REV=1e6f147c686a91b9a5996f529e32ee1852f946b7 POKY_PATH=${ENV_TOP_OUT}/poky if [ ! -e "${POKY_PATH}" ]; then @@ -101,6 +101,9 @@ EXTERNALSRC_SYMLINKS = "" ENV_TOP_DIR = "${ENV_TOP_DIR}" ENV_CFG_ROOT = "${ENV_CFG_ROOT}" ENV_BUILD_MODE = "${ENV_BUILD_MODE}" + +INHERIT += "create-spdx" +SPDX_PRETTY = "1" EOF *3. First build, success* jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng$ source scripts/yocto.env jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng/build/qemux86-64$ bitbake kconfig-native *4. Remove output and re-build, report an error* jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng/build/qemux86-64$ cd - jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng$ rm -rf build/qemux86-64 jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng$ source scripts/yocto.env jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng/build/qemux86-64$ bitbake kconfig-native /usr/lib/python3/dist-packages/html5lib/_trie/_base.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working from collections import Mapping WARNING: You are using a local hash equivalence server but have configured an sstate mirror. This will likely mean no sstate will match from the mirror. You may wish to disable the hash equivalence use (BB_HASHSERVE), or use a hash equivalence server alongside the sstate mirror. Loading cache: 100% | | ETA: --:--:-- Loaded 0 entries from dependency cache. Parsing recipes: 100% |##########################################################################################################################| Time: 0:00:05 Parsing of 885 .bb files complete (0 cached, 885 parsed). 1647 targets, 44 skipped, 0 masked, 0 errors. NOTE: Resolving any missing task queue dependencies Build Configuration: BB_VERSION = "2.0.0" BUILD_SYS = "x86_64-linux" NATIVELSBSTRING = "ubuntu-20.04" TARGET_SYS = "x86_64-poky-linux" MACHINE = "qemux86-64" DISTRO = "poky" DISTRO_VERSION = "4.0.10" TUNE_FEATURES = "m64 core2" TARGET_FPU = "" meta meta-poky meta-yocto-bsp = "kirkstone:1e6f147c686a91b9a5996f529e32ee1852f946b7" meta-cbuild = "main:10f9581f393bf67cd06e8849bad409ba91d4d9ad" Initialising tasks: 100% |#######################################################################################################################| Time: 0:00:00 Sstate summary: Wanted 37 Local 37 Mirrors 0 Missed 0 Current 0 (100% match, 0% complete) NOTE: Executing Tasks ERROR: kconfig-native-1.0-r0 do_create_spdx: Error executing a python function in exec_func_python() autogenerated: The stack trace of python calls that resulted in this exception/failure was: File: 'exec_func_python() autogenerated', lineno: 2, function: <module> 0001: *** 0002:do_create_spdx(d) 0003: File: '/yocto/system-team/jleng/cbuild-ng/build/poky/meta/classes/create-spdx.bbclass', lineno: 521, function: do_create_spdx 0517: 0518: if archive is not None: 0519: recipe.packageFileName = str( recipe_archive.name ( https://urldefense.com/v3/__http:/recipe_archive.name__;!!PeEy7nZLVv0!iEYzUu9NzMxHXrypN88tmMw1XE3I6xJIUHhsKfEvEXx92zX6oPews42VwnTYHjVn4vmJRzKP95UT$ ) ) 0520: *** 0521: dep_recipes = collect_dep_recipes(d, doc, recipe) 0522: 0523: doc_sha1 = oe.sbom.write_doc(d, doc, "recipes", indent=get_json_indent(d)) 0524: dep_recipes.append(oe.sbom.DepRecipe(doc, doc_sha1, recipe)) 0525: File: '/yocto/system-team/jleng/cbuild-ng/build/poky/meta/classes/create-spdx.bbclass', lineno: 350, function: collect_dep_recipes 0346: )) 0347: for dep_pn in deps: 0348: dep_recipe_path = deploy_dir_spdx / "recipes" / ("recipe-%s.spdx.json" % dep_pn) 0349: *** 0350: spdx_dep_doc, spdx_dep_sha1 = oe.sbom.read_doc(dep_recipe_path) 0351: 0352: for pkg in spdx_dep_doc.packages: 0353: if pkg.name ( https://urldefense.com/v3/__http:/pkg.name__;!!PeEy7nZLVv0!iEYzUu9NzMxHXrypN88tmMw1XE3I6xJIUHhsKfEvEXx92zX6oPews42VwnTYHjVn4vmJR1_uoEsE$ ) == dep_pn: 0354: spdx_dep_recipe = pkg File: '/yocto/system-team/jleng/cbuild-ng/build/poky/meta/lib/oe/sbom.py', lineno: 67, function: read_doc 0063: else: 0064: with fn.open("rb") as f: 0065: yield f 0066: *** 0067: with get_file() as f: 0068: sha1 = hashlib.sha1() 0069: while True: 0070: chunk = f.read(4096) 0071: if not chunk: File: '/usr/lib/python3.8/contextlib.py', lineno: 113, function: __enter__ 0109: # do not keep args and kwds alive unnecessarily 0110: # they are only needed for recreation, which is not possible anymore 0111: del self.args, self.kwds, self.func 0112: try: *** 0113: return next(self.gen) 0114: except StopIteration: 0115: raise RuntimeError("generator didn't yield") from None 0116: 0117: def __exit__(self, type, value, traceback): File: '/yocto/system-team/jleng/cbuild-ng/build/poky/meta/lib/oe/sbom.py', lineno: 64, function: get_file 0060: def get_file(): 0061: if isinstance(fn, io.IOBase): 0062: yield fn 0063: else: *** 0064: with fn.open("rb") as f: 0065: yield f 0066: 0067: with get_file() as f: 0068: sha1 = hashlib.sha1() File: '/usr/lib/python3.8/pathlib.py', lineno: 1222, function: open 1218: the built-in open() function does. 1219: """ 1220: if self._closed: 1221: self._raise_closed() *** 1222: return io.open(self, mode, buffering, encoding, errors, newline, 1223: opener=self._opener) 1224: 1225: def read_bytes(self): 1226: """ File: '/usr/lib/python3.8/pathlib.py', lineno: 1078, function: _opener 1074: raise ValueError("I/O operation on closed path") 1075: 1076: def _opener(self, name, flags, mode=0o666): 1077: # A stub for the opener argument to built-in open() *** 1078: return self._accessor.open(self, flags, mode) 1079: 1080: def _raw_open(self, flags, mode=0o777): 1081: """ 1082: Open the file pointed by this path and return a file descriptor, Exception: FileNotFoundError: [Errno 2] No such file or directory: '/yocto/system-team/jleng/cbuild-ng/build/qemux86-64/tmp/deploy/spdx/qemux86-64/recipes/recipe-autoconf-native.spdx.json' ERROR: Logfile of failure stored in: /yocto/system-team/jleng/cbuild-ng/build/qemux86-64/tmp/work/x86_64-linux/kconfig-native/1.0-r0/temp/log.do_create_spdx.1001451 ERROR: Task (virtual:native:/yocto/system-team/jleng/cbuild-ng/scripts/meta-cbuild/recipes-devtools/kconfig/kconfig.bb:do_create_spdx) failed with exit code '1' NOTE: Tasks Summary: Attempted 133 tasks of which 128 didn't need to be rerun and 1 failed. Summary: 1 task failed: virtual:native:/yocto/system-team/jleng/cbuild-ng/scripts/meta-cbuild/recipes-devtools/kconfig/kconfig.bb:do_create_spdx Summary: There was 1 WARNING message. Summary: There was 1 ERROR message, returning a non-zero exit code. jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng/build/qemux86-64$ ls tmp/work/x86_64-linux/ bison-native gettext-minimal-native kconfig-native m4-native pkgconfig-native texinfo-dummy-native flex-native gnu-config-native libtool-native ncurses-native quilt-native xz-native *==== Currently, we found a workaround solution to fix this issue ====* jleng@shbuild8:/yocto/system-team/jleng/cbuild-ng/build/poky$ git diff diff --git a/meta/classes/sstate.bbclass b/ meta/classes/sstate.bbclass index dd6cf12920..f47febeb32 100644 --- a/meta/classes/sstate.bbclass +++ b/meta/classes/sstate.bbclass @@ -1084,7 +1084,7 @@ def setscene_depvalid(task, taskdependees, notneeded, d, log=None): logit("Considering setscene task: %s" % (str(taskdependees[task])), log) - directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_create_spdx" , "do_deploy_archives"] + directtasks = ["do_populate_lic", "do_deploy_source_date_epoch", "do_shared_workdir", "do_stash_locale", "do_gcc_stash_builddir", "do_deploy_archives"] def isNativeCross(x): return x.endswith("-native") or "-cross-" in x or "-crosssdk" in x or x.endswith("-cross")
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#60224): https://lists.yoctoproject.org/g/yocto/message/60224 Mute This Topic: https://lists.yoctoproject.org/mt/99403018/21656 Group Owner: [email protected] Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
