On Thu, 2022-07-28 at 15:32 +0100, Jose Quaresma wrote: > > > Jose Quaresma via lists.openembedded.org > <[email protected]> escreveu no dia > quinta, 28/07/2022 à(s) 12:46: > > > > > > Richard Purdie <[email protected]> escreveu no dia > > quinta, 28/07/2022 à(s) 12:01: > > > On Thu, 2022-07-28 at 09:54 +0100, Jose Quaresma wrote: > > > > Otherwise spdx can have references for data that is not packed. > > > > in the package delivered. > > > > > > > > During do_package_write_ipk task in do_package_ipk some files > > > > is cleaned up from packages-split directory in the funcion > > > > cleanupcontrol. > > > > > > > > This also fixes the following race condictions when the > > > > do_create_spdx > > > > task runs the add_package_files function and these files is > > > > been deleted > > > > at same time in the task do_package_write_ipk: > > > > > > > > ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error > > > > executing a python function in exec_func_python() > > > > autogenerated: > > > > > > > > The stack trace of python calls that resulted in this > > > > exception/failure was: > > > > File: 'exec_func_python() autogenerated', lineno: 2, function: > > > > <module> > > > > 0001: > > > > *** 0002:do_create_spdx(d) > > > > 0003: > > > > File: '/srv/oe/build/conf/../../layers/openembedded- > > > > core/meta/classes/create-spdx.bbclass', lineno: 567, function: > > > > do_create_spdx > > > > 0563: > > > > package_doc.add_relationship(package_doc, "DESCRIBES", > > > > spdx_package) > > > > 0564: > > > > 0565: package_archive = deploy_dir_spdx / > > > > "packages" / (package_doc.name + ".tar.zst") > > > > 0566: with optional_tarfile(package_archive, > > > > archive_packaged) as archive: > > > > *** 0567: package_files = add_package_files( > > > > 0568: d, > > > > 0569: package_doc, > > > > 0570: spdx_package, > > > > 0571: pkgdest / package, > > > > File: '/srv/oe/build/conf/../../layers/openembedded- > > > > core/meta/classes/create-spdx.bbclass', lineno: 234, function: > > > > add_package_files > > > > 0230: info.mtime = > > > > source_date_epoch > > > > 0231: > > > > 0232: archive.addfile(info, f) > > > > 0233: > > > > *** 0234: sha1 = bb.utils.sha1_file(filepath) > > > > 0235: sha1s.append(sha1) > > > > 0236: > > > > spdx_file.checksums.append(oe.spdx.SPDXChecksum( > > > > 0237: algorithm="SHA1", > > > > 0238: checksumValue=sha1, > > > > File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: > > > > sha1_file > > > > 0555: """ > > > > 0556: Return the hex string representation of the SHA1 > > > > checksum of the filename > > > > 0557: """ > > > > 0558: import hashlib > > > > *** 0559: return _hasher(hashlib.sha1(), filename) > > > > 0560: > > > > 0561:def sha384_file(filename): > > > > 0562: """ > > > > 0563: Return the hex string representation of the > > > > SHA384 checksum of the filename > > > > File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: > > > > _hasher > > > > 0524: > > > > 0525:def _hasher(method, filename): > > > > 0526: import mmap > > > > 0527: > > > > *** 0528: with open(filename, "rb") as f: > > > > 0529: try: > > > > 0530: with mmap.mmap(f.fileno(), 0, > > > > access=mmap.ACCESS_READ) as mm: > > > > 0531: for chunk in iter(lambda: > > > > mm.read(8192), b''): > > > > 0532: method.update(chunk) > > > > Exception: FileNotFoundError: [Errno 2] No such file or > > > > directory: '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa- > > > > topology-conf/1.2.5.1-r0/packages-split/alsa-topology- > > > > conf/CONTROL/control' > > > > > > We have other places where we've had to teach the code to ignore > > > the > > > control files and we'll probably have to do that here (grep for > > > CONTROL). > > > > > > > > > The snip that delete this files is in cleanupcontrol > > https://git.yoctoproject.org/poky/tree/meta/classes/package_ipk.bbclass#n51 > > > > So these files are not shipped in the ipk package produced. > > > > > What I mean is those files are not part of the content deployed and > it is a requirement of the packing > standard used so tracking it the sbom doesn't make much sense imo.
Right, it would be fine to make the spdx class skip them if they were present. > ar -xv deploy/ipk/all/alsa-topology-conf_1.2.5.1-r0_all.ipk > x - debian-binary > x - control.tar.gz > x - data.tar.xz > > For and deb and ipk we have in deb_write_pkg and ipk_write_pkg: > > finally: > cleanupcontrol(root) > bb.utils.unlockfile(lf) > > For rpm we have in do_package_rpm: > > # rpm 4 creates various empty directories in _topdir, let's > clean them up > cleanupcmd = "rm -rf %s/BUILDROOT %s/SOURCES %s/SPECS > %s/SRPMS" % (workdir, workdir, workdir, workdir) > > So all of them delete this package control content at the end but as > do_create_spdx > runs at the same time it can see and track this content as well. Yes, it is a race. We've generally tweaked the code using these directories to ignore these files rather than change the dependencies though. Ideally we'd change dpkg/opkg to place the files somewhere else during package build but we've not worked out how to do that as yet. Cheers, Richard
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168619): https://lists.openembedded.org/g/openembedded-core/message/168619 Mute This Topic: https://lists.openembedded.org/mt/92666627/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
