Jose Quaresma via lists.openembedded.org <quaresma.jose= [email protected]> escreveu no dia quinta, 28/07/2022 à(s) 12:46:
> > > Richard Purdie <[email protected]> escreveu no dia > quinta, 28/07/2022 à(s) 12:01: > >> On Thu, 2022-07-28 at 09:54 +0100, Jose Quaresma wrote: >> > Otherwise spdx can have references for data that is not packed. >> > in the package delivered. >> > >> > During do_package_write_ipk task in do_package_ipk some files >> > is cleaned up from packages-split directory in the funcion >> > cleanupcontrol. >> > >> > This also fixes the following race condictions when the do_create_spdx >> > task runs the add_package_files function and these files is been deleted >> > at same time in the task do_package_write_ipk: >> > >> > ERROR: alsa-topology-conf-1.2.5.1-r0 do_create_spdx: Error executing a >> python function in exec_func_python() autogenerated: >> > >> > The stack trace of python calls that resulted in this exception/failure >> was: >> > File: 'exec_func_python() autogenerated', lineno: 2, function: <module> >> > 0001: >> > *** 0002:do_create_spdx(d) >> > 0003: >> > File: >> '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', >> lineno: 567, function: do_create_spdx >> > 0563: package_doc.add_relationship(package_doc, >> "DESCRIBES", spdx_package) >> > 0564: >> > 0565: package_archive = deploy_dir_spdx / "packages" / ( >> package_doc.name + ".tar.zst") >> > 0566: with optional_tarfile(package_archive, >> archive_packaged) as archive: >> > *** 0567: package_files = add_package_files( >> > 0568: d, >> > 0569: package_doc, >> > 0570: spdx_package, >> > 0571: pkgdest / package, >> > File: >> '/srv/oe/build/conf/../../layers/openembedded-core/meta/classes/create-spdx.bbclass', >> lineno: 234, function: add_package_files >> > 0230: info.mtime = source_date_epoch >> > 0231: >> > 0232: archive.addfile(info, f) >> > 0233: >> > *** 0234: sha1 = bb.utils.sha1_file(filepath) >> > 0235: sha1s.append(sha1) >> > 0236: >> spdx_file.checksums.append(oe.spdx.SPDXChecksum( >> > 0237: algorithm="SHA1", >> > 0238: checksumValue=sha1, >> > File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 559, function: >> sha1_file >> > 0555: """ >> > 0556: Return the hex string representation of the SHA1 checksum >> of the filename >> > 0557: """ >> > 0558: import hashlib >> > *** 0559: return _hasher(hashlib.sha1(), filename) >> > 0560: >> > 0561:def sha384_file(filename): >> > 0562: """ >> > 0563: Return the hex string representation of the SHA384 >> checksum of the filename >> > File: '/srv/oe/bitbake/lib/bb/utils.py', lineno: 528, function: _hasher >> > 0524: >> > 0525:def _hasher(method, filename): >> > 0526: import mmap >> > 0527: >> > *** 0528: with open(filename, "rb") as f: >> > 0529: try: >> > 0530: with mmap.mmap(f.fileno(), 0, >> access=mmap.ACCESS_READ) as mm: >> > 0531: for chunk in iter(lambda: mm.read(8192), b''): >> > 0532: method.update(chunk) >> > Exception: FileNotFoundError: [Errno 2] No such file or directory: >> '/srv/oe/build/tmp-lmp/work/all-lmp-linux/alsa-topology-conf/1.2.5.1-r0/packages-split/alsa-topology-conf/CONTROL/control' >> >> We have other places where we've had to teach the code to ignore the >> control files and we'll probably have to do that here (grep for >> CONTROL). >> > > The snip that delete this files is in cleanupcontrol > https://git.yoctoproject.org/poky/tree/meta/classes/package_ipk.bbclass#n51 > > So these files are not shipped in the ipk package produced. > What I mean is those files are not part of the content deployed and it is a requirement of the packing standard used so tracking it the sbom doesn't make much sense imo. ar -xv deploy/ipk/all/alsa-topology-conf_1.2.5.1-r0_all.ipk x - debian-binary x - control.tar.gz x - data.tar.xz For and deb and ipk we have in deb_write_pkg and ipk_write_pkg: finally: cleanupcontrol(root) bb.utils.unlockfile(lf) For rpm we have in do_package_rpm: # rpm 4 creates various empty directories in _topdir, let's clean them up cleanupcmd = "rm -rf %s/BUILDROOT %s/SOURCES %s/SPECS %s/SRPMS" % (workdir, workdir, workdir, workdir) So all of them delete this package control content at the end but as do_create_spdx runs at the same time it can see and track this content as well. Thanks, Jose It breaks the do_create_spdx because it runs after the do_package > and the same happens for do_package_write_ipk so the two tasks > can run at the same time. > > >> Ideally we'd fix opkg (and maybe dpkg) to allow creation of packages >> with the files placed externally. >> > > I think I didn't quite understand this part, can you please explain it a > bit more? > > >> >> I have also wondered if we should create a specific hardlinked tree to >> handle this but that does come at an IO cost. >> >> I don't really want to add a dependency constraint like this for >> something we already handle differently elsewhere though. >> > > yeah, It's a bit ugly because it adds some tasks that don't even run (all > types of packages > in this case). Since I don't know what package types are chosen by the > user, I added all of them. > > Jose > > >> >> Cheers, >> >> Richard >> > > > -- > Best regards, > > José Quaresma > > > > -- Best regards, José Quaresma
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#168609): https://lists.openembedded.org/g/openembedded-core/message/168609 Mute This Topic: https://lists.openembedded.org/mt/92666627/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
