On Mon, Jun 29, 2026 at 4:55 PM Marko, Peter <[email protected]> wrote:
> > > > > *From:* Joshua Watt <[email protected]> > *Sent:* Monday, June 29, 2026 10:03 PM > *To:* Marko, Peter (FT D EU SK BFS1) <[email protected]> > *Cc:* [email protected] > *Subject:* Re: [OE-core] [PATCH 2/2] rootfs,spdx: handle removed packages > > > > > > > > On Mon, Jun 29, 2026 at 6:22 AM Marko, Peter <[email protected]> > wrote: > > > > > -----Original Message----- > > From: Joshua Watt <[email protected]> > > Sent: Friday, June 26, 2026 4:11 PM > > To: Marko, Peter (FT D EU SK BFS1) <[email protected]> > > Cc: [email protected] > > Subject: Re: [OE-core] [PATCH 2/2] rootfs,spdx: handle removed packages > > > > On Mon, Jun 8, 2026 at 7:56 AM Marko, Peter <[email protected]> > wrote: > > > > > > > > > > > > > -----Original Message----- > > > > From: Joshua Watt <[email protected]> > > > > Sent: Friday, June 5, 2026 4:16 PM > > > > To: Marko, Peter (FT D EU SK BFS1) <[email protected]> > > > > Cc: [email protected] > > > > Subject: Re: [OE-core] [PATCH 2/2] rootfs,spdx: handle removed > packages > > > > > > > > On Fri, Jun 5, 2026 at 6:05 AM Peter Marko via > lists.openembedded.org > > > > <[email protected]> wrote: > > > > > > > > > > From: Peter Marko <[email protected]> > > > > > > > > > > SPDX should not list packages which were removed from rootfs as > installed. > > > > > > > > > > The list of installed packages does not contain them directly, but > as > > > > > dependencies of other installed packages. > > > > > > > > > > Siwtch them to "other" to keep them in SPDX as part of the build > and > > > > > installation process. > > > > > > > > I agree the packages should be removed from the rootfs, but using the > > > > method described. > > > > > > > > Instead, the list of removed packages should be passed to > > > > collect_build_package_inputs(), and anything removed should skip > > > > having it's files added to files_by_hash (but, should still be added > > > > to build_deps, since it is a build dependency). > > > > > > I have tried several approaches here but all without success until > now.. > > > The problem is that these removed packages are "rdepends" and not > "depends". > > > Thus in collect_build_package_inputs they are not present yet. > > > And even if they would be, the rdepends would anyway add it back later. > > > > > > SPDX entries for rdepend clauses are added in expand_collection > function. > > > I obviously cannot remove the rdepends clause when the recipe sbom is > created. > > > > > > The best approach probably is to remove the rdepends when iterating > over sbomx > > in expand_collection. > > > Would you agree when I try to send a patch in that direction? > > > Or do you have any other suggestion? > > > > That's going to be tricky because you can't just remove the packages; > > you also need to remove any reference to the package (e.g. if they are > > in a relationship or somewhere else); otherwise the resulting document > > will not be fully linked and you'll get errors when creating the SBoM. > > For an example of what that would sort of look like, you can look at > > the "duplicate" function which is used to replace references to > > licenses and CreationInfo with equivalent ones to reduce the document > > size. > > > > But, this is going to complicate an otherwise pretty simple merging > > algorithm; I'm just not sure we can currently accurately express what > > is going on here (packages being uninstalled at install time), and > > until we have a good idea how to express it, I don't want to make it > > more confusing by doing something weird (because, then people rely on > > the weird behavior and get upset when it changes because we found a > > better solution). > > > > Little sad that there is currently no clear vision how to handle this > scenario in our classes. > In case you get a new idea, let me know and I can spend the effort to > convert that to a patch. > > I have created https://bugzilla.yoctoproject.org/show_bug.cgi?id=16336 > for this so it's not forgotten. > > > > I think we will need something like this: > https://github.com/spdx/spdx-3-model/pull/1297 but I will see if the SPDX > community has a better idea in the next few weeks. Basically the idea would > be to add a relationship that indicates that the package was uninstalled > from the rootfs, while preserving the runtime dependencies. > > > > That however does not help scarthgap/wrynose as it will be in spdx 3.1.0, > correct? > Correct, but if we can figure out it _will_ be done in 3.1, we can implement a similar mechanism in 3.0.1 for the older versions using extensions etc. I just don't want to make up something that isn't expressiable any other way than by custom extensions. > > > > Peter > > > > > > > > > > > > This will prevent that rootfs package from "containing" the files for > > > > removed packages, and break the linkage between the rootfs and the > > > > removed packages. > > > > > > > > I'm not exactly sure the reasoning behind changing the type the > > > > "other", but we should not be doing that. > > > > > > Well, "other" because there was no good value for that available in > enum > > (installation helper). > > > > > > > > > > > > > > > > > Signed-off-by: Peter Marko <[email protected]> > > > > > --- > > > > > meta/classes-recipe/create-spdx-image-3.0.bbclass | 7 +++++++ > > > > > meta/lib/oe/rootfs.py | 6 ++++++ > > > > > meta/lib/oe/sbom30.py | 8 +++++++- > > > > > meta/lib/oe/spdx30_tasks.py | 14 > +++++++++++++- > > > > > 4 files changed, 33 insertions(+), 2 deletions(-) > > > > > > > > > > diff --git a/meta/classes-recipe/create-spdx-image-3.0.bbclass > b/meta/classes- > > > > recipe/create-spdx-image-3.0.bbclass > > > > > index 15a91e90e2..dfbd2961b3 100644 > > > > > --- a/meta/classes-recipe/create-spdx-image-3.0.bbclass > > > > > +++ b/meta/classes-recipe/create-spdx-image-3.0.bbclass > > > > > @@ -6,6 +6,7 @@ > > > > > # SPDX image tasks > > > > > > > > > > SPDX_ROOTFS_PACKAGES = "${SPDXDIR}/rootfs-packages.json" > > > > > +SPDX_ROOTFS_REMOVED_PACKAGES = "${SPDXDIR}/rootfs-removed- > > > > packages.json" > > > > > SPDXIMAGEDEPLOYDIR = "${SPDXDIR}/image-deploy" > > > > > SPDXROOTFSDEPLOY = "${SPDXDIR}/rootfs-deploy" > > > > > > > > > > @@ -15,14 +16,20 @@ python spdx_collect_rootfs_packages() { > > > > > from oe.rootfs import image_list_installed_packages > > > > > > > > > > root_packages_file = Path(d.getVar("SPDX_ROOTFS_PACKAGES")) > > > > > + root_removed_packages_file = > > > > Path(d.getVar("SPDX_ROOTFS_REMOVED_PACKAGES")) > > > > > > > > > > packages = image_list_installed_packages(d) > > > > > if not packages: > > > > > packages = {} > > > > > > > > > > + removed_packages = (d.getVar("ROOTFS_REMOVED_PACKAGES") or > > > > "").split() > > > > > + > > > > > root_packages_file.parent.mkdir(parents=True, exist_ok=True) > > > > > with root_packages_file.open("w") as f: > > > > > json.dump(packages, f) > > > > > + > > > > > + with root_removed_packages_file.open("w") as f: > > > > > + json.dump(removed_packages, f) > > > > > } > > > > > ROOTFS_POSTUNINSTALL_COMMAND =+ > > "spdx_collect_rootfs_packages" > > > > > > > > > > diff --git a/meta/lib/oe/rootfs.py b/meta/lib/oe/rootfs.py > > > > > index 5eee48f587..b8830596ed 100644 > > > > > --- a/meta/lib/oe/rootfs.py > > > > > +++ b/meta/lib/oe/rootfs.py > > > > > @@ -261,10 +261,13 @@ class Rootfs(object, metaclass=ABCMeta): > > > > > > > > > > > > > > > def _uninstall_unneeded(self): > > > > > + removed_pkgs = set() > > > > > + > > > > > # Remove the run-postinsts package if no delayed > postinsts are found > > > > > delayed_postinsts = self._get_delayed_postinsts() > > > > > if delayed_postinsts is None: > > > > > if > > > > > os.path.exists(self.d.expand("${IMAGE_ROOTFS}${sysconfdir}/init.d/run- > > postinsts")) > > > > or > > > os.path.exists(self.d.expand("${IMAGE_ROOTFS}${systemd_system_unitdir}/run- > > > > postinsts.service")): > > > > > + removed_pkgs.add("run-postinsts") > > > > > self.pm.remove(["run-postinsts"]) > > > > > > > > > > image_rorfs = bb.utils.contains("IMAGE_FEATURES", > "read-only-rootfs", > > > > > @@ -285,6 +288,7 @@ class Rootfs(object, metaclass=ABCMeta): > > > > > # to be uninstalled or to be managed correctly > otherwise. > > > > > provider = > self.d.getVar("VIRTUAL-RUNTIME_update-alternatives") > > > > > pkgs_to_remove = sorted([pkg for pkg in > pkgs_installed if pkg in > > > > unneeded_pkgs], key=lambda x: x == provider) > > > > > + removed_pkgs.update(pkgs_to_remove) > > > > > > > > > > # update-alternatives provider is removed in its own > remove() > > > > > # call because all package managers do not guarantee > the packages > > > > > @@ -296,6 +300,8 @@ class Rootfs(object, metaclass=ABCMeta): > > > > > if len(pkgs_to_remove) > 0: > > > > > self.pm.remove([pkgs_to_remove[-1]], False) > > > > > > > > > > + self.d.setVar("ROOTFS_REMOVED_PACKAGES", " > > > > ".join(sorted(removed_pkgs))) > > > > > + > > > > > if delayed_postinsts: > > > > > self._save_postinsts() > > > > > if image_rorfs: > > > > > diff --git a/meta/lib/oe/sbom30.py b/meta/lib/oe/sbom30.py > > > > > index b379ff947c..4fa32266fa 100644 > > > > > --- a/meta/lib/oe/sbom30.py > > > > > +++ b/meta/lib/oe/sbom30.py > > > > > @@ -1122,7 +1122,7 @@ def find_by_spdxid(d, spdxid, *, > required=False): > > > > > return find_jsonld(d, *jsonld_hash_path(hash_id(spdxid)), > required=required) > > > > > > > > > > > > > > > -def create_sbom(d, name, root_elements, add_objectsets=[]): > > > > > +def create_sbom(d, name, root_elements, add_objectsets=[], > > > > removed_packages=[]): > > > > > objset = ObjectSet.new_objset(d, name) > > > > > > > > > > sbom = objset.add( > > > > > @@ -1142,6 +1142,12 @@ def create_sbom(d, name, root_elements, > > > > add_objectsets=[]): > > > > > + "\n ".join(sorted(list(missing_spdxids))) > > > > > ) > > > > > > > > > > + if removed_packages: > > > > > + for pkg in > objset.foreach_type(oe.spdx30.software_Package): > > > > > + if pkg.name in removed_packages and > pkg.software_primaryPurpose > > == > > > > oe.spdx30.software_SoftwarePurpose.install: > > > > > + pkg.software_primaryPurpose = > > > > oe.spdx30.software_SoftwarePurpose.other > > > > > + bb.note("Reclassified removed package %s SPDX > entry from install > > to > > > > other" % pkg.name) > > > > > + > > > > > # Filter out internal extensions from final SBoMs > > > > > objset.remove_internal_extensions() > > > > > > > > > > diff --git a/meta/lib/oe/spdx30_tasks.py > b/meta/lib/oe/spdx30_tasks.py > > > > > index 7cc46d579b..18c68f47de 100644 > > > > > --- a/meta/lib/oe/spdx30_tasks.py > > > > > +++ b/meta/lib/oe/spdx30_tasks.py > > > > > @@ -1532,6 +1532,7 @@ def create_image_sbom_spdx(d): > > > > > image_link_name = d.getVar("IMAGE_LINK_NAME") > > > > > imgdeploydir = Path(d.getVar("SPDXIMAGEDEPLOYDIR")) > > > > > machine = d.getVar("MACHINE") > > > > > + root_removed_packages_file = > > > > Path(d.getVar("SPDX_ROOTFS_REMOVED_PACKAGES")) > > > > > > > > > > spdx_path = imgdeploydir / (image_name + ".spdx.json") > > > > > > > > > > @@ -1553,7 +1554,18 @@ def create_image_sbom_spdx(d): > > > > > for o in image_objset.foreach_root(oe.spdx30.software_File): > > > > > root_elements.append(oe.sbom30.get_element_link_id(o)) > > > > > > > > > > - objset, sbom = oe.sbom30.create_sbom(d, image_name, > root_elements) > > > > > + try: > > > > > + with root_removed_packages_file.open("r") as f: > > > > > + removed_packages = json.load(f) > > > > > + except FileNotFoundError: > > > > > + removed_packages = [] > > > > > + > > > > > + objset, sbom = oe.sbom30.create_sbom( > > > > > + d, > > > > > + image_name, > > > > > + root_elements, > > > > > + removed_packages=removed_packages, > > > > > + ) > > > > > > > > > > # Set supplier on root elements if SPDX_IMAGE_SUPPLIER is > defined > > > > > supplier = objset.new_agent("SPDX_IMAGE_SUPPLIER", add=False) > > > > > > > > > > > > > > > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#239830): https://lists.openembedded.org/g/openembedded-core/message/239830 Mute This Topic: https://lists.openembedded.org/mt/119660891/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
