On Wed, Jul 10, 2024 at 6:18 AM Richard Purdie <[email protected]> wrote: > > On Fri, 2024-07-05 at 08:17 +0100, Richard Purdie via lists.openembedded.org > wrote: > > > On Wed, 2024-07-03 at 07:59 -0600, Joshua Watt via > > > lists.openembedded.org wrote: > > > > > This patch series add support for SPDX 3.0 and sets it as the > > > > > default. > > > > > Currently it is not possible to have SPDX 2.2 and SPDX 3.0 enabled at > > > > > the same time > > > > > > > > > > v2: Added tests and addressed feedback > > > > > v3: Fixed several oe-selftest and build failures > > > > > v4: Fixed silly typo mistake in staging.bbclass > > > > > v5: Reworked to make SPDX 3 output reproducible by default. Variables > > > > > that introduce non-reproducible output are documented as such. > > > > > > Thanks for working on this. The patches generally seem to working well > > > in testing but this does look related: > > > > > > https://autobuilder.yoctoproject.org/typhoon/#/builders/83/builds/7109/steps/23/logs/stdio > > > > > > Particularly these bits: > > > > > > AssertionError: 10 != 0 : Missing objects in the cache: > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/a6/eb/sstate:perf:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:a6eb451ebf8142b51d88b66b79ebbfc43019458d7e335e0a1b2a79e19cf3eed6_create_spdx.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/73/bd/sstate:perf:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:73bd840dbf3fb04c5adf5c09dd6f45d7f65dd1808bc4cf8c2f7a20ea551eaabd_create_package_spdx.tar.zst > > > > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/73/bd/sstate:perf:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:73bd840dbf3fb04c5adf5c09dd6f45d7f65dd1808bc4cf8c2f7a20ea551eaabd_create_package_spdx.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/31/ce/sstate:core-image-full-cmdline:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:31ce987a3b0b34b0d2bbacab96b4b9848f851caadd1f1fb1522d38c2b392c65d_create_image_sbom.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/2f/cf/sstate:core-image-sato-sdk:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:2fcfbfd93928c643c30f92b69cd17e19f4dd9136506e0bf1373a28883593d3a9_create_rootfs_spdx.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/1d/de/sstate:core-image-sato-sdk:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:1dded8f18500fb63356331c5e4f5eee5fc35c113398ec98ad7bed1d82c3f330d_create_image_spdx.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/6a/92/sstate:core-image-sato-sdk:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:6a924154d59877cfc4527602e72d0984c1fa7b685ac7f93fa8bee225a5de57e3_create_image_sbom.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/62/47/sstate:core-image-full-cmdline:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:624764dcea54752c3f6f3928f6719b440728eecd7516b85d80d37b305c56f530_create_rootfs_spdx.tar.zst > > > /srv/autobuilder/autobuilder.yocto.io/pub/sstate/7c/6f/sstate:core-image-full-cmdline:qemux86_64-poky-linux:1.0:r0:qemux86_64:12:7c6f212fe1a869ae8edc251f2f9c02e939a5f2b659fd77e9b4bb262eb60f6f5d_create_image_spdx.tar.zst > > I think I understand what might have happened here. I suspect we've had > test builds on the new test cluster which can write to the hash > equivalence server but not the sstate. I think this means entries were > added for artefacts which don't exist on sstate. > > In theory those should be created on new build runs but in practise I > suspect they're shadowed by other artefacts so newer builds probably > don't create them. That would at least explain how we end up where we > are.
There is also https://git.yoctoproject.org/poky/tree/meta/lib/oeqa/selftest/cases/sstatetests.py#n933 which I beleive is still accurate and probably why the images themselves show up; we'll need to adjust the exclusions to match the new SPDX tasks I added. I think '.*:do_create.*_spdx' would be OK? > > If I'm correct, if there are changes to the code (as we discussed on > yesterdays tech call), everything will rerun and assuming we run this > only on the main cluster, things should work out ok. > > I would be curious to understand how these things aren't being > recreated when missing though? I'm not sure. I can't really think of any reason in the SPDX code that it wouldn't recreate a missing sstate object; I'm not doing anything particularly strange with sstate > > I did also notice an interesting issue in that if you run a build with > PACKAGE_CLASSES = "package_rpm", then add deb/ipk, all the > do_collect_spdx_deps tasks and do_create_spdx tasks change hashes and > rerun. This isn't ideal as it effectively breaks our ability to turn > package backends on/off without rebuilds :(. Can we avoid that or at > least minimise things a bit more? Ya, that's a leftover from when I was trying to tie the SPDX packages to the produced package files. I had to give up thought because it's actually really difficult to figure out what .rpm and .deb files correspond to a PACKAGE (at least, I couldn't see an easy way to do it). We can just remove the dependency on do_package_write_* for now and figure out how to do it later. > > Cheers, > > Richard > > >
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#201734): https://lists.openembedded.org/g/openembedded-core/message/201734 Mute This Topic: https://lists.openembedded.org/mt/107019821/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
