Hello Paul,
On Thursday, March 5, 2026 at 5:22 PM, Benjamin Robin wrote:
> On Thursday, March 5, 2026 at 2:47 PM, Paul Barker wrote:
> > We would prefer not to override do_fetch for sbom-cve-check-update-*.bb.
> > We should be able to use the standard git fetcher here, with a hardcoded
> > SRCREV to allow offline parsing to succeed. A config fragment should
> > then be defined which enables the sbom-cve-check bbclass and sets the
> > srcrevs for the update recipes to ${AUTOREV}.
>
> Honestly, I've been considering the best approach for fetching the CVE
> databases. While using the Yocto internal fetcher is indeed cleaner, it
> raises a few questions:
>
> - Is it possible to implement updates at fixed intervals (e.g., every X
> hours)? If so, how could this be done?
> If this isn't feasible, it's not a major concern, having the latest
> updates is more important than performance.
>
> - Would there be any objections to updating the `RM_WORK_EXCLUDE`
> variable within the database update recipes to exclude the recipe
> itself? Unpacking the CVE database is quite slow, especially given its
> size (~3GB).
>
> - By retaining the unpacked databases, we could store the database index
> in the `$workdir`. This would avoid the need to recompute the database
> index each time, which is something we'd prefer to avoid.
>
> - However, it feels questionable to use an extracted Git repository from
> another recipe: My whole (new) idea on how to fix this looks wrong.
> I checked how `cve-update-nvd2-native.bb` handles this, the database
> is moved to the download directory. But if we do this, the database
> will still be unpacked for every analysis, which we try to avoid.
>
> My primary aim is to avoid extracting the database repeatedly for every
> build, and to be able to keep the database index somewhere.
I am proposing two solutions to address this issue:
- First RFC [1]: A refined version of the original solution using a
custom `do_fetch`. While performance on an NFS-mounted download directory
may not be optimal, this approach now includes a configurable variable to
specify the CVE database storage location.
- Second RFC [2]: An alternative approach leveraging BitBake’s internal
fetcher.
I prefer the first solution, as it appears cleaner to me. It supports
shallow cloning (depth of 1) and allows explicit control over update
intervals.
Once feedback is received, I will prepare a formal patch based on the
chosen solution.
> > Running sbom-cve-check offline should be supported, but manual config
> > may be needed to set an appropriate srcrev. We should provide an example
> > of this in the docs.
For fully offline use of `sbom-cve-check`, there is already a commit in
the main branch that introduces the `--disable-auto-updates` flag. I
plan to release this update for `sbom-cve-check` soon (within one or two
weeks) to integrate it into the OE-core class. If this timeline is not OK,
please let me know.
> I plan to write documentation (in yocto-docs) as soon as this series is
> merged :)
>
> > We should also be able to avoid setting do_sbom_cve_check[nostamp]. With
> > dependencies set correctly, this should only re-run if the image changes
> > or the cve database has been updated.
>
> I am going to fix that (at least try, see discussion above)!
[1]
https://lore.kernel.org/r/[email protected]
[2]
https://lore.kernel.org/r/[email protected]
Best regards,
--
Benjamin Robin, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#232702):
https://lists.openembedded.org/g/openembedded-core/message/232702
Mute This Topic: https://lists.openembedded.org/mt/118015491/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
