On Mon, Nov 10, 2025 at 11:42 PM Peter Marko via lists.openembedded.org <[email protected]> wrote: > > > > > -----Original Message----- > > From: Michael Halstead <[email protected]> > > Sent: Tuesday, November 11, 2025 6:31 > > To: Marko, Peter (FT D EU SK BFS1) <[email protected]> > > Cc: Marta Rybczynska <[email protected]>; Richard Purdie > > <[email protected]>; [email protected]; openembedded- > > [email protected]; [email protected] > > Subject: Re: [OE-core] [PATCH 5/5] cve-update: Keep mtime stamp in the > > database itself > > > > On Mon, Nov 10, 2025 at 7:31 AM Peter Marko via lists.openembedded.org > > <[email protected]> wrote: > > > > > > > > > > > > > -----Original Message----- > > > > From: Marta Rybczynska <[email protected]> > > > > Sent: Monday, November 10, 2025 16:29 > > > > To: Marko, Peter (FT D EU SK BFS1) <[email protected]> > > > > Cc: Richard Purdie <[email protected]>; > > > > [email protected]; [email protected]; > > > > [email protected] > > > > Subject: Re: [OE-core] [PATCH 5/5] cve-update: Keep mtime stamp in the > > > > database itself > > > > > > > > > > > > > > > > On Mon, Nov 10, 2025 at 4:18 PM Marko, Peter <[email protected] > > > > <mailto:[email protected]> > wrote: > > > > > > > > > > > > > > > > > > > > > -----Original Message----- > > > > > From: Marta Rybczynska <[email protected] > > > > <mailto:[email protected]> > > > > > > Sent: Monday, November 10, 2025 16:00 > > > > > To: Richard Purdie <[email protected] > > > > <mailto:[email protected]> > > > > > > Cc: [email protected] <mailto:[email protected]> ; > > > > [email protected] <mailto:openembedded- > > > > [email protected]> ; > > > > > [email protected] <mailto:[email protected]> ; Marko, Peter > > > > (FT D EU SK BFS1) > > > > > <[email protected] <mailto:[email protected]> > > > > > > Subject: Re: [OE-core] [PATCH 5/5] cve-update: Keep mtime stamp > > > > in > > > > the > > > > > database itself > > > > > > > > > > > > > > > > > > > > On Mon, Nov 10, 2025 at 3:16 PM Richard Purdie > > > > > <[email protected] > > > > <mailto:[email protected]> > > > > <mailto:[email protected] > > > > <mailto:[email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > On Mon, 2025-11-03 at 14:31 +0000, Niko Mauno via > > > > > lists.openembedded.org <http://lists.openembedded.org> > > > > <http://lists.openembedded.org> wrote: > > > > > > This should help to avoid problems that will occur if > > > > the > > > > modification > > > > > > time of database file itself is altered e.g. by > > > > unassociated > > > > > > process(es) on the file system which hosts the database > > > > file. > > > > > > > > > > > > Since this change updates the database structure by > > > > adding a new > > > > table, > > > > > > bump the 'minor' version number in database file names > > > > to enforce > > > > full > > > > > > database fetch. This should also iron out e.g. > > > > situation where the > > > > > > database might have inconspicuously omitted entries due > > > > to way > > > > in > > > > > which > > > > > > the mtime of database file itself was relied upon. > > > > > > > > > > > > Signed-off-by: Niko Mauno <[email protected] > > > > <mailto:[email protected]> > > > > > <mailto:[email protected] <mailto:[email protected]> > > > > > > > > > > > > --- > > > > > > meta/classes/cve-check.bbclass | 2 +- > > > > > > .../recipes-core/meta/cve-update-db-native.bb > > > > <http://cve-update- > > > > db-native.bb> <http://cve-update-db- > > > > > native.bb <http://native.bb> > | 3 + > > > > > > meta/recipes-core/meta/cve-update-native.inc | 59 > > > > ++++++++++++------- > > > > > > .../meta/cve-update-nvd2-native.bb > > > > <http://cve-update-nvd2- > > > > native.bb> <http://cve-update-nvd2-native.bb> > > > > > | 3 + > > > > > > 4 files changed, 46 insertions(+), 21 deletions(-) > > > > > > > > > > I am a bit worried about this since it takes what is a > > > > simple mtime > > > > > comparison and means that to check the database you now > > > > have to > > > > lock > > > > > and open it which is comparatively expensive. > > > > > > > > > > What kind of "unassociated processes" are updating the > > > > mtime? > > > > > > > > > > > > > > > > > > > > > > > > > I second Richard here. If we would really want to keep the date > > > > somewhere else > > > > > than the file > > > > > timestamp, we could create a simple file that stores it, what > > > > is way > > > > simpler and > > > > > doesn't require > > > > > to open the database just to see if you want to update it. > > > > > > > > > > I second Richard's question: in what kind of a situation have > > > > you seen an > > > > issue? > > > > > > > > The issue is on Yocto AB. > > > > It has not updated statistics for more than 5 months now. > > > > > > > > Looking at the logs, something is updating mtime very > > > > frequently... > > > > I don't know the AB infrastructure, but it looks like there is > > > > some filesystem > > > > crawler, maybe backup of sync process. > > > > On 2024-12-26 we set workers in the > > autobulder.yoctoproject.org/valkyrie cluster to run "touch > > /srv/autobuilder/valkyrie.yocto.io/current_sources/CVE_CHECK/*.db > > /srv/autobuilder/valkyrie.yocto.io/current_sources/CVE_CHECK2/*.db" > > every 3 minute to ensure no stale NFS cache existed for these files. > > This explains a lot. > Touching file while we rely on mtime is obviously breaking cve-check code. > > Note that stopping the touch service is not enough, at least not for LTS > branches, > because NVD2 code will update new CVEs only since the mtime timestamp. > Could you also delete the databases now so it can start fresh without 5 > months of missing data?
I've removed those files. > > Peter > > > That job was disabled for all builders on 2025-09-02. The performance > > testing workers were still running the job though. I've disabled it on > > those now. > > > > The issue of stale cached NFS data may return now. Something to keep an eye > > on. > > > > If we want to keep the timestamp in a separate file I suggest creating > > a new file named timestamp_$(date +%s) for each timestamp and deleting > > previous timestamp_ files. That should avoid most caching issues and > > keep overhead minimal. > > > > > > Or the nfs mount is plain broken there? > > > > > > > > Example is here: > > > > https://valkyrie.yocto.io/pub/non-release/20251109- > > > > 43/testresults/metrics/db-update/ > > > > NOTE: Current time: Sun Nov 9 07:17:52 2025; DB time: Sun Nov 9 > > > > 07:14:51 2025 > > > > NOTE: Current time: Sun Nov 9 07:19:22 2025; DB time: Sun Nov 9 > > > > 07:18:14 2025 > > > > 2 jobs are 90 seconds from each other, but something managed to > > > > change > > > > the timestamp in between and only 3.5 minutes after last change! > > > > > > > > > > > > > > > > > > > > The autobuilder was expected to force the complete download every day. > > > > Something > > > > has changed? The code does open the database after checking the > > timestamp, so > > > > frequent jobs couldn't cause it. I think that machine requires some > > > > monitoring > > to > > > > see > > > > what is actually changing those timestamps, because we do not know what > > > > it > > is > > > > also > > > > changing. > > > > > > 1 day doesn't beat the 3.5 minutes crawler... > > > This is probably AB issue only. > > > > > > And maybe other example: > > > https://valkyrie.yocto.io/pub/non-release/20251110-45/testresults/metrics/db- > > update/ > > > NOTE: Current time: Mon Nov 10 07:17:47 2025; DB time: Mon Nov 10 07:16:33 > > 2025 > > > NOTE: Current time: Mon Nov 10 07:19:17 2025; DB time: Mon Nov 10 07:16:33 > > 2025 > > > > > > This shows that the first job does not modify it (at least not in the > > > first 90 > > seconds). > > > I guess it fit the 3.5 minutes window of opportunity. > > > And my local build does not show any timestamp manipulation either. > > > > > > Peter > > > > > > > > > > > I haven't seen such an effect on my systems. > > > > > > > > Kind regards, > > > > Marta > > > > > > > > > > > > > > > > > > -- > > Michael Halstead > > Linux Foundation / Yocto Project > > Systems Operations Engineer > > > -- Michael Halstead Linux Foundation / Yocto Project Systems Operations Engineer
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#226152): https://lists.openembedded.org/g/openembedded-core/message/226152 Mute This Topic: https://lists.openembedded.org/mt/116098343/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
