Hi Yoann, Thanks for picking this up
> What value are you using? Especially in the last few days... I've seen much better results with a cap of 500/1000 than the default of 2000. > I'm not sure CVE_DB_RESULTS_PER_PAGE_MAX should be a bitbake variable (I > don't see a reason for it to be configurable). Can you simplify the code > a little by making it a constant in the code instead? We already have a > handfull of hardcoded limits from NVD. Well it is a valid request parameter for starters. This allows the users to control it from the configurations rather than hard coding to any limits someone sees fitting for their environment so more flexibility. Not setting the variable defaults to the limits recommended by NVD however my analysis shows that these days due to bulk updates on the NVD side the fetch has failed most of the time due to an "incomplete read" error. On Thu, Jun 25, 2026 at 4:43 PM Yoann Congal <[email protected]> wrote: > Hello, > > Thanks for looking at this! > > On Thu Jun 25, 2026 at 1:15 PM CEST, Awais Belal via > lists.openembedded.org wrote: > > From: Awais B <[email protected]> > > > > It is seen that during bulk updates on the NVD side the server > > struggles to keep up with the default/max of 2000 entries per > > page and we see a lot of incomplete read errors resulting in > > proper db sync failures most of the times. Lowering the per > > page value noticably increases the reliability of the process > > and hence should ideally be configurable. > > What value are you using? Especially in the last few days... > > > > > Signed-off-by: Awais B <[email protected]> > > --- > > meta/recipes-core/meta/cve-update-nvd2-native.bb | 14 ++++++++++++++ > > 1 file changed, 14 insertions(+) > > > > diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb > b/meta/recipes-core/meta/cve-update-nvd2-native.bb > > index 945bd1d927..5d8b76e62c 100644 > > --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb > > +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb > > @@ -34,6 +34,11 @@ CVE_DB_INCR_UPDATE_AGE_THRES ?= "10368000" > > # Number of attempts for each http query to nvd server before giving up > > CVE_DB_UPDATE_ATTEMPTS ?= "5" > > > > +# Maximum number of CVE records per API response. Default/max is 2000. > > +# Lowering this value can help avoid incomplete read errors during bulk > NVD updates. > > +CVE_DB_RESULTS_PER_PAGE ?= "" > > +CVE_DB_RESULTS_PER_PAGE_MAX ?= "2000" > > + > > CVE_CHECK_DB_DLDIR_FILE ?= > "${DL_DIR}/CVE_CHECK/${CVE_CHECK_DB_FILENAME}" > > CVE_CHECK_DB_DLDIR_LOCK ?= "${CVE_CHECK_DB_DLDIR_FILE}.lock" > > CVE_CHECK_DB_TEMP_FILE ?= "${CVE_CHECK_DB_FILE}.tmp" > > @@ -217,6 +222,15 @@ def update_db_file(db_tmp_file, d, database_time): > > api_key = d.getVar("NVDCVE_API_KEY") or None > > attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) > > > > + results_per_page = d.getVar("CVE_DB_RESULTS_PER_PAGE") > > + results_per_page_max = > int(d.getVar("CVE_DB_RESULTS_PER_PAGE_MAX")) > > I'm not sure CVE_DB_RESULTS_PER_PAGE_MAX should be a bitbake variable (I > don't see a reason for it to be configurable). Can you simplify the code > a little by making it a constant in the code instead? We already have a > handfull of hardcoded limits from NVD. > > > + if results_per_page: > > + results_per_page = int(results_per_page) > > + if results_per_page > results_per_page_max: > > + bb.warn("CVE_DB_RESULTS_PER_PAGE exceeds maximum of %d, > capping" % results_per_page_max) > > + results_per_page = results_per_page_max > > + req_args['resultsPerPage'] = results_per_page > > + > > # Recommended by NVD > > wait_time = 6 > > if api_key: > > > -- > Yoann Congal > Smile ECS > > -- Regards, Awais Belal
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#239556): https://lists.openembedded.org/g/openembedded-core/message/239556 Mute This Topic: https://lists.openembedded.org/mt/119971554/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
