From: Mehmet Fide <[email protected]> The fetch task calls urllib.request.urlopen() with no timeout argument, so when an NVD endpoint accepts the TCP connection but stops sending data, the call blocks forever and the existing retry loop driven by CVE_DB_UPDATE_ATTEMPTS never gets a chance to run. We observed worker processes wedged for over an hour on a single recv() syscall before the build was killed manually.
Re-introduce the CVE_SOCKET_TIMEOUT variable (removed in commit d6d94eed1e "cve-update-nvd2-native: remove unused variable CVE_SOCKET_TIMEOUT" as it was a leftover from the JSON 1.1 feed) and plumb it through update_db_file() and nvd_request_next() so it is actually honoured by urlopen(). The default of 60 seconds matches the prior historical default; users behind slow proxies may raise it. With the timeout in place, a stalled NVD endpoint produces a clean exception, the retry loop runs, and after CVE_DB_UPDATE_ATTEMPTS failures the task returns False and the build falls back to the previously cached database (bb.warn, not a hard error). Signed-off-by: Mehmet Fide <[email protected]> --- meta/recipes-core/meta/cve-update-nvd2-native.bb | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb b/meta/recipes-core/meta/cve-update-nvd2-native.bb index 32a14a932b..271679b7bd 100644 --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb @@ -34,6 +34,12 @@ CVE_DB_INCR_UPDATE_AGE_THRES ?= "10368000" # Number of attempts for each http query to nvd server before giving up CVE_DB_UPDATE_ATTEMPTS ?= "5" +# Per-request socket timeout (seconds) for HTTP queries to the NVD server. +# Without this, urllib uses the global default (None) and a stalled connection +# can block the do_fetch task indefinitely, preventing the retry loop driven +# by CVE_DB_UPDATE_ATTEMPTS from ever running. +CVE_SOCKET_TIMEOUT ?= "60" + CVE_CHECK_DB_DLDIR_FILE ?= "${DL_DIR}/CVE_CHECK2/${CVE_CHECK_DB_FILENAME}" CVE_CHECK_DB_DLDIR_LOCK ?= "${CVE_CHECK_DB_DLDIR_FILE}.lock" CVE_CHECK_DB_TEMP_FILE ?= "${CVE_CHECK_DB_FILE}.tmp" @@ -134,7 +140,7 @@ def cleanup_db_download(db_file, db_tmp_file): def nvd_request_wait(attempt, min_wait): return min ( ( (2 * attempt) + min_wait ) , 30) -def nvd_request_next(url, attempts, api_key, args, min_wait): +def nvd_request_next(url, attempts, api_key, args, min_wait, timeout): """ Request next part of the NVD database NVD API documentation: https://nvd.nist.gov/developers/vulnerabilities @@ -153,7 +159,7 @@ def nvd_request_next(url, attempts, api_key, args, min_wait): for attempt in range(attempts): try: - r = urllib.request.urlopen(request) + r = urllib.request.urlopen(request, timeout=timeout) if (r.headers['content-encoding'] == 'gzip'): buf = r.read() @@ -216,6 +222,7 @@ def update_db_file(db_tmp_file, d, database_time): url = d.getVar("NVDCVE_URL") api_key = d.getVar("NVDCVE_API_KEY") or None attempts = int(d.getVar("CVE_DB_UPDATE_ATTEMPTS")) + timeout = int(d.getVar("CVE_SOCKET_TIMEOUT")) # Recommended by NVD wait_time = 6 @@ -224,7 +231,7 @@ def update_db_file(db_tmp_file, d, database_time): while True: req_args['startIndex'] = index - raw_data = nvd_request_next(url, attempts, api_key, req_args, wait_time) + raw_data = nvd_request_next(url, attempts, api_key, req_args, wait_time, timeout) if raw_data is None: # We haven't managed to download data return False -- 2.54.0
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#237595): https://lists.openembedded.org/g/openembedded-core/message/237595 Mute This Topic: https://lists.openembedded.org/mt/119493686/21656 Group Owner: [email protected] Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
