Thank you Peter for debugging this. Could you dump us a log of one of your
typical runs to see what the errors are?
We might consider mirroring at some point.
Kind regards,
Marta
On Tue, Jul 11, 2023 at 8:37 AM Peter Marko via lists.openembedded.org
<peter.marko=siemens....@lists.openembedded.org> wrote:
> From: Peter Marko <peter.ma...@siemens.com>
>
> Last couple days it is not possible to update NVD DB as servers
> are returning lot of errors.
> Mostly "HTTP Error 503: Service Unavailable" is observed but
> sporadially also some others.
>
> Retrying helps in most cases, so extend retries to all errors.
>
> Additionally add sleep which is recommended by NVD between requests.
> These retries are already implemented between successful requests,
> but giving servers time between failed ones is important, too.
>
> Signed-off-by: Peter Marko <peter.ma...@siemens.com>
> ---
> meta/recipes-core/meta/cve-update-nvd2-native.bb | 11 ++++-------
> 1 file changed, 4 insertions(+), 7 deletions(-)
>
> diff --git a/meta/recipes-core/meta/cve-update-nvd2-native.bb
> b/meta/recipes-core/meta/cve-update-nvd2-native.bb
> index 4585126f73..a7392405e0 100644
> --- a/meta/recipes-core/meta/cve-update-nvd2-native.bb
> +++ b/meta/recipes-core/meta/cve-update-nvd2-native.bb
> @@ -119,6 +119,7 @@ def nvd_request_next(url, api_key, args):
> import urllib.parse
> import gzip
> import http
> + import time
>
> headers = {}
> if api_key:
> @@ -140,13 +141,9 @@ def nvd_request_next(url, api_key, args):
>
> r.close()
>
> - except UnicodeDecodeError:
> - # Received garbage, retry
> - bb.debug(2, "CVE database: received malformed data, retrying
> (request: %s)" %(full_request))
> - pass
> - except http.client.IncompleteRead:
> - # Read incomplete, let's try again
> - bb.debug(2, "CVE database: received incomplete data, retrying
> (request: %s)" %(full_request))
> + except Exception as e:
> + bb.debug(2, "CVE database: received error (%s), retrying
> (request: %s)" %(e, full_request))
> + time.sleep(6)
> pass
> else:
> return raw_data
> --
> 2.30.2
>
>
>
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#184136):
https://lists.openembedded.org/g/openembedded-core/message/184136
Mute This Topic: https://lists.openembedded.org/mt/100074006/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
